How mattj23/latex-compileservice works
Main Components
• Web/API Service – The Flask application (located in files like latex/api_routes.py and launched by wsgi.py or gunicorn). This component exposes REST endpoints for session creation, file uploads, template rendering, and session status checks.
• Celery Worker – Referenced in worker.py, this process performs asynchronous background tasks such as invoking LaTeX compilers, rendering templates (using Jinja2), and converting generated PDFs to images.
• Celery Scheduler – Started by scheduler.py, this process is responsible for periodic tasks like cleaning up expired sessions.
• Redis Service – Acts as both message broker and shared session store, with sessions information being stored via SessionManager in latex/session.py.
• Shared File Storage – Although implemented by the local filesystem (mounted via Docker volumes), it is critical since it is used by both the Flask app and the worker to exchange the source, compilation results, logs, etc.
• External Clients – These are the systems or users interacting with the service through HTTP/REST APIs (as demonstrated by example_client.py).
Relationships and Interactions
• External clients communicate with the Flask API over HTTP.
• The Flask web service handles session creation, accepts file/template uploads, and updates session states.
• Once a session is “finalized” by the client, its information is stored (in Redis) and on the shared file system.
• The Celery Worker picks up finalized tasks from Redis, accesses the shared file storage, runs the necessary LaTeX compilation and possibly the image conversion process, then posts back logs and compiled output.
• The Celery Scheduler periodically polls (or schedules a cleanup task via Celery beat) to remove expired sessions.
• The Redis instance is a central glue between these asynchronous processes, ensuring consistency in session data and coordinating workflow.
Architectural Patterns and Technologies
• Separation of concerns: Three processes (Flask web server, Celery worker, Celery scheduler) enforce clear boundaries.
• Asynchronous processing: The use of Celery decouples request handling from heavy processing tasks.
• Shared Volume: Both the API service and the worker share a common filesystem volume (which is critical due to the size of the TeX Live installation and for the exchange of temporary session data).
• Underlying tech stack: Python 3, Flask, Celery, Redis, Docker, various LaTeX compilers (pdflatex, xelatex, lualatex), Jinja2 templating engine, and utility tools like pdftoppm.
Final Design Focus
• This approach demonstrates a direct mapping from the code repository (including the clear division between “latex/” directory for core logic and the separate entrypoint scripts) to the deployed architecture.