# mattj23/latex-compileservice architecture

> Architecture diagram of the GitHub repository mattj23/latex-compileservice, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mattj23/latex-compileservice
- Repository: https://github.com/mattj23/latex-compileservice

## Overview

##### Main Components
- Web/API Service – The Flask application (located in files like latex/api_routes.py and launched by wsgi.py or gunicorn). This component exposes REST endpoints for session creation, file uploads, template rendering, and session status checks.
- Celery Worker – Referenced in worker.py, this process performs asynchronous background tasks such as invoking LaTeX compilers, rendering templates (using Jinja2), and converting generated PDFs to images.
- Celery Scheduler – Started by scheduler.py, this process is responsible for periodic tasks like cleaning up expired sessions.
- Redis Service – Acts as both message broker and shared session store, with sessions information being stored via SessionManager in latex/session.py.
- Shared File Storage – Although implemented by the local filesystem (mounted via Docker volumes), it is critical since it is used by both the Flask app and the worker to exchange the source, compilation results, logs, etc.
- External Clients – These are the systems or users interacting with the service through HTTP/REST APIs (as demonstrated by example_client.py).

##### Relationships and Interactions
- External clients communicate with the Flask API over HTTP.
- The Flask web service handles session creation, accepts file/template uploads, and updates session states.
- Once a session is “finalized” by the client, its information is stored (in Redis) and on the shared file system.
- The Celery Worker picks up finalized tasks from Redis, accesses the shared file storage, runs the necessary LaTeX compilation and possibly the image conversion process, then posts back logs and compiled output.
- The Celery Scheduler periodically polls (or schedules a cleanup task via Celery beat) to remove expired sessions.
- The Redis instance is a central glue between these asynchronous processes, ensuring consistency in session data and coordinating workflow.

##### Architectural Patterns and Technologies
- Separation of concerns: Three processes (Flask web server, Celery worker, Celery scheduler) enforce clear boundaries.
- Asynchronous processing: The use of Celery decouples request handling from heavy processing tasks.
- Shared Volume: Both the API service and the worker share a common filesystem volume (which is critical due to the size of the TeX Live installation and for the exchange of temporary session data).
- Underlying tech stack: Python 3, Flask, Celery, Redis, Docker, various LaTeX compilers (pdflatex, xelatex, lualatex), Jinja2 templating engine, and utility tools like pdftoppm.

##### Final Design Focus
- This approach demonstrates a direct mapping from the code repository (including the clear division between “latex/” directory for core logic and the separate entrypoint scripts) to the deployed architecture.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Client
    ExternalClients["External Clients"]:::external

    %% Application Containers Subgraph
    subgraph "Application Containers"
        WebAPI["Flask Web/API Service"]:::app
        CeleryWorker["Celery Worker"]:::app
        CeleryScheduler["Celery Scheduler"]:::app
    end

    %% Infrastructure Subgraph
    subgraph "Infrastructure"
        Redis["Redis (Session Management)"]:::infra
        SharedFS["Shared File Storage"]:::infra
    end

    %% Connections
    ExternalClients -->|"HTTP/REST"| WebAPI
    WebAPI -->|"Read/WriteSessions"| Redis
    Redis -->|"SessionData"| WebAPI

    WebAPI -->|"FileUploads"| SharedFS

    Redis -->|"TaskPickup"| CeleryWorker
    CeleryWorker -->|"OutputLogs"| SharedFS

    CeleryScheduler -->|"CleanupTasks"| Redis
    CeleryScheduler -->|"CleanExpired"| SharedFS

    %% Click Events
    click WebAPI "https://github.com/mattj23/latex-compileservice/blob/master/latex/api_routes.py"
    click WebAPI "https://github.com/mattj23/latex-compileservice/blob/master/wsgi.py"
    click CeleryWorker "https://github.com/mattj23/latex-compileservice/blob/master/worker.py"
    click CeleryScheduler "https://github.com/mattj23/latex-compileservice/blob/master/scheduler.py"
    click Redis "https://github.com/mattj23/latex-compileservice/blob/master/latex/session.py"
    click SharedFS "https://github.com/mattj23/latex-compileservice/tree/master/latex/"
    click SharedFS "https://github.com/mattj23/latex-compileservice/blob/master/deployment/docker-hub/docker-compose.yaml"
    click SharedFS "https://github.com/mattj23/latex-compileservice/tree/master/deployment/docker-hub/Dockerfile"
    click ExternalClients "https://github.com/mattj23/latex-compileservice/blob/master/example_client.py"

    %% Styles
    classDef app fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef infra fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef external fill:#fff3cd,stroke:#856404,stroke-width:2px;
```
