# swiftlatex/htmlizer architecture

> Architecture diagram of the GitHub repository swiftlatex/htmlizer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/swiftlatex/htmlizer
- Repository: https://github.com/swiftlatex/htmlizer

## Overview

##### Project type and purpose
- This is a standalone RESTful web service (“htmlizer”) whose sole purpose is to convert uploaded PDFs into HTML.
- Key indicators: the README calls it “A Restful Service Converting PDF to HTML,” there is an app.py/WSGI.py pair, and a worker.py that implies asynchronous task processing.

##### File structure
- Top-level files:
- app.py / WSGI.py – the web entrypoints serving the REST API (probably via Flask, Django or similar)
- worker.py – background process for off-line or delay-tolerant conversions
- htmlizer.py / pdf2htmlex/ – core conversion logic and any shims around the pdf2htmlex binary or library
- cleaner.py, string_utils.py – post-processing utilities for sanitizing or formatting the converted HTML
- requests_utils.py – helper code for making HTTP requests (possibly for callbacks or external API integration)
- config.py – central configuration (broker URLs, timeouts, paths to converters)
- Dockerfile / docker-compose.yml – containerization and orchestration of web + worker + supporting services
- test.py – unit or integration tests
- Notice there are no explicit database folders, so persistent storage is likely the file system, object store, or ephemeral only.

##### Main components
a. Client(s)
– Web or CLI clients that upload PDFs and request HTML.
b. API Gateway / Web Service
– app.py served via WSGI.py (Gunicorn/uwsgi). Accepts HTTP requests, handles authentication/validation, and routes to the conversion pipeline.
c. Task Queue / Message Broker
– e.g., Redis or RabbitMQ (referenced in docker-compose.yml) used by app.py to enqueue PDF conversion jobs.
d. Worker Process
– worker.py that pulls jobs from the broker, invokes htmlizer.py (which wraps pdf2htmlex), and then post-processes output via cleaner.py/string_utils.py.
e. Conversion Engine
– pdf2htmlex binary or library, plus compatibility.js/manifest, where the bulk of PDF → HTML transformation happens.
f. Storage / Result Delivery
– local file system or cloud storage for holding generated HTML until the client fetches it, or piping back over HTTP.
g. Utilities and Shared Libraries
– config.py for settings, requests_utils.py for any downstream callbacks, string_utils.py for text handling.

##### Relationships and interactions
- Client → HTTP POST to API Gateway →
- If synchronous: Gateway invokes conversion directly (htmlizer.py) and streams HTML back.
- If asynchronous: Gateway pushes job to Message Broker and returns a Job ID.
- Worker → consumes job → calls htmlizer.py → calls pdf2htmlex → passes output to cleaner/string_utils → stores HTML → (optional) notifies client via HTTP callback (requests_utils) or client polls status endpoint.
- All services run in containers orchestrated by docker-compose.

##### Architectural patterns and design principles
- Separation of concerns: web tier vs. worker tier
- Asynchronous processing with a message broker for scalability and reliability
- Utility modules abstracting configuration and HTTP calls
- Containerization for consistent deployments (Dockerfile + docker-compose)

##### Technologies and frameworks
- Python web framework (Flask/Django) behind WSGI
- Celery or similar for task queue (implied)
- Redis or RabbitMQ as broker
- pdf2htmlex converter
- Docker + Docker-Compose for deployment
- Standard Python libraries (requests, etc.)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Clients Layer
    subgraph "Clients"
        direction TB
        Client["Client\n(Web/CLI)"]:::external
    end

    %% API/Web Tier
    subgraph "API/Web Tier"
        direction TB
        API["API Gateway\n(app.py, WSGI.py)"]:::web
    end

    %% Message Broker
    subgraph "Message Broker"
        direction TB
        Broker["Message Broker\n(Redis/RabbitMQ)"]:::db
    end

    %% Worker & Conversion Tier
    subgraph "Worker/Conversion Tier"
        direction TB
        Worker["Worker Process\n(worker.py)"]:::worker
        Conversion["Conversion Engine\n(htmlizer.py, pdf2htmlex/)"]:::service
        PostProc["Post-Processing\n(cleaner.py, string_utils.py)"]:::util
    end

    %% Storage & Delivery
    subgraph "Storage/Delivery"
        direction TB
        Storage["Storage / Delivery\n(FileSystem/ObjectStore)"]:::db
    end

    %% Utilities & Deployment
    subgraph "Utilities & Deployment"
        direction TB
        Config["Configuration\n(config.py)"]:::util
        HTTPutils["HTTP Utilities\n(requests_utils.py)"]:::util
        Test["Test Suite\n(test.py)"]:::util
        Deploy["Containerization\n(Dockerfile, docker-compose.yml)"]:::infra
    end

    %% Flows
    Client -->|"HTTP POST (sync/async)"| API
    API -->|"HTTP (sync)"| Conversion
    API -->|"AMQP/Enqueue"| Broker
    Broker -->|"AMQP/Dequeue"| Worker
    Worker -->|"invoke"| Conversion
    Conversion -->|"output HTML"| PostProc
    PostProc -->|"store"| Storage
    Storage -->|"HTTP GET"| Client
    Worker -->|"HTTP Callback"| HTTPutils
    HTTPutils -->|"notify"| Client

    %% Configuration dependencies
    Config -.-> API
    Config -.-> Worker
    Config -.-> Conversion

    %% Click Events
    click API "https://github.com/swiftlatex/htmlizer/blob/master/app.py"
    click API "https://github.com/swiftlatex/htmlizer/blob/master/WSGI.py"
    click Worker "https://github.com/swiftlatex/htmlizer/blob/master/worker.py"
    click Conversion "https://github.com/swiftlatex/htmlizer/blob/master/htmlizer.py"
    click Conversion "https://github.com/swiftlatex/htmlizer/tree/master/pdf2htmlex/"
    click PostProc "https://github.com/swiftlatex/htmlizer/blob/master/cleaner.py"
    click PostProc "https://github.com/swiftlatex/htmlizer/blob/master/string_utils.py"
    click Broker "https://github.com/swiftlatex/htmlizer/blob/master/docker-compose.yml"
    click Config "https://github.com/swiftlatex/htmlizer/blob/master/config.py"
    click HTTPutils "https://github.com/swiftlatex/htmlizer/blob/master/requests_utils.py"
    click Test "https://github.com/swiftlatex/htmlizer/blob/master/test.py"
    click Deploy "https://github.com/swiftlatex/htmlizer/tree/master/Dockerfile"
    click Deploy "https://github.com/swiftlatex/htmlizer/blob/master/docker-compose.yml"

    %% Styles
    classDef external fill:#FFDDAA,stroke:#333,stroke-width:1px
    classDef web fill:#CDEBFF,stroke:#333,stroke-width:1px
    classDef db fill:#FFEBCC,stroke:#333,stroke-width:1px
    classDef worker fill:#E3FFC9,stroke:#333,stroke-width:1px
    classDef service fill:#F0E6FF,stroke:#333,stroke-width:1px
    classDef util fill:#FFEFFD,stroke:#333,stroke-width:1px
    classDef infra fill:#D0D0D0,stroke:#333,stroke-width:1px
```
