# rkstgr/papermake architecture

> Architecture diagram of the GitHub repository rkstgr/papermake, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rkstgr/papermake
- Repository: https://github.com/rkstgr/papermake

## Overview

##### Project Type and Purpose
- Papermake is a full-stack PDF generation service.
- It consists of:
- A Web UI (Next.js/React) for users to manage templates and request renders
- An HTTP API server (Rust) exposing template and render endpoints
- Background worker(s) to process heavy PDF generation jobs
- A core library (Rust) that drives Typst to compile templates into PDFs
- Storage layers (database for metadata, S3-compatible object store for PDFs)

##### File Structure and Architecture
Top-level crates/ directories:
- papermake/          Core library (template building, caching, rendering)
- papermake-registry/ Template versioning & storage abstraction (File, SQLite, Postgres, S3, TiKV)
- papermake-server/   HTTP API server + background worker harness
- papermake-worker/   Standalone worker binary (for horizontal scaling)
- webui/               Next.js front-end (React + TypeScript + Tailwind)
- docker-compose.yml   Definitions for MinIO (S3) and other services

##### Main Components
A. WebUI
- Next.js frontend (Pages, API client)
- Communicates via HTTP(S) to the API server
B. API Server
- Routes (templates, renders, batch renders, analytics, websockets)
- Services layer orchestrating registry & render logic
- Configuration (DB URL, S3 endpoint, Worker settings)
C. Template Registry
- CRUD operations on templates and versions
- Storage backends (SQLite/Postgres, filesystem, S3, etc.)
D. Background Worker(s)
- Pull jobs from a queue or invoked by server’s worker.rs
- Uses papermake core library + Typst to generate PDF
- Stores PDF in S3 and updates job status in DB
E. Core Library (papermake)
- TemplateBuilder, Schema validation, caching, render API
- Embedded Typst engine
F. Storage & Infrastructure
- Relational DB (SQLite, Postgres) for metadata & job states
- S3-compatible object store (MinIO) for storing rendered PDFs
- Docker Compose orchestrates these in local/dev

##### Relationships & Data Flows
1. User (Browser) → WebUI
- Template creation / listing / render requests
2. WebUI → API Server (HTTP)
- POST /api/templates → Template Registry
- POST /api/renders → Enqueue render job
3. API Server → Database
- Persist template metadata, versions, job records
4. API Server → Worker(s)
- Enqueue jobs (could be in-process or external message queue)
5. Worker → Core Library
- Load template & data → Render via Typst → Generate PDF bytes
6. Worker → S3 Storage
- Upload PDF file → Get object URL/path
7. Worker → Database
- Update job status (queued → in-progress → completed)
8. API Server → WebUI
- Expose job status & PDF download URLs via HTTP or WebSocket

##### Important Architectural Patterns
- Modular Rust crates (core / registry / server / worker)
- Separation of concerns: HTTP API vs. background processing
- Pluggable storage backends via traits (registry crate)
- Caching layer in core library for high-throughput rendering
- Optional horizontal scaling of worker binary

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Web UI" 
        direction TB
        UI_Browser["User Browser"]:::external
        WebUI["Next.js Frontend"]:::frontend
        UI_Browser -->|"Interacts via UI"| WebUI
    end

    %% API Server Layer
    subgraph "API Server (Rust)" 
        direction TB
        APIServer["papermake-server"]:::backend
        Routes["HTTP Routes"]:::backend
        Services["Business Services"]:::backend
        Config["Configuration"]:::backend
        Models["Data Models"]:::backend
        WorkerHarness["In-Process Worker Harness"]:::backend
        ErrorHandling["Error Handling"]:::backend

        APIServer --> Routes
        APIServer --> Services
        APIServer --> Config
        APIServer --> Models
        APIServer --> WorkerHarness
        APIServer --> ErrorHandling
    end

    WebUI -->|"HTTP POST /api/templates, /api/renders"| APIServer

    %% Worker Layer
    subgraph "Background Worker" 
        direction TB
        WorkerBinary["papermake-worker"]:::backend
        WorkerBinary -->|"Invokes core library"| CoreLib
    end

    APIServer -->|"Enqueue Job"| WorkerBinary
    WorkerHarness -->|"Execute Job"| CoreLib

    %% Libraries Layer
    subgraph "Shared Libraries"
        direction TB
        CoreLib["papermake (Core Library)"]:::library
        Registry["papermake-registry"]:::library

        CoreLib --> Registry
    end

    WorkerBinary --> CoreLib
    CoreLib --> Registry

    %% Storage Layer
    subgraph "Storage & Infrastructure"
        direction TB
        Database["Relational DB\n(SQLite/Postgres)"]:::storage
        S3Storage["S3-Compatible Object Store\n(MinIO)"]:::storage
        DockerCompose["docker-compose.yml"]:::storage

        Database --> DockerCompose
        S3Storage --> DockerCompose
    end

    APIServer -->|"SQL INSERT/UPDATE"| Database
    WorkerBinary -->|"Update Job Status"| Database
    WorkerBinary -->|"S3 PUT object"| S3Storage
    APIServer -->|"HTTP/WebSocket"| WebUI

    %% Click Events
    click WebUI "https://github.com/rkstgr/papermake/tree/main/webui/"
    click UI_Browser "https://github.com/rkstgr/papermake/tree/main/webui/src/app"
    click WebUI "https://github.com/rkstgr/papermake/tree/main/webui/src/app"
    click WebUI "https://github.com/rkstgr/papermake/blob/main/webui/src/lib/api.ts"
    click WebUI "https://github.com/rkstgr/papermake/blob/main/webui/src/lib/types.ts"
    click WebUI "https://github.com/rkstgr/papermake/tree/main/webui/src/ui/components"
    click WebUI "https://github.com/rkstgr/papermake/blob/main/webui/src/app/globals.css"
    click WebUI "https://github.com/rkstgr/papermake/blob/main/webui/tailwind.config.js"

    click APIServer "https://github.com/rkstgr/papermake/tree/main/crates/papermake-server/"
    click APIServer "https://github.com/rkstgr/papermake/blob/main/crates/papermake-server/src/main.rs"
    click Config "https://github.com/rkstgr/papermake/blob/main/crates/papermake-server/src/config.rs"
    click Routes "https://github.com/rkstgr/papermake/blob/main/crates/papermake-server/src/routes.rs"
    click Routes "https://github.com/rkstgr/papermake/tree/main/crates/papermake-server/src/routes/"
    click Models "https://github.com/rkstgr/papermake/blob/main/crates/papermake-server/src/models.rs"
    click Models "https://github.com/rkstgr/papermake/tree/main/crates/papermake-server/src/models/"
    click Services "https://github.com/rkstgr/papermake/blob/main/crates/papermake-server/src/services.rs"
    click Services "https://github.com/rkstgr/papermake/tree/main/crates/papermake-server/src/services/"
    click WorkerHarness "https://github.com/rkstgr/papermake/blob/main/crates/papermake-server/src/worker.rs"
    click ErrorHandling "https://github.com/rkstgr/papermake/blob/main/crates/papermake-server/src/error.rs"

    click WorkerBinary "https://github.com/rkstgr/papermake/blob/main/crates/papermake-worker/src/main.rs"
    click WorkerBinary "https://github.com/rkstgr/papermake/blob/main/crates/papermake-worker/Cargo.toml"

    click CoreLib "https://github.com/rkstgr/papermake/blob/main/crates/papermake/src/lib.rs"
    click CoreLib "https://github.com/rkstgr/papermake/blob/main/crates/papermake/src/template.rs"
    click CoreLib "https://github.com/rkstgr/papermake/blob/main/crates/papermake/src/schema.rs"
    click CoreLib "https://github.com/rkstgr/papermake/blob/main/crates/papermake/src/typst.rs"
    click CoreLib "https://github.com/rkstgr/papermake/blob/main/crates/papermake/src/render.rs"
    click CoreLib "https://github.com/rkstgr/papermake/blob/main/crates/papermake/src/cache.rs"
    click CoreLib "https://github.com/rkstgr/papermake/blob/main/crates/papermake/src/error.rs"
    click CoreLib "https://github.com/rkstgr/papermake/blob/main/crates/papermake/src/macros.rs"

    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/src/lib.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/src/registry.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/src/entities.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/src/storage/file_storage.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/src/storage/sqlite_storage.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/src/storage/postgres.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/src/storage/s3_storage.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/src/storage/tikv_storage.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/examples/manual_test.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/tests/integration_tests.rs"
    click Registry "https://github.com/rkstgr/papermake/blob/main/crates/papermake-registry/data/papermake.db"

    click DockerCompose "https://github.com/rkstgr/papermake/blob/main/docker-compose.yml"

    %% Styles
    classDef frontend fill:#D0E8FF,stroke:#0288D1,color:#000
    classDef backend fill:#E0F2F1,stroke:#00796B,color:#000
    classDef library fill:#FFF3E0,stroke:#FB8C00,color:#000
    classDef storage fill:#ECEFF1,stroke:#607D8B,color:#000
    classDef external fill:#FFFFFF,stroke:#90A4AE,color:#000
```
