# calligrapher2025/calligrapher architecture

> Architecture diagram of the GitHub repository calligrapher2025/calligrapher, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/calligrapher2025/calligrapher
- Repository: https://github.com/calligrapher2025/calligrapher

## Overview

##### Project type and purpose
- This is an open-source machine-learning inference tool for “freestyle text image customization” called Calligrapher.
- It combines pretrained diffusion models (FLUX.1-Fill-dev), an image encoder (SigLIP), and custom model weights (Calligrapher) into a Python-based CLI and Gradio UI for batch or interactive inpainting of text regions in images.

##### File structure
Top-level directories and files:
- README.md, env.yml, requirements.txt
- path_dict.json – centralizes all dataset and model paths
- pipeline_calligrapher.py – core inference pipeline
- models/ – modules defining attention, projection, transformer and Calligrapher models
- infer_*.py – CLI scripts for self-reference and cross-reference modes
- gradio_demo*.py – interactive web UI scripts (basic + upload-mask)
- utils.py – shared utilities
- samples/ – example inputs/outputs
- docs/ – landing page, static assets & teaser/framework images
- gradio_save_dir, cli_save_dir, data_dir, gradio_temp_dir – runtime artifacts (configured via path_dict.json)

##### README for architecture clues
- Describes three customization modes: self-reference, cross-reference, non-text reference
- Emphasizes two interfaces (CLI batch vs. Gradio interactive)
- Shows a training framework image (style injection → diffusion learning)
- Lists external dependencies: PyTorch, CUDA, Huggingface Hub downloads for FLUX.1-Fill-dev, SigLIP, Calligrapher

##### Main components
a. User Interfaces
- CLI Scripts (infer_calligrapher_self_custom.py, infer_calligrapher_cross_custom.py)
- Gradio Web UI (gradio_demo.py, gradio_demo_upload_mask.py)
b. Inference Pipeline
- pipeline_calligrapher.py (orchestrates preprocessing, model invocation, postprocessing)
- utils.py (common I/O, mask handling)
c. Model Layer
- Base Diffusion Model (FLUX.1-Fill-dev)
- Image Encoder (SigLIP)
- Calligrapher Custom Model (calligrapher.bin loaded via projection_models.py, transformer_flux_inpainting.py, attention_processor.py)
d. Storage & Data
- Local Data Dir (path_dict.json → data_dir for test dataset)
- Model Weights Repositories (Huggingface Hub)
- Output Directories (cli_save_dir, gradio_save_dir, gradio_temp_dir)
- samples/ for demonstration files
e. External Services & Dependencies
- Huggingface Hub API for snapshot_download
- PyTorch runtime (GPU/CUDA)
- Gradio server for web UI

##### Relationships & data flow
- User → (CLI or Gradio UI) → pipeline_calligrapher
- pipeline_calligrapher invokes:
1. Load paths from path_dict.json
2. Download or load pretrained weights & encoder
3. Preprocess input image, mask & reference
4. Forward through SigLIP encoder → style embeddings
5. Inpainting via FLUX diffusion + style injection (Calligrapher modules)
6. Postprocess & save result
- Outputs written to designated save directories
- Gradio UI also serves frontend assets (docs/static) and may hold temp mask files

##### Architectural patterns & design principles
- Modular layering: UI layer, pipeline orchestrator, model implementations, utilities
- Separation of concerns: CLI vs. UI vs. core logic
- Config-driven design via path_dict.json
- Extensibility: new modes or models plugged into existing pipeline

##### Essential details to capture
- Distinct CLI vs. Gradio paths into the same pipeline
- Path configuration driving dataset/model discovery
- Sequence: user input → mask generation → style encoding → diffusion inpainting → saved artifacts
- External model dependencies & download mechanism

## Diagram (Mermaid)

```mermaid
flowchart TD
    User((User)):::external

    subgraph "User Interfaces"
        CLI1["infer_calligrapher_self_custom.py"]:::ui
        CLI2["infer_calligrapher_cross_custom.py"]:::ui
        GR1["gradio_demo.py"]:::ui
        GR2["gradio_demo_upload_mask.py"]:::ui
        UIassets["docs/static/"]:::ui
    end

    subgraph "Inference Pipeline"
        PIPE["pipeline_calligrapher.py"]:::pipeline
        UTILS["utils.py"]:::pipeline
        PATH["path_dict.json"]:::pipeline
    end

    subgraph "Model Layer"
        TFINP["models/transformer_flux_inpainting.py"]:::model
        ATT["models/attention_processor.py"]:::model
        PROJ["models/projection_models.py"]:::model
        CALL["models/calligrapher.py"]:::model
    end

    subgraph "Storage & Data"
        SAMPLES["samples/"]:::storage
        IMAGES["docs/static/images"]:::storage
    end

    subgraph "External Dependencies & Services"
        HF["Huggingface Hub API"]:::external
        PT["PyTorch/CUDA"]:::external
        GRserver["Gradio Server"]:::external
        REQ["requirements.txt"]:::external
        ENV["env.yml"]:::external
        README["README.md"]:::external
    end

    %% User to UI
    User --> CLI1
    User --> CLI2
    User --> GR1
    User --> GR2

    %% UI to Pipeline
    CLI1 -->|invokes| PIPE
    CLI2 -->|invokes| PIPE
    GR1 -->|invokes| PIPE
    GR2 -->|invokes| PIPE
    UIassets -->|serves assets to| GR1

    %% Gradio server
    GRserver -->|hosts| GR1
    GRserver -->|hosts| GR2

    %% Pipeline interactions
    PIPE -->|loads config| PATH
    PIPE -->|uses utils| UTILS
    SAMPLES -->|provides input| PIPE

    %% Pipeline to Model Layer
    PIPE -->|inpainting| TFINP
    PIPE -->|style injection| ATT
    PIPE -->|projection| PROJ
    PIPE -->|custom model| CALL

    %% External dependencies
    HF -->|downloads weights| PIPE
    PT -->|executes compute| PIPE

    %% Storage & Data usage
    PIPE -->|saves outputs| SAMPLES

    %% Click Events
    click CLI1 "https://github.com/calligrapher2025/calligrapher/blob/main/infer_calligrapher_self_custom.py"
    click CLI2 "https://github.com/calligrapher2025/calligrapher/blob/main/infer_calligrapher_cross_custom.py"
    click GR1 "https://github.com/calligrapher2025/calligrapher/blob/main/gradio_demo.py"
    click GR2 "https://github.com/calligrapher2025/calligrapher/blob/main/gradio_demo_upload_mask.py"
    click UIassets "https://github.com/calligrapher2025/calligrapher/tree/main/docs/static/"
    click PIPE "https://github.com/calligrapher2025/calligrapher/blob/main/pipeline_calligrapher.py"
    click UTILS "https://github.com/calligrapher2025/calligrapher/blob/main/utils.py"
    click PATH "https://github.com/calligrapher2025/calligrapher/blob/main/path_dict.json"
    click TFINP "https://github.com/calligrapher2025/calligrapher/blob/main/models/transformer_flux_inpainting.py"
    click ATT "https://github.com/calligrapher2025/calligrapher/blob/main/models/attention_processor.py"
    click PROJ "https://github.com/calligrapher2025/calligrapher/blob/main/models/projection_models.py"
    click CALL "https://github.com/calligrapher2025/calligrapher/blob/main/models/calligrapher.py"
    click SAMPLES "https://github.com/calligrapher2025/calligrapher/tree/main/samples/"
    click IMAGES "https://github.com/calligrapher2025/calligrapher/tree/main/docs/static/images"
    click REQ "https://github.com/calligrapher2025/calligrapher/blob/main/requirements.txt"
    click ENV "https://github.com/calligrapher2025/calligrapher/blob/main/env.yml"
    click README "https://github.com/calligrapher2025/calligrapher/blob/main/README.md"

    %% Styles
    classDef ui fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef pipeline fill:#90EE90,stroke:#000,stroke-width:1px
    classDef model fill:#FFA500,stroke:#000,stroke-width:1px
    classDef storage fill:#D3D3D3,stroke:#000,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px
```
