# datalab-to/surya architecture

> Architecture diagram of the GitHub repository datalab-to/surya, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/datalab-to/surya
- Repository: https://github.com/datalab-to/surya

## Overview

##### Project Type and Purpose
- Surya is an open-source, modular document OCR toolkit (Python library + CLI + optional interactive Streamlit apps) that provides pipelines for:
- Text detection
- Multilingual OCR
- Layout analysis & reading order
- Table recognition
- LaTeX equation OCR
- It is not a monolithic web app; rather a library with command-line scripts (`surya_detect`, `surya_ocr`, `surya_layout`, etc), Python API (`Predictor` classes), and optional Streamlit UIs.

##### High-Level File Structure & Architectural Layers
- Top-level:
- `surya/` – main Python package, subdivided by task (detection, recognition, layout, table_rec, ocr_error, common utilities, debug aids)
- CLI wrappers & convenience scripts: `detect_layout.py`, `ocr_app.py`, `texify_app.py`, plus `surya/scripts/` for Streamlit entrypoints
- `benchmark/` – benchmarking harness for each task
- `static/` – fonts/images for documentation & apps
- CI/CD: `.github/workflows/*`
- Tests: `tests/`
- Config: `surya/settings.py`, `pyproject.toml`, `pytest.ini`

A. Input Layer
- Supported inputs: image files, PDFs, Word docs, PowerPoints
- Preprocessing: loaders in `surya/input/*`
B. Core Task Modules (each as a separate “micro-pipeline”)
1. Detection Predictor
- `surya/detection/`
- Model: semantic segmentation (EfficientViT/Craft), heatmap → bboxes
2. Recognition Predictor
- `surya/recognition/`
- Models: Donut-derived encoder/decoder for line-level OCR
3. Layout Predictor & Reading Order
- `surya/layout/`
- Models: encoder/decoder for classifying regions, assigning read positions
4. TableRec Predictor
- `surya/table_rec/`
- Models: encoder/decoder for rows/cols/cells
5. Texify Predictor (LaTeX OCR)
- `surya/common/donut` + `surya/texify.py`
C. Shared Utilities
- `surya/common/` – model loaders, config readers, S3 weight downloader, polygon & bbox utils
- `surya/logging.py`, `surya/settings.py`
D. CLI & API Layer
- Python entrypoints (`surya/scripts/…`, console_scripts in `pyproject.toml`)
- User calls pipeline:
Input → Task Predictor → JSON + optional annotated images
E. Optional Interactive GUI
- Streamlit apps (`surya/scripts/streamlit_app.py`, `texify_gui`)
- Uses same Predictor classes under the hood
F. Benchmarking & CI/CD
- `benchmark/` modules run pipelines at scale, produce charts
- GitHub Actions for CI, benchmarks, CLA, publishing

##### Relationships and Data Flow
- branches to [Recognition] for text OCR
- branches to [Layout] for region classification & reading order
- branches to [TableRec] for table cell extraction
- branches to [Texify] for LaTeX if equations are cropped
- Each Predictor uses shared utilities (`common`) and model configs, downloads weights from S3 as needed
- CLI scripts orchestrate these calls; Streamlit UIs wrap the same pipelines

##### Architectural Patterns & Design Principles
- Modular monolith / plugin-style: each OCR task is a self-contained subpackage
- Pipeline pattern: sequential processing stages, with clear inputs/outputs (JSON schemas in `schema.py`)
- Shared core utilities for DRY (config loading, logging, bbox math)
- Config-driven: `surya/settings.py` + environment-variable overrides
- Optional compilation flags for JIT compiling PyTorch models

##### Technologies, Frameworks, External Services
- Python 3.10+, PyTorch, HuggingFace Transformers / Donut, timm, EfficientViT
- Streamlit for interactive UIs
- S3 (via `surya/common/s3.py`) for model weight hosting
- CI/CD: GitHub Actions
- Packaging: Poetry (pyproject.toml)
- Testing: pytest

##### Essential Elements to Capture
- All five task modules as separate components
- Shared common utilities & config layer
- CLI entrypoints vs Python API
- Streamlit GUI as an alternative front end
- Benchmarking harness & CI/CD pipeline

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input Layer
    subgraph "Input Layer"
        IL1["Loaders"]:::input
        IL2["Processing"]:::input
    end

    %% Core OCR Modules
    subgraph "Core OCR Modules"
        DP["Detection Predictor"]:::core
        RP["Recognition Predictor"]:::core
        LP["Layout Predictor"]:::core
        TP["TableRec Predictor"]:::core
        XP["Texify Predictor"]:::core
    end

    %% Shared Utilities & Config
    subgraph "Shared Utilities & Config"
        CU1["common/load.py"]:::util
        CU2["common/predictor.py"]:::util
        CU3["common/s3.py"]:::util
        CU4["common/polygon.py"]:::util
        CU5["common/util.py"]:::util
        CU6["config (settings.py)"]:::util
        CU7["schema utils"]:::util
        LG["logging.py"]:::util
    end

    %% CLI/API Layer
    subgraph "CLI/API & GUI"
        CLI1["detect_layout.py"]:::cli
        CLI2["detect_text.py"]:::cli
        CLI3["ocr_app.py"]:::cli
        CLI4["ocr_text.py"]:::cli
        CLI5["table_recognition.py"]:::cli
        S1["streamlit_app.py"]:::cli
        S2["run_streamlit_app.py"]:::cli
        T1["texify_app.py"]:::cli
    end

    %% Benchmarking & CI/CD
    subgraph "Benchmarking"
        B1["benchmark/detection.py"]:::bench
        B2["benchmark/recognition.py"]:::bench
        B3["benchmark/layout.py"]:::bench
        B4["benchmark/table_recognition.py"]:::bench
        B5["benchmark/texify.py"]:::bench
        B6["benchmark/utils"]:::bench
    end

    subgraph "CI/CD"
        CI1["ci.yml"]:::bench
        CI2["benchmarks.yml"]:::bench
        CI3["cla.yml"]:::bench
        CI4["publish.yml"]:::bench
        CI5["scripts.yml"]:::bench
    end

    %% Data Flows
    IL1 -->|"images/PDF/DOCX"| DP
    IL2 --> DP
    DP -->|"bboxes"| RP
    DP --> LP
    DP --> TP
    DP --> XP
    RP --> CU1
    LP --> CU1
    TP --> CU1
    XP --> CU1

    CLI1 --> DP
    CLI2 --> RP
    CLI3 --> DP
    CLI4 --> RP
    CLI5 --> TP
    S1 --> DP
    S1 --> RP
    S1 --> LP
    T1 --> XP

    B1 --> DP
    B2 --> RP
    B3 --> LP
    B4 --> TP
    B5 --> XP

    %% Click Events - Input
    click IL1 "https://github.com/datalab-to/surya/blob/master/surya/input/load.py"
    click IL2 "https://github.com/datalab-to/surya/blob/master/surya/input/processing.py"

    %% Click Events - Detection Predictor Files
    click DP "https://github.com/datalab-to/surya/tree/master/surya/detection/"
    click DP "https://github.com/datalab-to/surya/blob/master/surya/detection/loader.py"
    click DP "https://github.com/datalab-to/surya/blob/master/surya/detection/model/encoderdecoder.py"
    click DP "https://github.com/datalab-to/surya/blob/master/surya/detection/heatmap.py"
    click DP "https://github.com/datalab-to/surya/blob/master/surya/detection/processor.py"
    click DP "https://github.com/datalab-to/surya/blob/master/surya/detection/schema.py"

    %% Click Events - Recognition Predictor Files
    click RP "https://github.com/datalab-to/surya/blob/master/surya/recognition/loader.py"
    click RP "https://github.com/datalab-to/surya/blob/master/surya/recognition/cache.py"
    click RP "https://github.com/datalab-to/surya/blob/master/surya/recognition/postprocessing.py"
    click RP "https://github.com/datalab-to/surya/blob/master/surya/recognition/schema.py"

    %% Click Events - Layout Predictor Files
    click LP "https://github.com/datalab-to/surya/blob/master/surya/layout/loader.py"
    click LP "https://github.com/datalab-to/surya/blob/master/surya/layout/model/encoderdecoder.py"
    click LP "https://github.com/datalab-to/surya/blob/master/surya/layout/model/encoder.py"
    click LP "https://github.com/datalab-to/surya/blob/master/surya/layout/model/decoder.py"
    click LP "https://github.com/datalab-to/surya/blob/master/surya/layout/schema.py"
    click LP "https://github.com/datalab-to/surya/blob/master/surya/layout/util.py"

    %% Click Events - TableRec Predictor Files
    click TP "https://github.com/datalab-to/surya/blob/master/surya/table_rec/loader.py"
    click TP "https://github.com/datalab-to/surya/blob/master/surya/table_rec/model/encoderdecoder.py"
    click TP "https://github.com/datalab-to/surya/blob/master/surya/table_rec/model/encoder.py"
    click TP "https://github.com/datalab-to/surya/blob/master/surya/table_rec/model/decoder.py"
    click TP "https://github.com/datalab-to/surya/blob/master/surya/table_rec/processor.py"
    click TP "https://github.com/datalab-to/surya/blob/master/surya/table_rec/schema.py"

    %% Click Events - Texify Predictor Files
    click XP "https://github.com/datalab-to/surya/blob/master/surya/common/donut/encoder.py"
    click XP "https://github.com/datalab-to/surya/blob/master/surya/common/donut/processor.py"
    click XP "https://github.com/datalab-to/surya/blob/master/surya/scripts/ocr_latex.py"
    click XP "https://github.com/datalab-to/surya/blob/master/surya/scripts/texify_app.py"

    %% Click Events - Shared Utilities & Config
    click CU1 "https://github.com/datalab-to/surya/blob/master/surya/common/load.py"
    click CU2 "https://github.com/datalab-to/surya/blob/master/surya/common/predictor.py"
    click CU3 "https://github.com/datalab-to/surya/blob/master/surya/common/s3.py"
    click CU4 "https://github.com/datalab-to/surya/blob/master/surya/common/polygon.py"
    click CU5 "https://github.com/datalab-to/surya/blob/master/surya/common/util.py"
    click CU6 "https://github.com/datalab-to/surya/blob/master/surya/settings.py"
    click CU7 "https://github.com/datalab-to/surya/blob/master/surya/common/surya/processor/schema.py"
    click LG "https://github.com/datalab-to/surya/blob/master/surya/logging.py"

    %% Click Events - CLI/API & GUI
    click CLI1 "https://github.com/datalab-to/surya/blob/master/detect_layout.py"
    click CLI2 "https://github.com/datalab-to/surya/blob/master/detect_text.py"
    click CLI3 "https://github.com/datalab-to/surya/blob/master/ocr_app.py"
    click CLI4 "https://github.com/datalab-to/surya/blob/master/ocr_text.py"
    click CLI5 "https://github.com/datalab-to/surya/blob/master/table_recognition.py"
    click S1 "https://github.com/datalab-to/surya/blob/master/surya/scripts/streamlit_app.py"
    click S2 "https://github.com/datalab-to/surya/blob/master/surya/scripts/run_streamlit_app.py"
    click T1 "https://github.com/datalab-to/surya/blob/master/texify_app.py"

    %% Click Events - Benchmarking
    click B1 "https://github.com/datalab-to/surya/blob/master/benchmark/detection.py"
    click B2 "https://github.com/datalab-to/surya/blob/master/benchmark/recognition.py"
    click B3 "https://github.com/datalab-to/surya/blob/master/benchmark/layout.py"
    click B4 "https://github.com/datalab-to/surya/blob/master/benchmark/table_recognition.py"
    click B5 "https://github.com/datalab-to/surya/blob/master/benchmark/texify.py"
    click B6 "https://github.com/datalab-to/surya/tree/master/benchmark/utils/"

    %% Click Events - CI/CD
    click CI1 "https://github.com/datalab-to/surya/blob/master/.github/workflows/ci.yml"
    click CI2 "https://github.com/datalab-to/surya/blob/master/.github/workflows/benchmarks.yml"
    click CI3 "https://github.com/datalab-to/surya/blob/master/.github/workflows/cla.yml"
    click CI4 "https://github.com/datalab-to/surya/blob/master/.github/workflows/publish.yml"
    click CI5 "https://github.com/datalab-to/surya/blob/master/.github/workflows/scripts.yml"

    %% Styles
    classDef input fill:#b7e4c7,stroke:#2d6a4f,color:#081c15;
    classDef core fill:#a0c4ff,stroke:#4361ee,color:#03045e;
    classDef util fill:#ffd6a5,stroke:#e85d04,color:#6a040f;
    classDef cli fill:#e0c3fc,stroke:#9d4edd,color:#240046;
    classDef bench fill:#d3d3d3,stroke:#6c757d,color:#212529;
```
