# vikparuchuri/surya architecture

> Architecture diagram of the GitHub repository vikparuchuri/surya, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vikparuchuri/surya
- Repository: https://github.com/vikparuchuri/surya

## Overview

##### Main Components
- Input & Preprocessing Layer: Represented by files in the “surya/input” directory as well as configuration (“surya/settings.py”). This module handles loading images, PDFs, and any preprocessing (e.g., deskewing, binarizing) before sending to OCR pipelines.
- Text / Line Detection Module: Found in “surya/detection” (and benchmark/detection.py). Here the system detects where text is located in the document image. Include components like “affinity,” “heatmap,” and post-processing utilities.
- Layout Analysis & Reading Order: Shown by the “surya/layout” module that handles detecting various document regions (tables, headers, images, text blocks) and organizing the detected text blocks into a coherent reading order.
- Text Recognition: Located in “surya/recognition,” this module performs OCR on the detected text regions and supports 90+ languages. Include the use of models, encoders/decoders, tokenizers, and postprocessing.
- Table Recognition: The “surya/table_rec” directory captures the table extraction functionalities – it detects rows, columns, cells, and handles spanning items.
- LaTeX OCR Extraction: (in “surya/texify”) dedicated to recognizing equations and converting them into LaTeX-compatible output.
- Common Utilities & Debug Components: The “surya/common” (with submodules for shared utilities like S3 integration, polygon functions, etc.) and “surya/debug” modules support logging, drawing, and visual verification of predictions.
- Auxiliary Tools: A variety of command-line utilities (e.g., ocr_app.py, detect_layout.py, texify_app.py) and scripts in “surya/scripts” provide entry points for running prediction pipelines and even interactive streamlit interfaces.
- Benchmarking & Testing: The “benchmark” directory and “tests” folder provide frameworks for performance evaluation and quality tests. They resemble separate subsystems in the overall design.

##### Relationships & Data Flow
- The Input Handler loads the document (via surya/input) and, optionally, applies preliminary processing.
- The system then channels the detected regions to multiple specialized OCR modules:
– Recognition for generic text.
– Table Recognition for structured data extraction.
– LaTeX OCR for mathematical expressions.
- Throughout, helper libraries (in surya/common, surya/util, etc.) supply model configuration, data formatting, and I/O support.
- Results are aggregated and output (as JSON files with text, positions, and confidence scores) per the command usage illustrated in the README.
- Additionally, benchmarking tools and testing scripts validate performance and accuracy independently of production flow.

##### Architectural Patterns & Design Principles
- Modular Design: Each “feature” (detection, recognition, layout, table recognition, LaTeX OCR) is separated into its own module, allowing for independent development, testing, and potential compilation (as seen with the COMPILE_* environment variables).
- Extensibility: The use of common utilities and a dedicated scripts folder in “surya” indicates that the system is designed to be extensible both from development and operational perspectives.

##### Relevant Technologies & Frameworks
- Python 3.10+ and PyTorch (as primary frameworks for model training and inference).
- Streamlit for the interactive UI.
- Utilization of external libraries for model architectures (referencing Donut, EfficientViT, and timm).
- CI/CD pipelines (GitHub workflows) indicate automated testing and deployment.
- Environment variable driven configuration (e.g., TORCH_DEVICE, batch sizes, compile flags).

- Use clear, descriptive labels for each component/module: “Input Handler”, “Text Detection”, “Layout Analysis & Reading Order”, “Text Recognition”, “Table Recognition”, “LaTeX OCR”, “Common Utilities”, “Debug & Visualization Tools”, “CLI/Interactive App”, and “Benchmark & Testing Modules.”
- User/CLI input → Input Handler → (branching into) Detection, Layout Analysis.
- Detection output → Recognition module.
- Layout output → (branching into) Reading Order, Table Extraction.
- Detection/Layout outputs → LaTeX OCR (for equations, from pre-cropped regions).
- Data flow with key elements like JSON outputs (with bboxes, texts, and confidences) and include notes on performance tuning areas (batch sizes, GPU/CPU differentiation).
- External integrations (like S3 for weight downloads, or the hosted API in the README) as separate components linking to the main pipeline.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Global nodes
    User("User/CLI Input"):::user
    JSONOutput("Final JSON Output"):::output

    %% Subgraph - Input & Preprocessing Layer
    subgraph "Input & Preprocessing Layer"
        InputHandler("Input Handler"):::input
    end

    %% Subgraph - Core OCR Pipeline
    subgraph "Core OCR Pipeline"
        Detection("Text Detection"):::detection
        LayoutAnalysis("Layout Analysis & Reading Order"):::layout
        TextRecognition("Text Recognition"):::recognition
        TableRecognition("Table Recognition"):::table
        LatexOCR("LaTeX OCR Extraction"):::tex
    end

    %% Subgraph - Support & Auxiliary
    subgraph "Support & Auxiliary"
        CommonUtilities("Common Utilities & Debug Tools"):::common
        AuxTools("CLI/Interactive Apps"):::cli
        BenchmarkTesting("Benchmark & Testing Modules"):::benchmark
    end

    %% Data Flow Connections
    User --> InputHandler
    AuxTools --> InputHandler
    InputHandler --> Detection
    InputHandler --> LayoutAnalysis
    Detection --> TextRecognition
    Detection --> LatexOCR
    LayoutAnalysis --> TableRecognition
    LayoutAnalysis --> LatexOCR
    TextRecognition --> JSONOutput
    TableRecognition --> JSONOutput
    LatexOCR --> JSONOutput
    JSONOutput -->|"evaluated_by"| BenchmarkTesting

    %% Support Arrows from Core Modules to Common Utilities
    Detection -->|"supports"| CommonUtilities
    LayoutAnalysis -->|"supports"| CommonUtilities
    TextRecognition -->|"supports"| CommonUtilities
    TableRecognition -->|"supports"| CommonUtilities
    LatexOCR -->|"supports"| CommonUtilities

    %% Click Events for Input & Preprocessing Layer
    click InputHandler "https://github.com/vikparuchuri/surya/tree/master/surya/input"
    click InputHandler "https://github.com/vikparuchuri/surya/blob/master/surya/settings.py"

    %% Click Events for Text/Line Detection Module
    click Detection "https://github.com/vikparuchuri/surya/tree/master/surya/detection"
    click Detection "https://github.com/vikparuchuri/surya/blob/master/benchmark/detection.py"

    %% Click Event for Layout Analysis & Reading Order
    click LayoutAnalysis "https://github.com/vikparuchuri/surya/tree/master/surya/layout"

    %% Click Event for Text Recognition Module
    click TextRecognition "https://github.com/vikparuchuri/surya/tree/master/surya/recognition"

    %% Click Events for Table Recognition Module
    click TableRecognition "https://github.com/vikparuchuri/surya/tree/master/surya/table_rec"
    click TableRecognition "https://github.com/vikparuchuri/surya/blob/master/benchmark/table_recognition.py"

    %% Click Events for LaTeX OCR Extraction
    click LatexOCR "https://github.com/vikparuchuri/surya/tree/master/surya/texify"
    click LatexOCR "https://github.com/vikparuchuri/surya/blob/master/ocr_latex.py"
    click LatexOCR "https://github.com/vikparuchuri/surya/blob/master/surya/scripts/ocr_latex.py"

    %% Click Events for Common Utilities & Debug Components
    click CommonUtilities "https://github.com/vikparuchuri/surya/tree/master/surya/common"
    click CommonUtilities "https://github.com/vikparuchuri/surya/tree/master/surya/debug"

    %% Click Events for Auxiliary Tools
    click AuxTools "https://github.com/vikparuchuri/surya/tree/master/surya/scripts"
    click AuxTools "https://github.com/vikparuchuri/surya/blob/master/ocr_app.py"
    click AuxTools "https://github.com/vikparuchuri/surya/blob/master/detect_layout.py"
    click AuxTools "https://github.com/vikparuchuri/surya/blob/master/ocr_text.py"
    click AuxTools "https://github.com/vikparuchuri/surya/blob/master/texify_app.py"

    %% Click Events for Benchmarking & Testing Modules
    click BenchmarkTesting "https://github.com/vikparuchuri/surya/tree/master/benchmark"
    click BenchmarkTesting "https://github.com/vikparuchuri/surya/tree/master/tests"

    %% Styling Classes
    classDef user fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef input fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef detection fill:#DA70D6,stroke:#000,stroke-width:2px;
    classDef layout fill:#98FB98,stroke:#000,stroke-width:2px;
    classDef recognition fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef table fill:#66CDAA,stroke:#000,stroke-width:2px;
    classDef tex fill:#FFDAB9,stroke:#000,stroke-width:2px;
    classDef common fill:#D3D3D3,stroke:#000,stroke-width:2px;
    classDef cli fill:#87CEEB,stroke:#000,stroke-width:2px;
    classDef benchmark fill:#F0E68C,stroke:#000,stroke-width:2px;
    classDef output fill:#FFFACD,stroke:#000,stroke-width:2px;
```
