# baudm/parseq architecture

> Architecture diagram of the GitHub repository baudm/parseq, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/baudm/parseq
- Repository: https://github.com/baudm/parseq

## Overview

##### Main Components
- • Data Handling & Preprocessing: Represent the data module (located under strhub/data), which includes dataset loading, augmentation, and transformation routines (e.g., “augment.py”, “dataset.py”, “utils.py”). Also, note the “Datasets.md” file and the tools provided to convert and prepare datasets.
- • Configuration Management: Indicate that the system is highly configurable via numerous YAML and configuration files (located in the “configs” directory). Separate subdirectories such as “charset”, “dataset”, “experiment”, “model”, and “tune” should be highlighted as they control experiments, model variants, dataset options, and hyperparameter tuning.
- • Training & Evaluation Pipeline: Include components responsible for training (train.py), evaluation (test.py), benchmarking (bench.py), and even automated tuning (tune.py). Highlight the use of Hydra for configuration management (as noted in the README) and integration with PyTorch Lightning.
- • Demo & Deployment: Since there is a Gradio demo (and the “read.py” script for inference) and Torch Hub integration (via hubconf.py), this should be included to show how the system is packaged for inference.

##### Demonstrate Relationships and Data Flow
- • Start with the Data Source:
- Data is ingested from image files and LMDB datasets (mentioned in “Datasets.md”).
- The Data Module (strhub/data) applies pre-processing (augmentations, tokenization) and passes the data to the model.
- • Model Module:
- The processed input data flows into one or more model instances (PARSeq or its variants) that reside under strhub/models.
- That model components are further subdivided into “backbone”, “attention/transformer”, “decoder”, and “language module” (as evident in files like “model.py”, “system.py”, “attention.py”, etc.).
- • Training/Evaluation Components:
- Evaluate/test.py and bench.py measure performance, showing output metrics and latency.
- • Tuning and Demo:
- Finally, note the inference pipeline used by “read.py” and the Torch Hub integration for quick demos.

##### Architectural Patterns & Design Principles
- • Modularity & Configurability: Emphasize how the project is structured around configuration files (Hydra configurations in “configs”) making it extensible and reusable.
- • Unified Model Design: Explain how a single Transformer block can be configured to run in different modes (context-free, context-aware, and iterative refinement) – a central insight of PARSeq.
- • Separation of Concerns: Differentiate the components for data handling, modeling, training, and evaluation.
- • Extensibility and Reusability: Multiple submodules under “strhub/models” show the approach for supporting a range of STR models with similar components.

##### Relevant Technologies, Frameworks, and Libraries
- • PyTorch (and PyTorch 2.0): As the core deep learning framework utilized.
- • PyTorch Lightning: Used for managing training, evaluation, and configuration.
- • Hydra: For configuration management, making the experiments highly configurable.
- • Ray Tune: For automated hyperparameter tuning.
- • Torch Hub: For packaging models for easy deployment.
- • Gradio: To offer an interactive demo for real-time evaluation.
- • Supporting Tools: The “tools” directory for dataset conversion and other utilities.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Preprocessing Layer
    subgraph "Data & Preprocessing"
        D1["Data Module (strhub/data)"]:::data
        D2["augment.py"]:::data
        D3["dataset.py"]:::data
        D4["module.py"]:::data
        D5["utils.py"]:::data
        D6["Datasets.md"]:::data
        D1 --> D2
        D1 --> D3
        D1 --> D4
        D1 --> D5
        D6 --> D1
    end

    %% Model Implementations Layer
    subgraph "Model Implementations"
        M0["Unified STR Models<br/>(PARSeq-based)"]:::model

        subgraph "ABINet"
            AB_Folder["ABINet Folder"]:::model
            A1["attention.py"]:::model
            A2["backbone.py"]:::model
            A3["model.py"]:::model
            A4["system.py"]:::model
            AB_Folder --> A1
            A1 --> A2
            A2 --> A3
            A3 --> A4
        end

        subgraph "CRNN"
            CRNN_Folder["CRNN Folder"]:::model
            C1["model.py"]:::model
            C2["system.py"]:::model
            CRNN_Folder --> C1
            C1 --> C2
        end

        subgraph "PARSeq"
            PARSeq_Folder["PARSeq Folder"]:::model
            P1["model.py"]:::model
            P2["modules.py"]:::model
            P3["system.py"]:::model
            PARSeq_Folder --> P1
            P1 --> P2
            P2 --> P3
        end

        subgraph "TRBA"
            TRBA_Folder["TRBA Folder"]:::model
            T1["feature_extraction.py"]:::model
            T2["model.py"]:::model
            T3["prediction.py"]:::model
            T4["system.py"]:::model
            T5["transformation.py"]:::model
            TRBA_Folder --> T1
            T1 --> T2
            T2 --> T3
            T3 --> T4
            T4 --> T5
        end

        subgraph "ViTSTR"
            ViTSTR_Folder["ViTSTR Folder"]:::model
            V1["model.py"]:::model
            V2["system.py"]:::model
            ViTSTR_Folder --> V1
            V1 --> V2
        end

        Shared["Shared Utilities"]:::model
        SU1["base.py"]:::model
        SU2["modules.py"]:::model
        SU3["utils.py"]:::model
        Shared --> SU1
        Shared --> SU2
        Shared --> SU3

        M0 --> AB_Folder
        M0 --> CRNN_Folder
        M0 --> PARSeq_Folder
        M0 --> TRBA_Folder
        M0 --> ViTSTR_Folder
    end

    %% Configuration Management Layer
    subgraph "Configuration Management"
        CM["Config Manager (configs)"]:::config
        CSet1["Charset Config"]:::config
        CSet2["Dataset Config"]:::config
        CSet3["Experiment Config"]:::config
        CSet4["Model Config"]:::config
        CSet5["Tune Config (tune.yaml)"]:::config
        CSet6["Main Config (main.yaml)"]:::config
        CM --> CSet1
        CM --> CSet2
        CM --> CSet3
        CM --> CSet4
        CM --> CSet5
        CM --> CSet6
    end

    %% Training & Evaluation Pipeline Layer
    subgraph "Training & Evaluation"
        Tun["Hyperparameter Tuning<br/>(tune.py)"]:::train
        Tr["Training Pipeline<br/>(train.py)"]:::train
        Te["Evaluation<br/>(test.py)"]:::train
        Be["Benchmark<br/>(bench.py)"]:::train
        Tun --> Tr
        Tr --> Te
        Te --> Be
    end

    %% Demo & Deployment Layer
    subgraph "Demo & Deployment"
        INode["Inference<br/>(read.py)"]:::demo
        THNode["Torch Hub<br/>(hubconf.py)"]:::demo
        GNode["Gradio Demo"]:::demo
        INode --> GNode
    end

    %% Inter-layer connections
    D1 --> M0
    D1 --> Tr
    CM --> M0
    CM --> Tr
    M0 --> INode

    %% Click Events for Data & Preprocessing
    click D1 "https://github.com/baudm/parseq/tree/main/strhub/data"
    click D2 "https://github.com/baudm/parseq/blob/main/strhub/data/augment.py"
    click D3 "https://github.com/baudm/parseq/blob/main/strhub/data/dataset.py"
    click D4 "https://github.com/baudm/parseq/blob/main/strhub/data/module.py"
    click D5 "https://github.com/baudm/parseq/blob/main/strhub/data/utils.py"
    click D6 "https://github.com/baudm/parseq/blob/main/Datasets.md"

    %% Click Events for Model Implementations - ABINet
    click AB_Folder "https://github.com/baudm/parseq/tree/main/strhub/models/abinet"
    click A1 "https://github.com/baudm/parseq/blob/main/strhub/models/abinet/attention.py"
    click A2 "https://github.com/baudm/parseq/blob/main/strhub/models/abinet/backbone.py"
    click A3 "https://github.com/baudm/parseq/blob/main/strhub/models/abinet/model.py"
    click A4 "https://github.com/baudm/parseq/blob/main/strhub/models/abinet/system.py"
    
    %% Click Events for Model Implementations - CRNN
    click CRNN_Folder "https://github.com/baudm/parseq/tree/main/strhub/models/crnn"
    click C1 "https://github.com/baudm/parseq/blob/main/strhub/models/crnn/model.py"
    click C2 "https://github.com/baudm/parseq/blob/main/strhub/models/crnn/system.py"
    
    %% Click Events for Model Implementations - PARSeq
    click PARSeq_Folder "https://github.com/baudm/parseq/tree/main/strhub/models/parseq"
    click P1 "https://github.com/baudm/parseq/blob/main/strhub/models/parseq/model.py"
    click P2 "https://github.com/baudm/parseq/blob/main/strhub/models/parseq/modules.py"
    click P3 "https://github.com/baudm/parseq/blob/main/strhub/models/parseq/system.py"
    
    %% Click Events for Model Implementations - TRBA
    click TRBA_Folder "https://github.com/baudm/parseq/tree/main/strhub/models/trba"
    click T1 "https://github.com/baudm/parseq/blob/main/strhub/models/trba/feature_extraction.py"
    click T2 "https://github.com/baudm/parseq/blob/main/strhub/models/trba/model.py"
    click T3 "https://github.com/baudm/parseq/blob/main/strhub/models/trba/prediction.py"
    click T4 "https://github.com/baudm/parseq/blob/main/strhub/models/trba/system.py"
    click T5 "https://github.com/baudm/parseq/blob/main/strhub/models/trba/transformation.py"
    
    %% Click Events for Model Implementations - ViTSTR
    click ViTSTR_Folder "https://github.com/baudm/parseq/tree/main/strhub/models/vitstr"
    click V1 "https://github.com/baudm/parseq/blob/main/strhub/models/vitstr/model.py"
    click V2 "https://github.com/baudm/parseq/blob/main/strhub/models/vitstr/system.py"
    
    %% Click Events for Shared Utilities
    click SU1 "https://github.com/baudm/parseq/blob/main/strhub/models/base.py"
    click SU2 "https://github.com/baudm/parseq/blob/main/strhub/models/modules.py"
    click SU3 "https://github.com/baudm/parseq/blob/main/strhub/models/utils.py"
    
    %% Click Events for Configuration Management
    click CM "https://github.com/baudm/parseq/tree/main/configs"
    click CSet1 "https://github.com/baudm/parseq/tree/main/configs/charset"
    click CSet2 "https://github.com/baudm/parseq/tree/main/configs/dataset"
    click CSet3 "https://github.com/baudm/parseq/tree/main/configs/experiment"
    click CSet4 "https://github.com/baudm/parseq/tree/main/configs/model"
    click CSet5 "https://github.com/baudm/parseq/blob/main/configs/tune.yaml"
    click CSet6 "https://github.com/baudm/parseq/blob/main/configs/main.yaml"
    
    %% Click Events for Training & Evaluation Pipeline
    click Tun "https://github.com/baudm/parseq/blob/main/tune.py"
    click Tr "https://github.com/baudm/parseq/blob/main/train.py"
    click Te "https://github.com/baudm/parseq/blob/main/test.py"
    click Be "https://github.com/baudm/parseq/blob/main/bench.py"

    %% Click Events for Demo & Deployment
    click INode "https://github.com/baudm/parseq/blob/main/read.py"
    click THNode "https://github.com/baudm/parseq/blob/main/hubconf.py"

    %% Styles
    classDef data fill:#a8e6a3,stroke:#333,stroke-width:1.5px;
    classDef model fill:#a3b8e6,stroke:#333,stroke-width:1.5px;
    classDef config fill:#ffdf99,stroke:#333,stroke-width:1.5px;
    classDef train fill:#ffb3b3,stroke:#333,stroke-width:1.5px;
    classDef demo fill:#d0f0c0,stroke:#333,stroke-width:1.5px;
```
