# andreagemelli/doc2graph architecture

> Architecture diagram of the GitHub repository andreagemelli/doc2graph, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/andreagemelli/doc2graph
- Repository: https://github.com/andreagemelli/doc2graph

## Overview

##### Project Type and Purpose
- Doc2Graph is a Python-based, full-stack machine learning library for document understanding.
- It implements a task-agnostic graph neural network (GNN) pipeline to perform key-value detection, layout analysis, table detection, and related tasks on document images.

##### High-Level Architecture
At a glance, the system breaks down into these main components:
a. Data Ingestion & Preprocessing
- dataloader.py: downloads and reads raw document data (FUNSD, PAU, custom)
- preprocessing.py & utils.py: OCR/layout parsing and cleaning
b. Feature & Graph Construction
- feature_builder.py: computes node-level features (text embeddings, visual histograms, geometry)
- graph_builder.py: connects nodes based on proximity or k-NN, builds edge features (polar coords)
c. Model Definitions
- models/graphs.py: GNN backbone that unifies message passing, classification, link prediction heads
- models/unet: optional UNet visual encoder/decoder borrowed for pixel-level layout tasks
d. Training & Evaluation
- training/funsd.py & pau.py: dataset-specific training loops, loss computation, validation metrics
- training/utils.py: generic training helpers (checkpointing, logging)
- configs/*.yaml: hyperparameters for base pipeline, model variants (e2e, edge, gcn), and training settings
e. Inference & CLI
- main.py & inference.py: command-line interface orchestrating initialization, training, testing, inference
- paths.py: centralizes file paths for models, data, outputs

##### Data Flow & Component Interactions
1. User runs `python -m doc2graph.main [--init | --train | --test | --inference]`.
2. Configuration loader reads YAML (base + model + train) → constructs a `Config` object.
4. Graph Builder connects nodes → yields adjacency lists, edge features.
5. Model (selected by `--model`) takes graph inputs → message-passing → outputs node classifications and edge predictions.
6. Trainer/Evaluator orchestrates forward/backward passes, logs metrics, saves checkpoints.
7. Inference module consumes trained weights → runs full pipeline on new docs → outputs annotated images + JSON of relationships.

##### Architectural Patterns & Design Principles
- Modular package structure separating data, models, training, and inference.
- Pipeline pattern: clear, linear dataflow from raw input to graph-based predictions.
- Config-driven design: YAML files define hyperparameters, enabling reproducibility.
- Use of external libraries: PyTorch for deep learning, DGL for graph operations, uv for environment and CLI orchestration.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Data Sources
    FUNSD_PAU[("FUNSD/PAU Sources")]:::external
    DOWNLOAD[("Data Download")]:::external
    FUNSD_PAU -->|raw docs| DOWNLOAD

    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing"
        DATALOADER["Dataloader"]:::data
        PREPROCESS["Preprocessing"]:::process
        UTILS["Utils"]:::process
    end
    DOWNLOAD -->|downloads data| DATALOADER
    DATALOADER -->|raw docs| PREPROCESS
    PREPROCESS -->|parsed OCR/layout| UTILS

    %% Feature & Graph Construction
    subgraph "Feature & Graph Construction"
        FEATURE["Feature Builder"]:::feature
        GRAPH["Graph Builder"]:::feature
    end
    UTILS -->|cleaned data| FEATURE
    FEATURE -->|node features| GRAPH

    %% Model Definitions
    subgraph "Model Definitions"
        GNN["GNN Model (graphs.py)"]:::model
        subgraph "UNet Visual Encoder/Decoder"
            UNET_ENC["model.py"]:::model
            UNET_DEC["decoder.py"]:::model
        end
    end
    GRAPH -->|graph data| GNN
    GRAPH -->|visual features| UNET_ENC
    UNET_ENC -->|encoded features| UNET_DEC
    UNET_DEC -.->|optional branch| GNN

    %% Configuration Management
    CONFIG["Config Loader (YAML)"]:::config
    CONFIG -->|configs| DATALOADER
    CONFIG --> FEATURE
    CONFIG --> GRAPH
    CONFIG --> GNN

    %% Training & Evaluation
    subgraph "Training & Evaluation"
        TRAIN_FUNSD["Funsd Training"]:::train
        TRAIN_PAU["Pau Training"]:::train
        TRAIN_UTILS["Training Utils"]:::train
        CHECKPOINTS[("Checkpoint Store")]:::storage
    end
    GNN -->|model inputs| TRAIN_FUNSD
    GNN -->|model inputs| TRAIN_PAU
    TRAIN_FUNSD -->|loss & metrics| TRAIN_UTILS
    TRAIN_PAU -->|loss & metrics| TRAIN_UTILS
    TRAIN_UTILS -->|save weights| CHECKPOINTS

    %% Inference & CLI
    subgraph "Inference & CLI"
        MAIN_CLI["doc2graph/main.py"]:::cli
        ROOT_MAIN["main.py"]:::cli
        INFERENCE["Inference Module"]:::inference
        PATHS["Paths Manager"]:::inference
        OUTPUT_JSON[("Output JSON")]:::storage
        OUTPUT_IMG[("Annotated Images")]:::storage
    end
    ROOT_MAIN --> MAIN_CLI
    MAIN_CLI --> CONFIG
    MAIN_CLI --> DATALOADER
    MAIN_CLI --> INFERENCE
    PATHS --> INFERENCE
    INFERENCE -->|weights| CHECKPOINTS
    INFERENCE -->|new docs| DATALOADER
    INFERENCE -->|predictions| OUTPUT_JSON
    INFERENCE -->|visuals| OUTPUT_IMG

    %% Click Events
    click DATALOADER "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/dataloader.py"
    click DOWNLOAD "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/download.py"
    click PREPROCESS "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/preprocessing.py"
    click UTILS "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/utils.py"
    click FEATURE "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/feature_builder.py"
    click GRAPH "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/data/graph_builder.py"
    click GNN "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/models/graphs.py"
    click UNET_ENC "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/models/unet/model.py"
    click UNET_DEC "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/models/unet/decoder.py"
    click TRAIN_FUNSD "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/training/funsd.py"
    click TRAIN_PAU "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/training/pau.py"
    click TRAIN_UTILS "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/training/utils.py"
    click CONFIG "https://github.com/andreagemelli/doc2graph/blob/master/configs/base.yaml"
    click MAIN_CLI "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/main.py"
    click ROOT_MAIN "https://github.com/andreagemelli/doc2graph/blob/master/main.py"
    click PATHS "https://github.com/andreagemelli/doc2graph/blob/master/doc2graph/paths.py"

    %% Styles
    classDef external fill:#cccccc,stroke:#333,stroke-width:1px
    classDef data fill:#a6cee3,stroke:#333,stroke-width:1px
    classDef process fill:#1f78b4,stroke:#333,stroke-width:1px
    classDef feature fill:#b2df8a,stroke:#333,stroke-width:1px
    classDef model fill:#fb9a99,stroke:#333,stroke-width:1px
    classDef config fill:#cab2d6,stroke:#333,stroke-width:1px
    classDef train fill:#fdbf6f,stroke:#333,stroke-width:1px
    classDef inference fill:#ffff99,stroke:#333,stroke-width:1px
    classDef storage fill:#ffffff,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
    classDef cli fill:#b15928,stroke:#333,stroke-width:1px
```
