How andreagemelli/doc2graph works
Project Type and Purpose
• Doc2Graph is a Python-based, full-stack machine learning library for document understanding.
• It implements a task-agnostic graph neural network (GNN) pipeline to perform key-value detection, layout analysis, table detection, and related tasks on document images.
High-Level Architecture
At a glance, the system breaks down into these main components:
a. Data Ingestion & Preprocessing
• dataloader.py: downloads and reads raw document data (FUNSD, PAU, custom)
• preprocessing.py & utils.py: OCR/layout parsing and cleaning
b. Feature & Graph Construction
• feature_builder.py: computes node-level features (text embeddings, visual histograms, geometry)
• graph_builder.py: connects nodes based on proximity or k-NN, builds edge features (polar coords)
c. Model Definitions
• models/graphs.py: GNN backbone that unifies message passing, classification, link prediction heads
• models/unet: optional UNet visual encoder/decoder borrowed for pixel-level layout tasks
d. Training & Evaluation
• training/funsd.py & pau.py: dataset-specific training loops, loss computation, validation metrics
• training/utils.py: generic training helpers (checkpointing, logging)
• configs/*.yaml: hyperparameters for base pipeline, model variants (e2e, edge, gcn), and training settings
e. Inference & CLI
• main.py & inference.py: command-line interface orchestrating initialization, training, testing, inference
• paths.py: centralizes file paths for models, data, outputs
Data Flow & Component Interactions
1. User runs python -m doc2graph.main [--init | --train | --test | --inference].
2. Configuration loader reads YAML (base + model + train) → constructs a Config object.
4. Graph Builder connects nodes → yields adjacency lists, edge features.
5. Model (selected by --model) takes graph inputs → message-passing → outputs node classifications and edge predictions.
6. Trainer/Evaluator orchestrates forward/backward passes, logs metrics, saves checkpoints.
7. Inference module consumes trained weights → runs full pipeline on new docs → outputs annotated images + JSON of relationships.
Architectural Patterns & Design Principles
• Modular package structure separating data, models, training, and inference.
• Pipeline pattern: clear, linear dataflow from raw input to graph-based predictions.
• Config-driven design: YAML files define hyperparameters, enabling reproducibility.
• Use of external libraries: PyTorch for deep learning, DGL for graph operations, uv for environment and CLI orchestration.