# awasthiutk564/idp-420 architecture

> Architecture diagram of the GitHub repository awasthiutk564/idp-420, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/awasthiutk564/idp-420
- Repository: https://github.com/awasthiutk564/idp-420

## Overview

This is a hybrid document-extraction engine for PDFs and images. The principal workflow is file selection, extraction and classification, staged enrichment of a document graph, then formatted output. Its stages cover layout, text, equations, tables, metadata, semantics, chunking, fusion, validation, and output. PDF-library adapters and OCR/model components provide extraction capabilities; hierarchy and graph utilities support structured results. The README is empty, and main.py plus several stages are only partially excerpted, so some component links are intentionally omitted where invocation is not visible.

Main entry points: [`main.py`](https://github.com/awasthiutk564/idp-420/blob/main/main.py) (CLI application), [`utils/document_graph.py`](https://github.com/awasthiutk564/idp-420/blob/main/utils/document_graph.py) (Document graph), [`pipeline/pipeline.py`](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/pipeline.py) (Pipeline runner), [`utils/hierarchy_builder.py`](https://github.com/awasthiutk564/idp-420/blob/main/utils/hierarchy_builder.py) (Hierarchy builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Application"]
  node_main["CLI application<br/>[main.py]"]
  node_pipeline["Pipeline runner<br/>[pipeline.py]"]
end

subgraph group_ingest["Input and extraction"]
  node_adapters["PDF adapters"]
  node_image_adapter["Image extractor<br/>[image_extractor.py]"]
  node_processors["PDF processors"]
  node_preprocessing["Image preprocessing"]
end

subgraph group_enrich["Document enrichment"]
  node_stages["Extraction stages"]
  node_stage_math["Equation processing<br/>[stage_math.py]"]
  node_stage_layout["Layout processing<br/>[stage_layout.py]"]
  node_stage_text["Text processing<br/>[stage_text.py]"]
  node_stage_semantic["Semantic processing<br/>[stage_semantic.py]"]
  node_stage_tables["Table processing<br/>[stage_tables.py]"]
  node_stage_fusion["Extraction fusion<br/>[stage_fusion.py]"]
  node_stage_metadata["Metadata processing<br/>[stage_metadata.py]"]
  node_stage_chunks["Chunk generation<br/>[stage_chunks.py]"]
  node_stage_validation["Validation stage"]
  node_stage_output["Output stage<br/>[stage_output.py]"]
end

subgraph group_models["Models and classification"]
  node_classifiers["Document classifiers"]
  node_ocr["OCR models<br/>[ocr_model.py]"]
  node_layout_model["Layout model<br/>[layout_model.py]"]
  node_equation_model["Equation model<br/>[equation_model.py]"]
  node_table_model["Table model<br/>[table_model.py]"]
  node_chart_model["Chart model<br/>[chart_model.py]"]
end

subgraph group_structure["Document structure"]
  node_graph["Document graph<br/>[document_graph.py]"]
  node_hierarchy["Hierarchy builder"]
  node_formatter["Result formatter<br/>[formatter.py]"]
end

node_user(("Document user"))

node_user -->|"selects file"| node_main
node_main -->|"runs pipeline"| node_pipeline
node_main -->|"configures adapters"| node_adapters
node_main -->|"configures classifiers"| node_classifiers
node_main -->|"uses OCR"| node_ocr
node_main -->|"uses model"| node_layout_model
node_main -->|"uses model"| node_equation_model
node_main -->|"uses model"| node_table_model
node_main -->|"uses model"| node_chart_model
node_main -->|"uses preprocessing"| node_preprocessing
node_pipeline -->|"runs stages"| node_stages
node_pipeline -->|"creates graph"| node_graph
node_stage_math -->|"updates graph"| node_graph
node_stage_layout -->|"uses hierarchy"| node_hierarchy
node_stage_semantic -->|"uses hierarchy"| node_hierarchy
node_stage_text -->|"processes graph"| node_graph
node_stage_tables -->|"processes graph"| node_graph
node_stage_fusion -->|"processes graph"| node_graph
node_stage_metadata -->|"processes graph"| node_graph
node_stage_chunks -->|"processes graph"| node_graph
node_stage_output -->|"processes graph"| node_graph
node_stage_validation -->|"processes graph"| node_graph
node_main -->|"formats results"| node_formatter

click node_main "https://github.com/awasthiutk564/idp-420/blob/main/main.py"
click node_pipeline "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/pipeline.py"
click node_adapters "https://github.com/awasthiutk564/idp-420/tree/main/adapters"
click node_image_adapter "https://github.com/awasthiutk564/idp-420/blob/main/adapters/image_extractor.py"
click node_processors "https://github.com/awasthiutk564/idp-420/tree/main/processors"
click node_preprocessing "https://github.com/awasthiutk564/idp-420/blob/main/preprocess/image_preprocessor.py"
click node_classifiers "https://github.com/awasthiutk564/idp-420/tree/main/classifiers"
click node_ocr "https://github.com/awasthiutk564/idp-420/blob/main/models/ocr_model.py"
click node_layout_model "https://github.com/awasthiutk564/idp-420/blob/main/models/layout_model.py"
click node_equation_model "https://github.com/awasthiutk564/idp-420/blob/main/models/equation_model.py"
click node_table_model "https://github.com/awasthiutk564/idp-420/blob/main/models/table_model.py"
click node_chart_model "https://github.com/awasthiutk564/idp-420/blob/main/models/chart_model.py"
click node_stages "https://github.com/awasthiutk564/idp-420/tree/main/pipeline"
click node_stage_math "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_math.py"
click node_stage_layout "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_layout.py"
click node_stage_text "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_text.py"
click node_stage_semantic "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_semantic.py"
click node_stage_tables "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_tables.py"
click node_stage_fusion "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_fusion.py"
click node_stage_metadata "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_metadata.py"
click node_stage_chunks "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_chunks.py"
click node_stage_validation "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_validation.py"
click node_stage_output "https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_output.py"
click node_graph "https://github.com/awasthiutk564/idp-420/blob/main/utils/document_graph.py"
click node_hierarchy "https://github.com/awasthiutk564/idp-420/blob/main/utils/hierarchy_builder.py"
click node_formatter "https://github.com/awasthiutk564/idp-420/blob/main/utils/formatter.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_pipeline,node_user toneBlue
class node_adapters,node_image_adapter,node_processors,node_preprocessing toneAmber
class node_stages,node_stage_math,node_stage_layout,node_stage_text,node_stage_semantic,node_stage_tables,node_stage_fusion,node_stage_metadata,node_stage_chunks,node_stage_validation,node_stage_output toneMint
class node_classifiers,node_ocr,node_layout_model,node_equation_model,node_table_model,node_chart_model toneRose
class node_graph,node_hierarchy,node_formatter toneIndigo
```

## Components

### Application

- [CLI application](https://github.com/awasthiutk564/idp-420/blob/main/main.py): `main.py`
- [Pipeline runner](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/pipeline.py): `pipeline/pipeline.py`

### Input and extraction

- [PDF adapters](https://github.com/awasthiutk564/idp-420/tree/main/adapters): `adapters`
- [Image extractor](https://github.com/awasthiutk564/idp-420/blob/main/adapters/image_extractor.py): `adapters/image_extractor.py`
- [PDF processors](https://github.com/awasthiutk564/idp-420/tree/main/processors): `processors`
- [Image preprocessing](https://github.com/awasthiutk564/idp-420/blob/main/preprocess/image_preprocessor.py): `preprocess/image_preprocessor.py`

### Document enrichment

- [Extraction stages](https://github.com/awasthiutk564/idp-420/tree/main/pipeline): `pipeline`
- [Equation processing](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_math.py): `pipeline/stage_math.py`
- [Layout processing](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_layout.py): `pipeline/stage_layout.py`
- [Text processing](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_text.py): `pipeline/stage_text.py`
- [Semantic processing](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_semantic.py): `pipeline/stage_semantic.py`
- [Table processing](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_tables.py): `pipeline/stage_tables.py`
- [Extraction fusion](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_fusion.py): `pipeline/stage_fusion.py`
- [Metadata processing](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_metadata.py): `pipeline/stage_metadata.py`
- [Chunk generation](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_chunks.py): `pipeline/stage_chunks.py`
- [Validation stage](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_validation.py): `pipeline/stage_validation.py`
- [Output stage](https://github.com/awasthiutk564/idp-420/blob/main/pipeline/stage_output.py): `pipeline/stage_output.py`

### Models and classification

- [Document classifiers](https://github.com/awasthiutk564/idp-420/tree/main/classifiers): `classifiers`
- [OCR models](https://github.com/awasthiutk564/idp-420/blob/main/models/ocr_model.py): `models/ocr_model.py`
- [Layout model](https://github.com/awasthiutk564/idp-420/blob/main/models/layout_model.py): `models/layout_model.py`
- [Equation model](https://github.com/awasthiutk564/idp-420/blob/main/models/equation_model.py): `models/equation_model.py`
- [Table model](https://github.com/awasthiutk564/idp-420/blob/main/models/table_model.py): `models/table_model.py`
- [Chart model](https://github.com/awasthiutk564/idp-420/blob/main/models/chart_model.py): `models/chart_model.py`

### Document structure

- [Document graph](https://github.com/awasthiutk564/idp-420/blob/main/utils/document_graph.py): `utils/document_graph.py`
- [Hierarchy builder](https://github.com/awasthiutk564/idp-420/blob/main/utils/hierarchy_builder.py): `utils/hierarchy_builder.py`
- [Result formatter](https://github.com/awasthiutk564/idp-420/blob/main/utils/formatter.py): `utils/formatter.py`

### Other components

- **Document user**

## Connections

- Document user → CLI application: selects file (evidence: `main.py`)
- CLI application → Pipeline runner: runs pipeline (evidence: `main.py`)
- CLI application → PDF adapters: configures adapters (evidence: `main.py`)
- CLI application → Document classifiers: configures classifiers (evidence: `main.py`)
- CLI application → OCR models: uses OCR (evidence: `main.py`)
- CLI application → Layout model: uses model (evidence: `main.py`)
- CLI application → Equation model: uses model (evidence: `main.py`)
- CLI application → Table model: uses model (evidence: `main.py`)
- CLI application → Chart model: uses model (evidence: `main.py`)
- CLI application → Image preprocessing: uses preprocessing (evidence: `main.py`)
- Pipeline runner → Extraction stages: runs stages (evidence: `pipeline/pipeline.py`)
- Pipeline runner → Document graph: creates graph (evidence: `pipeline/pipeline.py`)
- Equation processing → Document graph: updates graph (evidence: `pipeline/stage_math.py`)
- Layout processing → Hierarchy builder: uses hierarchy (evidence: `pipeline/stage_layout.py`)
- Semantic processing → Hierarchy builder: uses hierarchy (evidence: `pipeline/stage_semantic.py`)
- Text processing → Document graph: processes graph (evidence: `pipeline/stage_text.py`)
- Table processing → Document graph: processes graph (evidence: `pipeline/stage_tables.py`)
- Extraction fusion → Document graph: processes graph (evidence: `pipeline/stage_fusion.py`)
- Metadata processing → Document graph: processes graph (evidence: `pipeline/stage_metadata.py`)
- Chunk generation → Document graph: processes graph (evidence: `pipeline/stage_chunks.py`)
- Output stage → Document graph: processes graph (evidence: `pipeline/stage_output.py`)
- Validation stage → Document graph: processes graph
- CLI application → Result formatter: formats results (evidence: `main.py`)
