# jubayerrafin/document-transformation architecture

> Architecture diagram of the GitHub repository jubayerrafin/document-transformation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jubayerrafin/document-transformation
- Repository: https://github.com/jubayerrafin/document-transformation

## Overview

Document Transformation is a local PDF extraction and review prototype. Its main workflow validates a PDF, runs one whole-document backend, normalizes and reconciles regions, renders previews and crops, then saves a run for inspection, comparison, annotation, and export. A separate shared-evaluation workflow imports normalized predictions and scores them against reviewed references. The graph follows documented workflow and sampled calls; several UI actions and backend details are represented only at the documented subsystem level because their implementation was not fully sampled.

Main entry points: [`app.py`](https://github.com/jubayerrafin/document-transformation/blob/main/app.py) (Review app), [`extraction/pipeline.py`](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/pipeline.py) (Extraction pipeline), [`evaluation/workflow.py`](https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/workflow.py) (Evaluation workflow), [`main.py`](https://github.com/jubayerrafin/document-transformation/blob/main/main.py) (CLI commands)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User interfaces"]
  node_web_app["Review app<br/>[app.py]"]
  node_cli["CLI commands<br/>[main.py]"]
  node_saved_run_index["Saved run discovery<br/>[saved_runs.py]"]
end

subgraph group_extraction["PDF extraction"]
  node_pipeline["Extraction pipeline<br/>[pipeline.py]"]
  node_native["PyMuPDF backend<br/>[native.py]"]
  node_docling["Docling backend<br/>[docling_backend.py]"]
  node_ocr_config["OCR configuration<br/>[ocr_config.py]"]
  node_ir_models["Shared document IR<br/>[models.py]"]
  node_reconcile["Overlap reconciliation<br/>[reconcile.py]"]
end

subgraph group_runs["Run artifacts"]
  node_run_files[("Run artifacts<br/>[pipeline.py]")]
  node_export["Run export<br/>[pipeline.py]"]
end

subgraph group_evaluation["Shared evaluation"]
  node_paddle_adapter["PaddleOCR adapter<br/>[adapt_paddleocr.py]"]
  node_eval_ui["Evaluation panel<br/>[ui.py]"]
  node_eval_workflow["Evaluation workflow<br/>[workflow.py]"]
  node_contracts["Evaluation contracts<br/>[contracts.py]"]
  node_scoring["Scoring metrics<br/>[evaluate.py]"]
  node_metric_helpers["Metric helpers<br/>[metrics.py]"]
  node_references[("Reviewed references")]
  node_predictions[("Prediction runs")]
  node_reports[("Evaluation reports")]
end

node_reviewer(("Reviewer"))
node_cli_user(("CLI user"))
node_pdf["PDF input"]
node_other_tool["Other OCR tool"]

node_reviewer -->|"uses"| node_web_app
node_cli_user -->|"runs"| node_cli
node_pdf -->|"uploads"| node_web_app
node_web_app -->|"starts extraction"| node_pipeline
node_cli -->|"starts extraction"| node_pipeline
node_web_app -->|"discovers runs"| node_saved_run_index
node_pipeline -->|"dispatches"| node_native
node_pipeline -->|"dispatches"| node_docling
node_docling -.->|"verifies OCR models"| node_ocr_config
node_native -->|"builds IR"| node_ir_models
node_docling -->|"builds IR"| node_ir_models
node_pipeline -->|"reconciles regions"| node_reconcile
node_reconcile -->|"validates IR"| node_ir_models
node_pipeline -->|"writes run"| node_run_files
node_pipeline -->|"provides ZIP export"| node_export
node_web_app -->|"downloads run"| node_export
node_web_app -->|"renders evaluation"| node_eval_ui
node_eval_ui -->|"requests evaluation"| node_eval_workflow
node_cli -->|"runs evaluation commands"| node_eval_workflow
node_other_tool -.->|"provides raw output"| node_paddle_adapter
node_paddle_adapter -->|"writes normalized prediction"| node_predictions
node_eval_workflow -->|"validates inputs"| node_contracts
node_eval_workflow -->|"scores collection"| node_scoring
node_scoring -->|"matches regions"| node_metric_helpers
node_eval_workflow -->|"reads references"| node_references
node_eval_workflow -->|"reads predictions"| node_predictions
node_eval_workflow -->|"writes reports"| node_reports
node_web_app -->|"inspects runs"| node_run_files

click node_web_app "https://github.com/jubayerrafin/document-transformation/blob/main/app.py"
click node_cli "https://github.com/jubayerrafin/document-transformation/blob/main/main.py"
click node_saved_run_index "https://github.com/jubayerrafin/document-transformation/blob/main/extraction/saved_runs.py"
click node_pipeline "https://github.com/jubayerrafin/document-transformation/blob/main/extraction/pipeline.py"
click node_native "https://github.com/jubayerrafin/document-transformation/blob/main/extraction/backends/native.py"
click node_docling "https://github.com/jubayerrafin/document-transformation/blob/main/extraction/backends/docling_backend.py"
click node_ocr_config "https://github.com/jubayerrafin/document-transformation/blob/main/extraction/ocr_config.py"
click node_ir_models "https://github.com/jubayerrafin/document-transformation/blob/main/extraction/models.py"
click node_reconcile "https://github.com/jubayerrafin/document-transformation/blob/main/extraction/reconcile.py"
click node_run_files "https://github.com/jubayerrafin/document-transformation/blob/main/extraction/pipeline.py"
click node_export "https://github.com/jubayerrafin/document-transformation/blob/main/extraction/pipeline.py"
click node_paddle_adapter "https://github.com/jubayerrafin/document-transformation/blob/main/adapt_paddleocr.py"
click node_eval_ui "https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/ui.py"
click node_eval_workflow "https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/workflow.py"
click node_contracts "https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/contracts.py"
click node_scoring "https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/evaluate.py"
click node_metric_helpers "https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/metrics.py"
click node_references "https://github.com/jubayerrafin/document-transformation/tree/main/references"
click node_predictions "https://github.com/jubayerrafin/document-transformation/tree/main/experiments"
click node_reports "https://github.com/jubayerrafin/document-transformation/tree/main/evaluation-results"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web_app,node_cli,node_saved_run_index,node_reviewer,node_cli_user toneBlue
class node_pipeline,node_native,node_docling,node_ocr_config,node_ir_models,node_reconcile toneAmber
class node_run_files,node_export toneMint
class node_paddle_adapter,node_eval_ui,node_eval_workflow,node_contracts,node_scoring,node_metric_helpers,node_references,node_predictions,node_reports toneRose
class node_pdf,node_other_tool toneIndigo
```

## Components

### User interfaces

- [Review app](https://github.com/jubayerrafin/document-transformation/blob/main/app.py): `app.py`
- [CLI commands](https://github.com/jubayerrafin/document-transformation/blob/main/main.py): `main.py`
- [Saved run discovery](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/saved_runs.py): `extraction/saved_runs.py`

### PDF extraction

- [Extraction pipeline](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/pipeline.py): `extraction/pipeline.py`
- [PyMuPDF backend](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/backends/native.py): `extraction/backends/native.py`
- [Docling backend](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/backends/docling_backend.py): `extraction/backends/docling_backend.py`
- [OCR configuration](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/ocr_config.py): `extraction/ocr_config.py`
- [Shared document IR](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/models.py): `extraction/models.py`
- [Overlap reconciliation](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/reconcile.py): `extraction/reconcile.py`

### Run artifacts

- [Run artifacts](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/pipeline.py): `extraction/pipeline.py`
- [Run export](https://github.com/jubayerrafin/document-transformation/blob/main/extraction/pipeline.py): `extraction/pipeline.py`

### Shared evaluation

- [PaddleOCR adapter](https://github.com/jubayerrafin/document-transformation/blob/main/adapt_paddleocr.py): `adapt_paddleocr.py`
- [Evaluation panel](https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/ui.py): `evaluation/ui.py`
- [Evaluation workflow](https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/workflow.py): `evaluation/workflow.py`
- [Evaluation contracts](https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/contracts.py): `evaluation/contracts.py`
- [Scoring metrics](https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/evaluate.py): `evaluation/evaluate.py`
- [Metric helpers](https://github.com/jubayerrafin/document-transformation/blob/main/evaluation/metrics.py): `evaluation/metrics.py`
- [Reviewed references](https://github.com/jubayerrafin/document-transformation/tree/main/references): `references`
- [Prediction runs](https://github.com/jubayerrafin/document-transformation/tree/main/experiments): `experiments`
- [Evaluation reports](https://github.com/jubayerrafin/document-transformation/tree/main/evaluation-results): `evaluation-results`

### Other components

- **Reviewer**
- **CLI user**
- **PDF input**
- **Other OCR tool**

## Connections

- Reviewer → Review app: uses
- CLI user → CLI commands: runs
- PDF input → Review app: uploads
- Review app → Extraction pipeline: starts extraction
- CLI commands → Extraction pipeline: starts extraction
- Review app → Saved run discovery: discovers runs
- Extraction pipeline → PyMuPDF backend: dispatches
- Extraction pipeline → Docling backend: dispatches
- Docling backend → OCR configuration: verifies OCR models
- PyMuPDF backend → Shared document IR: builds IR
- Docling backend → Shared document IR: builds IR
- Extraction pipeline → Overlap reconciliation: reconciles regions
- Overlap reconciliation → Shared document IR: validates IR
- Extraction pipeline → Run artifacts: writes run
- Extraction pipeline → Run export: provides ZIP export
- Review app → Run export: downloads run
- Review app → Evaluation panel: renders evaluation
- Evaluation panel → Evaluation workflow: requests evaluation
- CLI commands → Evaluation workflow: runs evaluation commands
- Other OCR tool → PaddleOCR adapter: provides raw output
- PaddleOCR adapter → Prediction runs: writes normalized prediction
- Evaluation workflow → Evaluation contracts: validates inputs
- Evaluation workflow → Scoring metrics: scores collection
- Scoring metrics → Metric helpers: matches regions
- Evaluation workflow → Reviewed references: reads references
- Evaluation workflow → Prediction runs: reads predictions
- Evaluation workflow → Evaluation reports: writes reports
- Review app → Run artifacts: inspects runs