How jubayerrafin/document-transformation works
Document Transformation is a local PDF extraction and review prototype. Its main workflow validates a PDF, runs one whole-document backend, normalizes and reconciles regions, renders previews and crops, then saves a run for inspection, comparison, annotation, and export. A separate shared-evaluation workflow imports normalized predictions and scores them against reviewed references. The graph follows documented workflow and sampled calls; several UI actions and backend details are represented only at the documented subsystem level because their implementation was not fully sampled.
Main entry points: app.py Review app · pipeline.py Extraction pipeline · workflow.py Evaluation workflow · main.py CLI commands
Components 24
User interfaces
- Review app
app.py - CLI commands
main.py - Saved run discovery
extraction/saved_runs.py
PDF extraction
- Extraction pipeline
extraction/pipeline.py - PyMuPDF backend
extraction/backends/native.py - Docling backend
extraction/backends/docling_backend.py - OCR configuration
extraction/ocr_config.py - Shared document IR
extraction/models.py - Overlap reconciliation
extraction/reconcile.py
Run artifacts
- Run artifacts
extraction/pipeline.py - Run export
extraction/pipeline.py
Shared evaluation
- PaddleOCR adapter
adapt_paddleocr.py - Evaluation panel
evaluation/ui.py - Evaluation workflow
evaluation/workflow.py - Evaluation contracts
evaluation/contracts.py - Scoring metrics
evaluation/evaluate.py - Metric helpers
evaluation/metrics.py - Reviewed references
references - Prediction runs
experiments - Evaluation reports
evaluation-results
Other components
- Reviewer
- CLI user
- PDF input
- Other OCR tool
Connections 28
- Reviewer → Review app: uses
- CLI user → CLI commands: runs
- PDF input → Review app: uploads
- Review app → Extraction pipeline: starts extraction
- CLI commands → Extraction pipeline: starts extraction
- Review app → Saved run discovery: discovers runs
- Extraction pipeline → PyMuPDF backend: dispatches
- Extraction pipeline → Docling backend: dispatches
- Docling backend → OCR configuration: verifies OCR models
- PyMuPDF backend → Shared document IR: builds IR
- Docling backend → Shared document IR: builds IR
- Extraction pipeline → Overlap reconciliation: reconciles regions
- Overlap reconciliation → Shared document IR: validates IR
- Extraction pipeline → Run artifacts: writes run
- Extraction pipeline → Run export: provides ZIP export
- Review app → Run export: downloads run
- Review app → Evaluation panel: renders evaluation
- Evaluation panel → Evaluation workflow: requests evaluation
- CLI commands → Evaluation workflow: runs evaluation commands
- Other OCR tool → PaddleOCR adapter: provides raw output
- PaddleOCR adapter → Prediction runs: writes normalized prediction
- Evaluation workflow → Evaluation contracts: validates inputs
- Evaluation workflow → Scoring metrics: scores collection
- Scoring metrics → Metric helpers: matches regions
- Evaluation workflow → Reviewed references: reads references
- Evaluation workflow → Prediction runs: reads predictions
- Evaluation workflow → Evaluation reports: writes reports
- Review app → Run artifacts: inspects runs