# opendatalab/pdf-extract-kit architecture

> Architecture diagram of the GitHub repository opendatalab/pdf-extract-kit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opendatalab/pdf-extract-kit
- Repository: https://github.com/opendatalab/pdf-extract-kit

## Overview

PDF-Extract-Kit is a modular toolkit of document-parsing models, with a sample PDF-to-Markdown application as its end-to-end workflow. That application renders pages, combines layout and formula detections with OCR and formula recognition, then merges spatial results into Markdown. Separately, users can run task-specific detection, recognition, OCR, and table-parsing tasks. The source excerpts show the PDF rendering, formula-detection, and result-merging links; some application-stage wiring and model internals are unsampled, so those links are omitted or marked unverified.

Main entry points: [`project/pdf2markdown/scripts/pdf2markdown.py`](https://github.com/opendatalab/pdf-extract-kit/blob/main/project/pdf2markdown/scripts/pdf2markdown.py) (PDF-to-Markdown), [`pdf_extract_kit/utils/config_loader.py`](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/utils/config_loader.py) (Config Loader), [`pdf_extract_kit/tasks/base_task.py`](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/base_task.py) (Base Task), [`pdf_extract_kit/tasks/layout_detection/task.py`](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/layout_detection/task.py) (Layout Task)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Task Runtime"]
  node_config["Config Loader<br/>[config_loader.py]"]
  node_registry["Task and Model Registry<br/>[registry.py]"]
  node_base["Base Task<br/>[base_task.py]"]
  node_layout_task["Layout Task<br/>[task.py]"]
  node_formula_detect_task["Formula Detection Task<br/>[task.py]"]
  node_formula_rec_task["Formula Recognition Task<br/>[task.py]"]
  node_ocr_task["OCR Task<br/>[task.py]"]
  node_table_task["Table Parsing Task<br/>[task.py]"]
end

subgraph group_models["Parsing Models"]
  node_layout_model["Layout Models"]
  node_formula_detect_model["YOLO Formula Detector<br/>[yolo.py]"]
  node_formula_rec_model["UniMERNet<br/>[unimernet.py]"]
  node_ocr_model["PaddleOCR<br/>[paddle_ocr.py]"]
  node_table_model["StructEqTable<br/>[struct_eqtable.py]"]
end

subgraph group_application["PDF to Markdown"]
  node_pdfmd["PDF-to-Markdown<br/>[pdf2markdown.py]"]
  node_markdown_output["Markdown Output<br/>[pdf2markdown.py]"]
end

subgraph group_utilities["Document Utilities"]
  node_pdf_render["PDF Page Rendering<br/>[data_preprocess.py]"]
  node_image_data["Image Datasets<br/>[dataset.py]"]
  node_merge["Block and Span Merging"]
  node_visualization["Result Visualization<br/>[visualization.py]"]
end

node_user(("User"))

node_user -->|"provides config"| node_config
node_config -->|"resolves classes"| node_registry
node_config -->|"instantiates"| node_layout_task
node_config -->|"instantiates"| node_formula_detect_task
node_config -->|"instantiates"| node_formula_rec_task
node_config -->|"instantiates"| node_ocr_task
node_config -->|"instantiates"| node_table_task
node_layout_task -->|"inherits"| node_base
node_formula_detect_task -->|"inherits"| node_base
node_formula_rec_task -->|"inherits"| node_base
node_ocr_task -->|"inherits"| node_base
node_formula_detect_task -->|"calls predict"| node_formula_detect_model
node_formula_rec_task -.->|"uses model"| node_formula_rec_model
node_ocr_task -->|"calls predict"| node_ocr_model
node_table_task -.->|"uses model"| node_table_model
node_layout_task -.->|"uses model"| node_layout_model
node_user -->|"runs extraction"| node_pdfmd
node_pdfmd -->|"renders pages"| node_pdf_render
node_pdfmd -->|"detects formulas"| node_formula_detect_model
node_pdfmd -.->|"requests layout"| node_layout_model
node_pdfmd -.->|"requests OCR"| node_ocr_model
node_pdfmd -.->|"recognizes formulas"| node_formula_rec_model
node_pdfmd -->|"uses dataset"| node_image_data
node_pdfmd -->|"merges spans"| node_merge
node_pdfmd -->|"writes Markdown"| node_markdown_output
node_layout_model -->|"visualizes results"| node_visualization
node_formula_detect_model -->|"loads batches"| node_image_data

click node_config "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/utils/config_loader.py"
click node_registry "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/registry/registry.py"
click node_base "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/base_task.py"
click node_layout_task "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/layout_detection/task.py"
click node_layout_model "https://github.com/opendatalab/pdf-extract-kit/tree/main/pdf_extract_kit/tasks/layout_detection/models"
click node_formula_detect_task "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/formula_detection/task.py"
click node_formula_detect_model "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/formula_detection/models/yolo.py"
click node_formula_rec_task "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/formula_recognition/task.py"
click node_formula_rec_model "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/formula_recognition/models/unimernet.py"
click node_ocr_task "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/ocr/task.py"
click node_ocr_model "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/ocr/models/paddle_ocr.py"
click node_table_task "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/table_parsing/task.py"
click node_table_model "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/table_parsing/models/struct_eqtable.py"
click node_pdfmd "https://github.com/opendatalab/pdf-extract-kit/blob/main/project/pdf2markdown/scripts/pdf2markdown.py"
click node_pdf_render "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/utils/data_preprocess.py"
click node_image_data "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/dataset/dataset.py"
click node_merge "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/utils/merge_blocks_and_spans.py"
click node_visualization "https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/utils/visualization.py"
click node_markdown_output "https://github.com/opendatalab/pdf-extract-kit/blob/main/project/pdf2markdown/scripts/pdf2markdown.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config,node_registry,node_base,node_layout_task,node_formula_detect_task,node_formula_rec_task,node_ocr_task,node_table_task,node_user toneBlue
class node_layout_model,node_formula_detect_model,node_formula_rec_model,node_ocr_model,node_table_model toneAmber
class node_pdfmd,node_markdown_output toneMint
class node_pdf_render,node_image_data,node_merge,node_visualization toneRose
```

## Components

### Task Runtime

- [Config Loader](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/utils/config_loader.py): `pdf_extract_kit/utils/config_loader.py`
- [Task and Model Registry](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/registry/registry.py): `pdf_extract_kit/registry/registry.py`
- [Base Task](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/base_task.py): `pdf_extract_kit/tasks/base_task.py`
- [Layout Task](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/layout_detection/task.py): `pdf_extract_kit/tasks/layout_detection/task.py`
- [Formula Detection Task](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/formula_detection/task.py): `pdf_extract_kit/tasks/formula_detection/task.py`
- [Formula Recognition Task](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/formula_recognition/task.py): `pdf_extract_kit/tasks/formula_recognition/task.py`
- [OCR Task](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/ocr/task.py): `pdf_extract_kit/tasks/ocr/task.py`
- [Table Parsing Task](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/table_parsing/task.py): `pdf_extract_kit/tasks/table_parsing/task.py`

### Parsing Models

- [Layout Models](https://github.com/opendatalab/pdf-extract-kit/tree/main/pdf_extract_kit/tasks/layout_detection/models): `pdf_extract_kit/tasks/layout_detection/models`
- [YOLO Formula Detector](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/formula_detection/models/yolo.py): `pdf_extract_kit/tasks/formula_detection/models/yolo.py`
- [UniMERNet](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/formula_recognition/models/unimernet.py): `pdf_extract_kit/tasks/formula_recognition/models/unimernet.py`
- [PaddleOCR](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/ocr/models/paddle_ocr.py): `pdf_extract_kit/tasks/ocr/models/paddle_ocr.py`
- [StructEqTable](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/tasks/table_parsing/models/struct_eqtable.py): `pdf_extract_kit/tasks/table_parsing/models/struct_eqtable.py`

### PDF to Markdown

- [PDF-to-Markdown](https://github.com/opendatalab/pdf-extract-kit/blob/main/project/pdf2markdown/scripts/pdf2markdown.py): `project/pdf2markdown/scripts/pdf2markdown.py`
- [Markdown Output](https://github.com/opendatalab/pdf-extract-kit/blob/main/project/pdf2markdown/scripts/pdf2markdown.py): `project/pdf2markdown/scripts/pdf2markdown.py`

### Document Utilities

- [PDF Page Rendering](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/utils/data_preprocess.py): `pdf_extract_kit/utils/data_preprocess.py`
- [Image Datasets](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/dataset/dataset.py): `pdf_extract_kit/dataset/dataset.py`
- [Block and Span Merging](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/utils/merge_blocks_and_spans.py): `pdf_extract_kit/utils/merge_blocks_and_spans.py`
- [Result Visualization](https://github.com/opendatalab/pdf-extract-kit/blob/main/pdf_extract_kit/utils/visualization.py): `pdf_extract_kit/utils/visualization.py`

### Other components

- **User**

## Connections

- User → Config Loader: provides config (evidence: `pdf_extract_kit/utils/config_loader.py`)
- Config Loader → Task and Model Registry: resolves classes (evidence: `pdf_extract_kit/utils/config_loader.py`)
- Config Loader → Layout Task: instantiates (evidence: `pdf_extract_kit/utils/config_loader.py`)
- Config Loader → Formula Detection Task: instantiates (evidence: `pdf_extract_kit/utils/config_loader.py`)
- Config Loader → Formula Recognition Task: instantiates (evidence: `pdf_extract_kit/utils/config_loader.py`)
- Config Loader → OCR Task: instantiates (evidence: `pdf_extract_kit/utils/config_loader.py`)
- Config Loader → Table Parsing Task: instantiates (evidence: `pdf_extract_kit/utils/config_loader.py`)
- Layout Task → Base Task: inherits (evidence: `pdf_extract_kit/tasks/layout_detection/task.py`)
- Formula Detection Task → Base Task: inherits (evidence: `pdf_extract_kit/tasks/formula_detection/task.py`)
- Formula Recognition Task → Base Task: inherits (evidence: `pdf_extract_kit/tasks/formula_recognition/task.py`)
- OCR Task → Base Task: inherits (evidence: `pdf_extract_kit/tasks/ocr/task.py`)
- Formula Detection Task → YOLO Formula Detector: calls predict (evidence: `pdf_extract_kit/tasks/formula_detection/task.py`)
- Formula Recognition Task → UniMERNet: uses model (evidence: `pdf_extract_kit/tasks/formula_recognition/task.py`)
- OCR Task → PaddleOCR: calls predict (evidence: `pdf_extract_kit/tasks/ocr/task.py`)
- Table Parsing Task → StructEqTable: uses model (evidence: `pdf_extract_kit/tasks/table_parsing/task.py`)
- Layout Task → Layout Models: uses model (evidence: `pdf_extract_kit/tasks/layout_detection/task.py`)
- User → PDF-to-Markdown: runs extraction (evidence: `project/pdf2markdown/scripts/pdf2markdown.py`)
- PDF-to-Markdown → PDF Page Rendering: renders pages (evidence: `project/pdf2markdown/scripts/pdf2markdown.py`)
- PDF-to-Markdown → YOLO Formula Detector: detects formulas (evidence: `project/pdf2markdown/scripts/pdf2markdown.py`)
- PDF-to-Markdown → Layout Models: requests layout
- PDF-to-Markdown → PaddleOCR: requests OCR
- PDF-to-Markdown → UniMERNet: recognizes formulas
- PDF-to-Markdown → Image Datasets: uses dataset (evidence: `project/pdf2markdown/scripts/pdf2markdown.py`)
- PDF-to-Markdown → Block and Span Merging: merges spans (evidence: `project/pdf2markdown/scripts/pdf2markdown.py`)
- PDF-to-Markdown → Markdown Output: writes Markdown (evidence: `project/pdf2markdown/scripts/pdf2markdown.py`)
- Layout Models → Result Visualization: visualizes results (evidence: `pdf_extract_kit/tasks/layout_detection/models/layoutlmv3.py`)
- YOLO Formula Detector → Image Datasets: loads batches (evidence: `pdf_extract_kit/tasks/formula_detection/models/yolo.py`)
