# allenai/olmocr architecture

> Architecture diagram of the GitHub repository allenai/olmocr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/allenai/olmocr
- Repository: https://github.com/allenai/olmocr

## Overview

olmOCR converts image-based documents into readable Markdown, preserving layout-sensitive content such as equations and tables while removing headers and footers. The primary runtime path accepts documents, renders PDF pages, supplies visual and anchor-text context to a vision-language model, validates responses, and produces text output. Supporting branches prepare OCR training data, train models, and generate synthetic examples. The README documents broad OCR capabilities, but source excerpts are partial; edges below are limited to relationships explicit in the excerpts, SOURCE INDEX references, or README.

Main entry points: [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) (OCR Pipeline), [`olmocr/train/dataloader.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py) (Training Loader), [`olmocr/data/renderpdf.py`](https://github.com/allenai/olmocr/blob/main/olmocr/data/renderpdf.py) (PDF Renderer), [`olmocr/synth/mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py) (Synthetic Data)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ocr["OCR Runtime"]
  node_pipeline["OCR Pipeline<br/>[pipeline.py]"]
  node_render["PDF Renderer<br/>[renderpdf.py]"]
  node_anchor["Page Anchors<br/>[anchor.py]"]
  node_prompts["OCR Prompts<br/>[prompts.py]"]
  node_vlm{{"VLM Inference"}}
end

subgraph group_quality["OCR Quality"]
  node_response["Response Schema<br/>[prompts.py]"]
  node_filter["PDF Filtering<br/>[filter.py]"]
  node_coherency["Coherency Checks<br/>[coherency.py]"]
  node_metrics["Run Metrics<br/>[metrics.py]"]
end

subgraph group_data["Data Preparation"]
  node_dataset["Dataset Preparation"]
  node_transcripts["Transcript Sources"]
  node_clean["Dataset Cleaning<br/>[clean_olmocrmix.py]"]
  node_workspace["Workspace Setup"]
  node_synthetic["Synthetic Data"]
  node_augment["Data Augmentation<br/>[augmentations.py]"]
end

subgraph group_learning["Model Learning"]
  node_trainer["Model Training<br/>[train.py]"]
  node_loader["Training Loader<br/>[dataloader.py]"]
  node_training_config["Training Config<br/>[config.py]"]
end

subgraph group_platform["Shared Services"]
  node_queue["Work Queue<br/>[work_queue.py]"]
  node_storage[("Object Storage<br/>[s3_utils.py]")]
  node_claude{{"Claude API"}}
end

node_user(("Document User"))

node_user -->|"submits documents"| node_pipeline
node_pipeline -->|"renders pages"| node_render
node_pipeline -->|"builds context"| node_anchor
node_pipeline -->|"formats requests"| node_prompts
node_pipeline -->|"requests inference"| node_vlm
node_vlm -->|"returns response"| node_response
node_pipeline -->|"filters inputs"| node_filter
node_pipeline -->|"checks output"| node_coherency
node_pipeline -->|"records metrics"| node_metrics
node_pipeline -->|"dispatches work"| node_queue
node_queue -->|"persists queue state"| node_storage
node_dataset -->|"uses storage"| node_storage
node_transcripts -.->|"provides source data"| node_dataset
node_clean -->|"uses rendering"| node_render
node_synthetic -->|"generates examples"| node_claude
node_synthetic -->|"uses augmentations"| node_augment
node_trainer -->|"loads configuration"| node_training_config
node_trainer -->|"loads batches"| node_loader
node_loader -->|"renders training pages"| node_render
node_loader -->|"builds anchors"| node_anchor
node_loader -->|"uses prompts"| node_prompts

click node_pipeline "https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py"
click node_render "https://github.com/allenai/olmocr/blob/main/olmocr/data/renderpdf.py"
click node_anchor "https://github.com/allenai/olmocr/blob/main/olmocr/prompts/anchor.py"
click node_prompts "https://github.com/allenai/olmocr/blob/main/olmocr/prompts/prompts.py"
click node_response "https://github.com/allenai/olmocr/blob/main/olmocr/prompts/prompts.py"
click node_filter "https://github.com/allenai/olmocr/blob/main/olmocr/filter/filter.py"
click node_coherency "https://github.com/allenai/olmocr/blob/main/olmocr/filter/coherency.py"
click node_metrics "https://github.com/allenai/olmocr/blob/main/olmocr/metrics.py"
click node_dataset "https://github.com/allenai/olmocr/blob/main/olmocr/data/prepare_olmocrmix.py"
click node_transcripts "https://github.com/allenai/olmocr/blob/main/olmocr/data/prepare_national_archive_transcripts.py"
click node_clean "https://github.com/allenai/olmocr/blob/main/olmocr/data/clean_olmocrmix.py"
click node_workspace "https://github.com/allenai/olmocr/blob/main/olmocr/data/prepare_workspace.py"
click node_synthetic "https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py"
click node_augment "https://github.com/allenai/olmocr/blob/main/olmocr/synth/augmentations.py"
click node_trainer "https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py"
click node_loader "https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py"
click node_training_config "https://github.com/allenai/olmocr/blob/main/olmocr/train/config.py"
click node_queue "https://github.com/allenai/olmocr/blob/main/olmocr/work_queue.py"
click node_storage "https://github.com/allenai/olmocr/blob/main/olmocr/s3_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline,node_render,node_anchor,node_prompts,node_vlm,node_user toneBlue
class node_response,node_filter,node_coherency,node_metrics toneAmber
class node_dataset,node_transcripts,node_clean,node_workspace,node_synthetic,node_augment toneMint
class node_trainer,node_loader,node_training_config toneRose
class node_queue,node_storage,node_claude toneIndigo
```

## Components

### OCR Runtime

- [OCR Pipeline](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py): `olmocr/pipeline.py`
- [PDF Renderer](https://github.com/allenai/olmocr/blob/main/olmocr/data/renderpdf.py): `olmocr/data/renderpdf.py`
- [Page Anchors](https://github.com/allenai/olmocr/blob/main/olmocr/prompts/anchor.py): `olmocr/prompts/anchor.py`
- [OCR Prompts](https://github.com/allenai/olmocr/blob/main/olmocr/prompts/prompts.py): `olmocr/prompts/prompts.py`
- **VLM Inference**

### OCR Quality

- [Response Schema](https://github.com/allenai/olmocr/blob/main/olmocr/prompts/prompts.py): `olmocr/prompts/prompts.py`
- [PDF Filtering](https://github.com/allenai/olmocr/blob/main/olmocr/filter/filter.py): `olmocr/filter/filter.py`
- [Coherency Checks](https://github.com/allenai/olmocr/blob/main/olmocr/filter/coherency.py): `olmocr/filter/coherency.py`
- [Run Metrics](https://github.com/allenai/olmocr/blob/main/olmocr/metrics.py): `olmocr/metrics.py`

### Data Preparation

- [Dataset Preparation](https://github.com/allenai/olmocr/blob/main/olmocr/data/prepare_olmocrmix.py): `olmocr/data/prepare_olmocrmix.py`
- [Transcript Sources](https://github.com/allenai/olmocr/blob/main/olmocr/data/prepare_national_archive_transcripts.py): `olmocr/data/prepare_national_archive_transcripts.py`
- [Dataset Cleaning](https://github.com/allenai/olmocr/blob/main/olmocr/data/clean_olmocrmix.py): `olmocr/data/clean_olmocrmix.py`
- [Workspace Setup](https://github.com/allenai/olmocr/blob/main/olmocr/data/prepare_workspace.py): `olmocr/data/prepare_workspace.py`
- [Synthetic Data](https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py): `olmocr/synth/mine_html_templates.py`
- [Data Augmentation](https://github.com/allenai/olmocr/blob/main/olmocr/synth/augmentations.py): `olmocr/synth/augmentations.py`

### Model Learning

- [Model Training](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py): `olmocr/train/train.py`
- [Training Loader](https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py): `olmocr/train/dataloader.py`
- [Training Config](https://github.com/allenai/olmocr/blob/main/olmocr/train/config.py): `olmocr/train/config.py`

### Shared Services

- [Work Queue](https://github.com/allenai/olmocr/blob/main/olmocr/work_queue.py): `olmocr/work_queue.py`
- [Object Storage](https://github.com/allenai/olmocr/blob/main/olmocr/s3_utils.py): `olmocr/s3_utils.py`
- **Claude API**

### Other components

- **Document User**

## Connections

- Document User → OCR Pipeline: submits documents
- OCR Pipeline → PDF Renderer: renders pages (evidence: `olmocr/pipeline.py`)
- OCR Pipeline → Page Anchors: builds context (evidence: `olmocr/pipeline.py`)
- OCR Pipeline → OCR Prompts: formats requests (evidence: `olmocr/pipeline.py`)
- OCR Pipeline → VLM Inference: requests inference (evidence: `olmocr/pipeline.py`)
- VLM Inference → Response Schema: returns response
- OCR Pipeline → PDF Filtering: filters inputs (evidence: `olmocr/pipeline.py`)
- OCR Pipeline → Coherency Checks: checks output
- OCR Pipeline → Run Metrics: records metrics (evidence: `olmocr/pipeline.py`)
- OCR Pipeline → Work Queue: dispatches work (evidence: `olmocr/pipeline.py`)
- Work Queue → Object Storage: persists queue state (evidence: `olmocr/work_queue.py`)
- Dataset Preparation → Object Storage: uses storage
- Transcript Sources → Dataset Preparation: provides source data
- Dataset Cleaning → PDF Renderer: uses rendering (evidence: `olmocr/data/clean_olmocrmix.py`)
- Synthetic Data → Claude API: generates examples (evidence: `olmocr/synth/mine_html_templates.py`)
- Synthetic Data → Data Augmentation: uses augmentations (evidence: `olmocr/synth/mine_html_templates.py`)
- Model Training → Training Config: loads configuration (evidence: `olmocr/train/train.py`)
- Model Training → Training Loader: loads batches (evidence: `olmocr/train/train.py`)
- Training Loader → PDF Renderer: renders training pages (evidence: `olmocr/train/dataloader.py`)
- Training Loader → Page Anchors: builds anchors (evidence: `olmocr/train/dataloader.py`)
- Training Loader → OCR Prompts: uses prompts (evidence: `olmocr/train/dataloader.py`)
