How allenai/olmocr works
olmOCR converts image-based documents into readable Markdown, preserving layout-sensitive content such as equations and tables while removing headers and footers. The primary runtime path accepts documents, renders PDF pages, supplies visual and anchor-text context to a vision-language model, validates responses, and produces text output. Supporting branches prepare OCR training data, train models, and generate synthetic examples. The README documents broad OCR capabilities, but source excerpts are partial; edges below are limited to relationships explicit in the excerpts, SOURCE INDEX references, or README.
Main entry points: pipeline.py OCR Pipeline · dataloader.py Training Loader · renderpdf.py PDF Renderer · mine_html_templates.py Synthetic Data
Components 22
OCR Runtime
- OCR Pipeline
olmocr/pipeline.py - PDF Renderer
olmocr/data/renderpdf.py - Page Anchors
olmocr/prompts/anchor.py - OCR Prompts
olmocr/prompts/prompts.py - VLM Inference
OCR Quality
- Response Schema
olmocr/prompts/prompts.py - PDF Filtering
olmocr/filter/filter.py - Coherency Checks
olmocr/filter/coherency.py - Run Metrics
olmocr/metrics.py
Data Preparation
- Dataset Preparation
olmocr/data/prepare_olmocrmix.py - Transcript Sources
olmocr/data/prepare_national_archive_transcripts.py - Dataset Cleaning
olmocr/data/clean_olmocrmix.py - Workspace Setup
olmocr/data/prepare_workspace.py - Synthetic Data
olmocr/synth/mine_html_templates.py - Data Augmentation
olmocr/synth/augmentations.py
Model Learning
- Model Training
olmocr/train/train.py - Training Loader
olmocr/train/dataloader.py - Training Config
olmocr/train/config.py
Shared Services
- Work Queue
olmocr/work_queue.py - Object Storage
olmocr/s3_utils.py - Claude API
Other components
- Document User
Connections 21
- Document User → OCR Pipeline: submits documents
- OCR Pipeline → PDF Renderer: renders pages
olmocr/pipeline.py - OCR Pipeline → Page Anchors: builds context
olmocr/pipeline.py - OCR Pipeline → OCR Prompts: formats requests
olmocr/pipeline.py - OCR Pipeline → VLM Inference: requests inference
olmocr/pipeline.py - VLM Inference → Response Schema: returns response
- OCR Pipeline → PDF Filtering: filters inputs
olmocr/pipeline.py - OCR Pipeline → Coherency Checks: checks output
- OCR Pipeline → Run Metrics: records metrics
olmocr/pipeline.py - OCR Pipeline → Work Queue: dispatches work
olmocr/pipeline.py - Work Queue → Object Storage: persists queue state
olmocr/work_queue.py - Dataset Preparation → Object Storage: uses storage
- Transcript Sources → Dataset Preparation: provides source data
- Dataset Cleaning → PDF Renderer: uses rendering
olmocr/data/clean_olmocrmix.py - Synthetic Data → Claude API: generates examples
olmocr/synth/mine_html_templates.py - Synthetic Data → Data Augmentation: uses augmentations
olmocr/synth/mine_html_templates.py - Model Training → Training Config: loads configuration
olmocr/train/train.py - Model Training → Training Loader: loads batches
olmocr/train/train.py - Training Loader → PDF Renderer: renders training pages
olmocr/train/dataloader.py - Training Loader → Page Anchors: builds anchors
olmocr/train/dataloader.py - Training Loader → OCR Prompts: uses prompts
olmocr/train/dataloader.py