# lukas-blecher/latex-ocr architecture

> Architecture diagram of the GitHub repository lukas-blecher/latex-ocr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lukas-blecher/latex-ocr
- Repository: https://github.com/lukas-blecher/latex-ocr

## Overview

pix2tex is a LaTeX OCR system: users submit or capture a formula image, the model preprocesses it and generates LaTeX, which an interface returns or displays. The repository also includes a dataset-building and model-training workflow, plus evaluation. The graph preserves documented product capabilities even where implementation excerpts were unavailable; edges for those relationships cite the README, and unsampled internals are represented only at paths supported by the tree.

Main entry points: [`pix2tex/cli.py`](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/cli.py) (LatexOCR), [`pix2tex/models/utils.py`](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/models/utils.py) (Model assembly), [`pix2tex/dataset/arxiv.py`](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/arxiv.py) (arXiv ingestion), [`pix2tex/gui.py`](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/gui.py) (Desktop GUI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User interfaces"]
  node_cli["CLI<br/>[cli.py]"]
  node_gui["Desktop GUI<br/>[gui.py]"]
  node_api["Prediction API<br/>[app.py]"]
  node_streamlit["Streamlit demo<br/>[streamlit.py]"]
  node_mathjax["MathJax rendering<br/>[MathJax.js]"]
end

subgraph group_inference["OCR inference"]
  node_ocr["LatexOCR<br/>[cli.py]"]
  node_imageprep["Image transforms<br/>[transforms.py]"]
  node_resizer["Resolution model<br/>[train_resizer.py]"]
  node_checkpoint["Checkpoint download"]
  node_modelbuilder["Model assembly<br/>[utils.py]"]
  node_hybrid["Hybrid ViT encoder<br/>[hybrid.py]"]
  node_vit["ViT encoder<br/>[vit.py]"]
  node_decoder["Transformer decoder<br/>[transformer.py]"]
end

subgraph group_data["Dataset preparation"]
  node_dataset["Paired dataset<br/>[dataset.py]"]
  node_scraping["Formula scraping<br/>[scraping.py]"]
  node_arxiv["arXiv ingestion<br/>[arxiv.py]"]
  node_extract["LaTeX extraction<br/>[extract_latex.py]"]
  node_demacro["Macro expansion<br/>[demacro.py]"]
  node_render["Formula rendering<br/>[render.py]"]
  node_latex2png["PDF to PNG<br/>[latex2png.py]"]
  node_preprocessing["Formula preprocessing"]
  node_vocab["Vocabulary generation"]
end

subgraph group_learning["Model development"]
  node_trainer["Model training<br/>[train.py]"]
  node_evaluator["Model evaluation<br/>[eval.py]"]
end

node_user(("Image user"))
node_weights[("Training data")]

node_user -->|"submits image"| node_cli
node_user -->|"captures image"| node_gui
node_user -->|"uploads image"| node_api
node_cli -->|"invokes"| node_ocr
node_gui -->|"predicts with"| node_ocr
node_api -->|"predicts with"| node_ocr
node_streamlit -->|"calls API"| node_api
node_ocr -->|"transforms image"| node_imageprep
node_ocr -->|"resizes image"| node_resizer
node_ocr -->|"loads model"| node_modelbuilder
node_ocr -->|"loads weights"| node_checkpoint
node_modelbuilder -->|"builds encoder"| node_hybrid
node_modelbuilder -->|"builds encoder"| node_vit
node_modelbuilder -->|"builds decoder"| node_decoder
node_gui -->|"renders output"| node_mathjax
node_weights -->|"pairs labels"| node_dataset
node_scraping -->|"extracts formulas"| node_extract
node_arxiv -->|"extracts formulas"| node_extract
node_arxiv -->|"uses scraping"| node_scraping
node_arxiv -->|"expands macros"| node_demacro
node_demacro -->|"removes labels"| node_extract
node_render -->|"converts PDF"| node_latex2png
node_preprocessing -.->|"supports tokenization"| node_vocab
node_dataset -->|"supplies batches"| node_trainer
node_trainer -->|"evaluates model"| node_evaluator
node_dataset -->|"supplies examples"| node_evaluator

click node_cli "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/cli.py"
click node_gui "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/gui.py"
click node_api "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/api/app.py"
click node_streamlit "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/api/streamlit.py"
click node_ocr "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/cli.py"
click node_imageprep "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/transforms.py"
click node_resizer "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/train_resizer.py"
click node_checkpoint "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/model/checkpoints/get_latest_checkpoint.py"
click node_modelbuilder "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/models/utils.py"
click node_hybrid "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/models/hybrid.py"
click node_vit "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/models/vit.py"
click node_decoder "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/models/transformer.py"
click node_mathjax "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/resources/MathJax.js"
click node_dataset "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/dataset.py"
click node_scraping "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/scraping.py"
click node_arxiv "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/arxiv.py"
click node_extract "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/extract_latex.py"
click node_demacro "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/demacro.py"
click node_render "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/render.py"
click node_latex2png "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/latex2png.py"
click node_preprocessing "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/preprocessing/preprocess_formulas.py"
click node_vocab "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/preprocessing/generate_latex_vocab.py"
click node_trainer "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/train.py"
click node_evaluator "https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/eval.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_gui,node_api,node_streamlit,node_mathjax,node_user toneBlue
class node_ocr,node_imageprep,node_resizer,node_checkpoint,node_modelbuilder,node_hybrid,node_vit,node_decoder,node_weights toneAmber
class node_dataset,node_scraping,node_arxiv,node_extract,node_demacro,node_render,node_latex2png,node_preprocessing,node_vocab toneMint
class node_trainer,node_evaluator toneRose
```

## Components

### User interfaces

- [CLI](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/cli.py): `pix2tex/cli.py`
- [Desktop GUI](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/gui.py): `pix2tex/gui.py`
- [Prediction API](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/api/app.py): `pix2tex/api/app.py`
- [Streamlit demo](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/api/streamlit.py): `pix2tex/api/streamlit.py`
- [MathJax rendering](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/resources/MathJax.js): `pix2tex/resources/MathJax.js`

### OCR inference

- [LatexOCR](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/cli.py): `pix2tex/cli.py`
- [Image transforms](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/transforms.py): `pix2tex/dataset/transforms.py`
- [Resolution model](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/train_resizer.py): `pix2tex/train_resizer.py`
- [Checkpoint download](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/model/checkpoints/get_latest_checkpoint.py): `pix2tex/model/checkpoints/get_latest_checkpoint.py`
- [Model assembly](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/models/utils.py): `pix2tex/models/utils.py`
- [Hybrid ViT encoder](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/models/hybrid.py): `pix2tex/models/hybrid.py`
- [ViT encoder](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/models/vit.py): `pix2tex/models/vit.py`
- [Transformer decoder](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/models/transformer.py): `pix2tex/models/transformer.py`

### Dataset preparation

- [Paired dataset](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/dataset.py): `pix2tex/dataset/dataset.py`
- [Formula scraping](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/scraping.py): `pix2tex/dataset/scraping.py`
- [arXiv ingestion](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/arxiv.py): `pix2tex/dataset/arxiv.py`
- [LaTeX extraction](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/extract_latex.py): `pix2tex/dataset/extract_latex.py`
- [Macro expansion](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/demacro.py): `pix2tex/dataset/demacro.py`
- [Formula rendering](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/render.py): `pix2tex/dataset/render.py`
- [PDF to PNG](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/latex2png.py): `pix2tex/dataset/latex2png.py`
- [Formula preprocessing](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/preprocessing/preprocess_formulas.py): `pix2tex/dataset/preprocessing/preprocess_formulas.py`
- [Vocabulary generation](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/dataset/preprocessing/generate_latex_vocab.py): `pix2tex/dataset/preprocessing/generate_latex_vocab.py`

### Model development

- [Model training](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/train.py): `pix2tex/train.py`
- [Model evaluation](https://github.com/lukas-blecher/latex-ocr/blob/main/pix2tex/eval.py): `pix2tex/eval.py`

### Other components

- **Image user**
- **Training data**

## Connections

- Image user → CLI: submits image
- Image user → Desktop GUI: captures image
- Image user → Prediction API: uploads image
- CLI → LatexOCR: invokes (evidence: `pix2tex/__main__.py`)
- Desktop GUI → LatexOCR: predicts with (evidence: `pix2tex/gui.py`)
- Prediction API → LatexOCR: predicts with (evidence: `pix2tex/api/app.py`)
- Streamlit demo → Prediction API: calls API
- LatexOCR → Image transforms: transforms image (evidence: `pix2tex/cli.py`)
- LatexOCR → Resolution model: resizes image
- LatexOCR → Model assembly: loads model (evidence: `pix2tex/cli.py`)
- LatexOCR → Checkpoint download: loads weights (evidence: `pix2tex/cli.py`)
- Model assembly → Hybrid ViT encoder: builds encoder (evidence: `pix2tex/models/utils.py`)
- Model assembly → ViT encoder: builds encoder (evidence: `pix2tex/models/utils.py`)
- Model assembly → Transformer decoder: builds decoder (evidence: `pix2tex/models/utils.py`)
- Desktop GUI → MathJax rendering: renders output
- Training data → Paired dataset: pairs labels
- Formula scraping → LaTeX extraction: extracts formulas (evidence: `pix2tex/dataset/scraping.py`)
- arXiv ingestion → LaTeX extraction: extracts formulas (evidence: `pix2tex/dataset/arxiv.py`)
- arXiv ingestion → Formula scraping: uses scraping (evidence: `pix2tex/dataset/arxiv.py`)
- arXiv ingestion → Macro expansion: expands macros (evidence: `pix2tex/dataset/arxiv.py`)
- Macro expansion → LaTeX extraction: removes labels (evidence: `pix2tex/dataset/demacro.py`)
- Formula rendering → PDF to PNG: converts PDF (evidence: `pix2tex/dataset/render.py`)
- Formula preprocessing → Vocabulary generation: supports tokenization
- Paired dataset → Model training: supplies batches (evidence: `pix2tex/train.py`)
- Model training → Model evaluation: evaluates model (evidence: `pix2tex/train.py`)
- Paired dataset → Model evaluation: supplies examples (evidence: `pix2tex/eval.py`)
