# datalab-to/marker architecture

> Architecture diagram of the GitHub repository datalab-to/marker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/datalab-to/marker
- Repository: https://github.com/datalab-to/marker

## Overview

#### 1. Purpose and principal workflow

Marker converts local documents into Markdown, JSON, HTML, or chunks, preserving structured content such as tables, equations, forms, images, and references. A command-line user, Python library caller, or HTTP API client initiates the main workflow by supplying a document and options; Marker builds a document/block representation, conditionally uses local Surya VLM inference and optional cloud LLM refinement, renders the result, and returns or saves formatted output.

#### 2. Components

##### Invocation and delivery

- **Command-line user** — null repository path — Runs the installed conversion commands described in the README.
- **Python library caller** — null repository path — Constructs converters and model artifacts directly, as shown in the documented Python API.
- **HTTP API client** — null repository path — Sends a file path or multipart upload to the conversion API.
- **Batch CLI command** — `marker/scripts/convert.py` — Provides the installed `marker` command for multi-file conversion, including worker and resume-oriented options documented in the README.
- **Single-file CLI command** — `marker/scripts/convert_single.py` — Provides the installed `marker_single` command for one input document and selectable output format.
- **Interactive Streamlit launcher** — `marker/scripts/run_streamlit_app.py` — Provides the documented `marker_gui` interactive entry point.
- **HTTP conversion API** — `marker/scripts/server.py` — A FastAPI application that initializes shared models, accepts `/marker` path requests and `/marker/upload` file requests, constructs a PDF converter, base64-encodes extracted images, and removes uploaded temporary files.
- **Configuration parser** — `marker/config/parser.py` — Converts request or user configuration into converter settings, processor selection, renderer selection, and optional LLM-service selection.
- **Model artifact manager** — `marker/models.py` — Supplies the model-artifact dictionary used by converters; the API creates it once during lifespan and shuts it down on exit.

##### Document construction and recognition

- **PDF conversion coordinator** — `marker/converters/pdf.py` — The documented primary converter builds a document from an input path, applies configured processing, and produces a rendered result.
- **Table-only conversion coordinator** — `marker/converters/table.py` — A documented alternative pipeline that extracts/converts tables rather than the complete PDF.
- **Source document providers** — `marker/providers/__init__.py` — Defines the provider contract for page images, page geometry, text lines, and references; the tree contains providers for PDF, image, HTML, EPUB, PowerPoint, spreadsheets, and documents.
- **Document-construction pipeline** — `marker/builders/document.py` — Represents the builder layer used to assemble a document from provider data and recognition outputs.
- **Layout analysis builder** — `marker/builders/layout.py` — Owns page-layout detection, which the README says uses Surya VLM in balanced mode and a lightweight detector in fast mode.
- **OCR builder** — `marker/builders/ocr.py` — Owns OCR/text-repair construction for scanned, garbled, forced-OCR, and relevant math content.
- **Structure builder** — `marker/builders/structure.py` — Owns assembly of recognized page content into structural document blocks.
- **Document/block graph** — `marker/schema/document.py` — Holds the mutable document hierarchy of pages and nested blocks that processors and callers can inspect or modify.
- **Table reconstruction processor** — `marker/processors/table_recon.py` — Represents the table-specific reconstruction stage; README documentation says it uses PDF text-layer reconstruction and can fall back to VLM inference.

##### Refinement and output

- **Core document processor suite** — `marker/processors` — Groups non-LLM transformations for artifacts and semantics, including headers/footers, lists, code, equations, references, tables, text, and table merging.
- **LLM enhancement coordinator** — `marker/processors/llm/llm_meta.py` — Consolidates simple LLM block processors so they share the configured LLM service rather than being independently inserted into the processing sequence.
- **Targeted LLM block repairs** — `marker/processors/llm` — Contains optional image-and-text refinements for complex regions, equations, forms, handwriting, tables, metadata, page correction, and section headers; sampled implementations validate responses before replacing block HTML and record error metadata on failures.
- **OpenRouter LLM adapter** — `marker/services/openrouter.py` — Implements an optional OpenAI-compatible OpenRouter client, converts images to data URLs, requests schema-constrained responses, retries timeouts/rate limits, and records token/request metadata on blocks.
- **OpenRouter API** — null repository path — External OpenAI-compatible endpoint used only when the OpenRouter adapter is selected.
- **Surya VLM inference server** — null repository path — External local inference service documented for layout, OCR, and table-recognition work; it may be auto-spawned or addressed through an existing inference URL.
- **Renderer family** — `marker/renderers` — Renders the processed document graph into Markdown, JSON, HTML, OCR JSON, or chunks.
- **Rendered-output normalizer** — `marker/output.py` — Extracts text and images from a renderer result for callers such as the HTTP API.

#### 3. Relationships

- Command-line user -> Batch CLI command: invokes batch conversion; evidence: installed command registration and README batch usage.
- Command-line user -> Single-file CLI command: invokes one-document conversion; evidence: installed command registration and README single-file usage.
- Python library caller -> Model artifact manager: creates artifacts before converter construction; evidence: documented Python example.
- Python library caller -> PDF conversion coordinator: directly invokes conversion or document building; evidence: documented Python example.
- HTTP API client -> HTTP conversion API: submits conversion requests; evidence: `/marker` and `/marker/upload` routes.
- HTTP conversion API -> Configuration parser: translates request fields into configuration and selected collaborators; evidence: `_convert_pdf`.
- HTTP conversion API -> Model artifact manager: initializes and later shuts down shared artifacts; evidence: FastAPI lifespan handler.
- HTTP conversion API -> PDF conversion coordinator: instantiates and invokes the converter with parsed configuration and shared artifacts; evidence: `_convert_pdf`.
- HTTP conversion API -> Rendered-output normalizer: extracts text and images from the rendered result; evidence: `text_from_rendered` call.
- HTTP conversion API -> HTTP API client: returns output, metadata, encoded images, and success/error status; evidence: route return values.
- Batch CLI command -> PDF conversion coordinator: runs configured document conversions; evidence: documented `marker` behavior.
- Single-file CLI command -> PDF conversion coordinator: runs configured conversion for one input; evidence: documented `marker_single` behavior.
- PDF conversion coordinator -> Source document providers: ingests supported input formats into page-level source data; evidence: documented supported formats and provider family.
- Table-only conversion coordinator -> Source document providers: ingests source documents for table extraction; evidence: documented table converter behavior.
- PDF conversion coordinator -> Document-construction pipeline: builds the mutable document representation; evidence: documented `build_document` API.
- Document-construction pipeline -> Layout analysis builder: owns the layout-analysis stage; evidence: builder organization.
- Document-construction pipeline -> OCR builder: owns OCR/text construction; evidence: builder organization.
- Document-construction pipeline -> Structure builder: owns structural assembly; evidence: builder organization.
- Document-construction pipeline -> Document/block graph: constructs pages and nested blocks; evidence: documented document/block model.
- Layout analysis builder -> Surya VLM inference server: conditionally requests VLM layout in balanced mode; evidence: README mode behavior.
- OCR builder -> Surya VLM inference server: conditionally requests VLM OCR or repair; evidence: README OCR behavior.
- Table reconstruction processor -> Surya VLM inference server: conditionally falls back for low-confidence or scanned tables; evidence: README table behavior.
- PDF conversion coordinator -> Core document processor suite: applies configured document transformations; evidence: converter configuration accepts processor lists.
- Core document processor suite -> Table reconstruction processor: includes table reconstruction as a specialized processing capability; evidence: processor organization.
- PDF conversion coordinator -> LLM enhancement coordinator: enables grouped LLM processing when LLM processors are configured; evidence: base-converter initialization logic.
- LLM enhancement coordinator -> Targeted LLM block repairs: groups selected simple LLM processors for shared execution; evidence: meta-processor construction.
- LLM enhancement coordinator -> OpenRouter LLM adapter: conditionally invokes it when it is the configured LLM service; evidence: injected `llm_service` wiring.
- Targeted LLM block repairs -> Document/block graph: replace validated block HTML or record LLM error metadata; evidence: sampled equation, form, complex-region, and handwriting processors.
- OpenRouter LLM adapter -> OpenRouter API: sends image-plus-prompt chat-completion requests and parses schema-shaped responses; evidence: adapter client call.
- PDF conversion coordinator -> Renderer family: produces a rendered output in the selected format; evidence: documented converter output formats.

#### 4. Coverage limits

The sampled source omits the implementations of the CLI handlers, PDF converter, configuration parser, concrete providers, builders, renderer internals, default processor order, and model lifecycle. Therefore, this map distinguishes documented stages from verified API wiring but cannot establish their precise runtime order, all mode-specific branches, all storage behavior of CLI output, or which non-OpenRouter LLM adapter is selected for a given configuration.

Main entry points: [`marker/converters/pdf.py`](https://github.com/datalab-to/marker/blob/master/marker/converters/pdf.py) (PDF conversion coordinator), [`marker/scripts/server.py`](https://github.com/datalab-to/marker/blob/master/marker/scripts/server.py) (HTTP conversion API), [`marker/builders/document.py`](https://github.com/datalab-to/marker/blob/master/marker/builders/document.py) (Document-construction pipeline), [`marker/processors/llm/llm_meta.py`](https://github.com/datalab-to/marker/blob/master/marker/processors/llm/llm_meta.py) (LLM enhancement coordinator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_invocation["Invocation & API"]
  node_cli_user(("Command-line user<br/>actor"))
  node_library_caller(("Python library caller<br/>actor"))
  node_http_client(("HTTP API client<br/>actor"))
  node_batch_cli["Batch CLI command<br/>entrypoint"]
  node_single_cli["Single-file CLI command<br/>entrypoint"]
  node_http_api["HTTP conversion API<br/>[server.py]"]
end

subgraph group_orchestration["Configuration &amp; Orchestration"]
  node_config_parser["Configuration parser<br/>[parser.py]"]
  node_model_manager["Model artifact manager<br/>[models.py]"]
  node_pdf_converter["PDF conversion coordinator<br/>[pdf.py]"]
  node_table_converter["Table-only conversion coordinator<br/>[table.py]"]
end

subgraph group_construction["Document Construction"]
  node_providers["Source document providers<br/>[__init__.py]"]
  node_document_pipeline["Document-construction pipeline<br/>[document.py]"]
  node_layout_builder["Layout analysis builder<br/>[layout.py]"]
  node_ocr_builder["OCR builder<br/>[ocr.py]"]
  node_structure_builder["Structure builder<br/>[structure.py]"]
  node_document_graph[("Document/block graph<br/>state")]
end

subgraph group_refinement["Refinement"]
  node_core_processors["Core document processor suite"]
  node_table_recon["Table reconstruction processor<br/>[table_recon.py]"]
  node_llm_coordinator["LLM enhancement coordinator<br/>[llm_meta.py]"]
  node_targeted_repairs["Targeted LLM block repairs"]
  node_openrouter_adapter["OpenRouter LLM adapter<br/>[openrouter.py]"]
end

subgraph group_delivery["Inference &amp; Delivery"]
  node_surya_server["Surya VLM inference server<br/>external service"]
  node_openrouter_api["OpenRouter API<br/>external service"]
  node_renderers["Renderer family"]
  node_output_normalizer["Rendered-output normalizer<br/>[output.py]"]
end

node_cli_user -->|"invokes batch"| node_batch_cli
node_cli_user -->|"invokes conversion"| node_single_cli
node_library_caller -->|"creates artifacts"| node_model_manager
node_library_caller -->|"invokes conversion"| node_pdf_converter
node_http_client -->|"submits request"| node_http_api
node_http_api -->|"parses configuration"| node_config_parser
node_http_api -->|"manages artifacts"| node_model_manager
node_http_api -->|"invokes converter"| node_pdf_converter
node_http_api -->|"extracts output"| node_output_normalizer
node_http_api -->|"returns result"| node_http_client
node_batch_cli -->|"runs conversions"| node_pdf_converter
node_single_cli -->|"runs conversion"| node_pdf_converter
node_pdf_converter -->|"ingests source"| node_providers
node_table_converter -->|"ingests source"| node_providers
node_pdf_converter -->|"builds document"| node_document_pipeline
node_document_pipeline -->|"analyzes layout"| node_layout_builder
node_document_pipeline -->|"builds OCR"| node_ocr_builder
node_document_pipeline -->|"assembles structure"| node_structure_builder
node_document_pipeline -->|"constructs graph"| node_document_graph
node_layout_builder -.->|"requests layout"| node_surya_server
node_ocr_builder -.->|"requests OCR"| node_surya_server
node_table_recon -.->|"requests fallback"| node_surya_server
node_pdf_converter -->|"applies transformations"| node_core_processors
node_core_processors -->|"includes reconstruction"| node_table_recon
node_pdf_converter -.->|"enables LLM processing"| node_llm_coordinator
node_llm_coordinator -->|"groups repairs"| node_targeted_repairs
node_llm_coordinator -.->|"invokes configured service"| node_openrouter_adapter
node_targeted_repairs -->|"updates blocks"| node_document_graph
node_openrouter_adapter -->|"sends requests"| node_openrouter_api
node_pdf_converter -->|"renders output"| node_renderers

click node_batch_cli "https://github.com/datalab-to/marker/blob/master/marker/scripts/convert.py"
click node_single_cli "https://github.com/datalab-to/marker/blob/master/marker/scripts/convert_single.py"
click node_http_api "https://github.com/datalab-to/marker/blob/master/marker/scripts/server.py"
click node_config_parser "https://github.com/datalab-to/marker/blob/master/marker/config/parser.py"
click node_model_manager "https://github.com/datalab-to/marker/blob/master/marker/models.py"
click node_pdf_converter "https://github.com/datalab-to/marker/blob/master/marker/converters/pdf.py"
click node_table_converter "https://github.com/datalab-to/marker/blob/master/marker/converters/table.py"
click node_providers "https://github.com/datalab-to/marker/blob/master/marker/providers/__init__.py"
click node_document_pipeline "https://github.com/datalab-to/marker/blob/master/marker/builders/document.py"
click node_layout_builder "https://github.com/datalab-to/marker/blob/master/marker/builders/layout.py"
click node_ocr_builder "https://github.com/datalab-to/marker/blob/master/marker/builders/ocr.py"
click node_structure_builder "https://github.com/datalab-to/marker/blob/master/marker/builders/structure.py"
click node_document_graph "https://github.com/datalab-to/marker/blob/master/marker/schema/document.py"
click node_core_processors "https://github.com/datalab-to/marker/tree/master/marker/processors"
click node_table_recon "https://github.com/datalab-to/marker/blob/master/marker/processors/table_recon.py"
click node_llm_coordinator "https://github.com/datalab-to/marker/blob/master/marker/processors/llm/llm_meta.py"
click node_targeted_repairs "https://github.com/datalab-to/marker/tree/master/marker/processors/llm"
click node_openrouter_adapter "https://github.com/datalab-to/marker/blob/master/marker/services/openrouter.py"
click node_renderers "https://github.com/datalab-to/marker/tree/master/marker/renderers"
click node_output_normalizer "https://github.com/datalab-to/marker/blob/master/marker/output.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli_user,node_library_caller,node_http_client,node_batch_cli,node_single_cli,node_http_api toneBlue
class node_config_parser,node_model_manager,node_pdf_converter,node_table_converter toneAmber
class node_providers,node_document_pipeline,node_layout_builder,node_ocr_builder,node_structure_builder,node_document_graph toneMint
class node_core_processors,node_table_recon,node_llm_coordinator,node_targeted_repairs,node_openrouter_adapter toneRose
class node_surya_server,node_openrouter_api,node_renderers,node_output_normalizer toneIndigo
```

## Components

### Invocation & API

User-facing entry points and request handling

- **Command-line user**: actor
- **Python library caller**: actor
- **HTTP API client**: actor
- [Batch CLI command](https://github.com/datalab-to/marker/blob/master/marker/scripts/convert.py): entrypoint · `marker/scripts/convert.py`
- [Single-file CLI command](https://github.com/datalab-to/marker/blob/master/marker/scripts/convert_single.py): entrypoint · `marker/scripts/convert_single.py`
- [HTTP conversion API](https://github.com/datalab-to/marker/blob/master/marker/scripts/server.py): service · `marker/scripts/server.py`

### Configuration & Orchestration

Shared configuration, artifacts, and conversion coordination

- [Configuration parser](https://github.com/datalab-to/marker/blob/master/marker/config/parser.py): service · `marker/config/parser.py`
- [Model artifact manager](https://github.com/datalab-to/marker/blob/master/marker/models.py): service · `marker/models.py`
- [PDF conversion coordinator](https://github.com/datalab-to/marker/blob/master/marker/converters/pdf.py): service · `marker/converters/pdf.py`
- [Table-only conversion coordinator](https://github.com/datalab-to/marker/blob/master/marker/converters/table.py): service · `marker/converters/table.py`

### Document Construction

Source ingestion, recognition, and document graph assembly

- [Source document providers](https://github.com/datalab-to/marker/blob/master/marker/providers/__init__.py): service · `marker/providers/__init__.py`
- [Document-construction pipeline](https://github.com/datalab-to/marker/blob/master/marker/builders/document.py): service · `marker/builders/document.py`
- [Layout analysis builder](https://github.com/datalab-to/marker/blob/master/marker/builders/layout.py): service · `marker/builders/layout.py`
- [OCR builder](https://github.com/datalab-to/marker/blob/master/marker/builders/ocr.py): service · `marker/builders/ocr.py`
- [Structure builder](https://github.com/datalab-to/marker/blob/master/marker/builders/structure.py): service · `marker/builders/structure.py`
- [Document/block graph](https://github.com/datalab-to/marker/blob/master/marker/schema/document.py): state · `marker/schema/document.py`

### Refinement

Document processors and optional model-based repairs

- [Core document processor suite](https://github.com/datalab-to/marker/tree/master/marker/processors): service · `marker/processors`
- [Table reconstruction processor](https://github.com/datalab-to/marker/blob/master/marker/processors/table_recon.py): service · `marker/processors/table_recon.py`
- [LLM enhancement coordinator](https://github.com/datalab-to/marker/blob/master/marker/processors/llm/llm_meta.py): service · `marker/processors/llm/llm_meta.py`
- [Targeted LLM block repairs](https://github.com/datalab-to/marker/tree/master/marker/processors/llm): service · `marker/processors/llm`
- [OpenRouter LLM adapter](https://github.com/datalab-to/marker/blob/master/marker/services/openrouter.py): service · `marker/services/openrouter.py`

### Inference & Delivery

External inference, rendering, and API output

- **Surya VLM inference server**: external service
- **OpenRouter API**: external service
- [Renderer family](https://github.com/datalab-to/marker/tree/master/marker/renderers): service · `marker/renderers`
- [Rendered-output normalizer](https://github.com/datalab-to/marker/blob/master/marker/output.py): service · `marker/output.py`

## Connections

- Command-line user → Batch CLI command: invokes batch
- Command-line user → Single-file CLI command: invokes conversion
- Python library caller → Model artifact manager: creates artifacts
- Python library caller → PDF conversion coordinator: invokes conversion
- HTTP API client → HTTP conversion API: submits request
- HTTP conversion API → Configuration parser: parses configuration
- HTTP conversion API → Model artifact manager: manages artifacts
- HTTP conversion API → PDF conversion coordinator: invokes converter
- HTTP conversion API → Rendered-output normalizer: extracts output
- HTTP conversion API → HTTP API client: returns result
- Batch CLI command → PDF conversion coordinator: runs conversions
- Single-file CLI command → PDF conversion coordinator: runs conversion
- PDF conversion coordinator → Source document providers: ingests source
- Table-only conversion coordinator → Source document providers: ingests source
- PDF conversion coordinator → Document-construction pipeline: builds document
- Document-construction pipeline → Layout analysis builder: analyzes layout
- Document-construction pipeline → OCR builder: builds OCR
- Document-construction pipeline → Structure builder: assembles structure
- Document-construction pipeline → Document/block graph: constructs graph
- Layout analysis builder → Surya VLM inference server: requests layout
- OCR builder → Surya VLM inference server: requests OCR
- Table reconstruction processor → Surya VLM inference server: requests fallback
- PDF conversion coordinator → Core document processor suite: applies transformations
- Core document processor suite → Table reconstruction processor: includes reconstruction
- PDF conversion coordinator → LLM enhancement coordinator: enables LLM processing
- LLM enhancement coordinator → Targeted LLM block repairs: groups repairs
- LLM enhancement coordinator → OpenRouter LLM adapter: invokes configured service
- Targeted LLM block repairs → Document/block graph: updates blocks
- OpenRouter LLM adapter → OpenRouter API: sends requests
- PDF conversion coordinator → Renderer family: renders output