datalab-to/marker

1. Purpose and principal workflow

Marker converts local documents into Markdown, JSON, HTML, or chunks, preserving structured content such as tables, equations, forms, images, and references. A command-line user, Python library caller, or HTTP API client initiates the main workflow by supplying a document and options; Marker builds a document/block representation, conditionally uses local Surya VLM inference and optional cloud LLM refinement, renders the result, and returns or saves formatted output.

2. Components

Invocation and delivery

• Command-line user — null repository path — Runs the installed conversion commands described in the README.

• Python library caller — null repository path — Constructs converters and model artifacts directly, as shown in the documented Python API.

• HTTP API client — null repository path — Sends a file path or multipart upload to the conversion API.

• Batch CLI command — marker/scripts/convert.py — Provides the installed marker command for multi-file conversion, including worker and resume-oriented options documented in the README.

• Single-file CLI command — marker/scripts/convert_single.py — Provides the installed marker_single command for one input document and selectable output format.

• Interactive Streamlit launcher — marker/scripts/run_streamlit_app.py — Provides the documented marker_gui interactive entry point.

• HTTP conversion API — marker/scripts/server.py — A FastAPI application that initializes shared models, accepts /marker path requests and /marker/upload file requests, constructs a PDF converter, base64-encodes extracted images, and removes uploaded temporary files.

• Configuration parser — marker/config/parser.py — Converts request or user configuration into converter settings, processor selection, renderer selection, and optional LLM-service selection.

• Model artifact manager — marker/models.py — Supplies the model-artifact dictionary used by converters; the API creates it once during lifespan and shuts it down on exit.

Document construction and recognition

• PDF conversion coordinator — marker/converters/pdf.py — The documented primary converter builds a document from an input path, applies configured processing, and produces a rendered result.

• Table-only conversion coordinator — marker/converters/table.py — A documented alternative pipeline that extracts/converts tables rather than the complete PDF.

• Source document providers — marker/providers/__init__.py — Defines the provider contract for page images, page geometry, text lines, and references; the tree contains providers for PDF, image, HTML, EPUB, PowerPoint, spreadsheets, and documents.

• Document-construction pipeline — marker/builders/document.py — Represents the builder layer used to assemble a document from provider data and recognition outputs.

• Layout analysis builder — marker/builders/layout.py — Owns page-layout detection, which the README says uses Surya VLM in balanced mode and a lightweight detector in fast mode.

• OCR builder — marker/builders/ocr.py — Owns OCR/text-repair construction for scanned, garbled, forced-OCR, and relevant math content.

• Structure builder — marker/builders/structure.py — Owns assembly of recognized page content into structural document blocks.

• Document/block graph — marker/schema/document.py — Holds the mutable document hierarchy of pages and nested blocks that processors and callers can inspect or modify.

• Table reconstruction processor — marker/processors/table_recon.py — Represents the table-specific reconstruction stage; README documentation says it uses PDF text-layer reconstruction and can fall back to VLM inference.

Refinement and output

• Core document processor suite — marker/processors — Groups non-LLM transformations for artifacts and semantics, including headers/footers, lists, code, equations, references, tables, text, and table merging.

• LLM enhancement coordinator — marker/processors/llm/llm_meta.py — Consolidates simple LLM block processors so they share the configured LLM service rather than being independently inserted into the processing sequence.

• Targeted LLM block repairs — marker/processors/llm — Contains optional image-and-text refinements for complex regions, equations, forms, handwriting, tables, metadata, page correction, and section headers; sampled implementations validate responses before replacing block HTML and record error metadata on failures.

• OpenRouter LLM adapter — marker/services/openrouter.py — Implements an optional OpenAI-compatible OpenRouter client, converts images to data URLs, requests schema-constrained responses, retries timeouts/rate limits, and records token/request metadata on blocks.

• OpenRouter API — null repository path — External OpenAI-compatible endpoint used only when the OpenRouter adapter is selected.

• Surya VLM inference server — null repository path — External local inference service documented for layout, OCR, and table-recognition work; it may be auto-spawned or addressed through an existing inference URL.

• Renderer family — marker/renderers — Renders the processed document graph into Markdown, JSON, HTML, OCR JSON, or chunks.

• Rendered-output normalizer — marker/output.py — Extracts text and images from a renderer result for callers such as the HTTP API.

3. Relationships

• Command-line user -> Batch CLI command: invokes batch conversion; evidence: installed command registration and README batch usage.

• Command-line user -> Single-file CLI command: invokes one-document conversion; evidence: installed command registration and README single-file usage.

• Python library caller -> Model artifact manager: creates artifacts before converter construction; evidence: documented Python example.

• Python library caller -> PDF conversion coordinator: directly invokes conversion or document building; evidence: documented Python example.

• HTTP API client -> HTTP conversion API: submits conversion requests; evidence: /marker and /marker/upload routes.

• HTTP conversion API -> Configuration parser: translates request fields into configuration and selected collaborators; evidence: _convert_pdf.

• HTTP conversion API -> Model artifact manager: initializes and later shuts down shared artifacts; evidence: FastAPI lifespan handler.

• HTTP conversion API -> PDF conversion coordinator: instantiates and invokes the converter with parsed configuration and shared artifacts; evidence: _convert_pdf.

• HTTP conversion API -> Rendered-output normalizer: extracts text and images from the rendered result; evidence: text_from_rendered call.

• HTTP conversion API -> HTTP API client: returns output, metadata, encoded images, and success/error status; evidence: route return values.

• Batch CLI command -> PDF conversion coordinator: runs configured document conversions; evidence: documented marker behavior.

• Single-file CLI command -> PDF conversion coordinator: runs configured conversion for one input; evidence: documented marker_single behavior.

• PDF conversion coordinator -> Source document providers: ingests supported input formats into page-level source data; evidence: documented supported formats and provider family.

• Table-only conversion coordinator -> Source document providers: ingests source documents for table extraction; evidence: documented table converter behavior.

• PDF conversion coordinator -> Document-construction pipeline: builds the mutable document representation; evidence: documented build_document API.

• Document-construction pipeline -> Layout analysis builder: owns the layout-analysis stage; evidence: builder organization.

• Document-construction pipeline -> OCR builder: owns OCR/text construction; evidence: builder organization.

• Document-construction pipeline -> Structure builder: owns structural assembly; evidence: builder organization.

• Document-construction pipeline -> Document/block graph: constructs pages and nested blocks; evidence: documented document/block model.

• Layout analysis builder -> Surya VLM inference server: conditionally requests VLM layout in balanced mode; evidence: README mode behavior.

• OCR builder -> Surya VLM inference server: conditionally requests VLM OCR or repair; evidence: README OCR behavior.

• Table reconstruction processor -> Surya VLM inference server: conditionally falls back for low-confidence or scanned tables; evidence: README table behavior.

• PDF conversion coordinator -> Core document processor suite: applies configured document transformations; evidence: converter configuration accepts processor lists.

• Core document processor suite -> Table reconstruction processor: includes table reconstruction as a specialized processing capability; evidence: processor organization.

• PDF conversion coordinator -> LLM enhancement coordinator: enables grouped LLM processing when LLM processors are configured; evidence: base-converter initialization logic.

• LLM enhancement coordinator -> Targeted LLM block repairs: groups selected simple LLM processors for shared execution; evidence: meta-processor construction.

• LLM enhancement coordinator -> OpenRouter LLM adapter: conditionally invokes it when it is the configured LLM service; evidence: injected llm_service wiring.

• Targeted LLM block repairs -> Document/block graph: replace validated block HTML or record LLM error metadata; evidence: sampled equation, form, complex-region, and handwriting processors.

• OpenRouter LLM adapter -> OpenRouter API: sends image-plus-prompt chat-completion requests and parses schema-shaped responses; evidence: adapter client call.

• PDF conversion coordinator -> Renderer family: produces a rendered output in the selected format; evidence: documented converter output formats.

4. Coverage limits

The sampled source omits the implementations of the CLI handlers, PDF converter, configuration parser, concrete providers, builders, renderer internals, default processor order, and model lifecycle. Therefore, this map distinguishes documented stages from verified API wiring but cannot establish their precise runtime order, all mode-specific branches, all storage behavior of CLI output, or which non-OpenRouter LLM adapter is selected for a given configuration.