How vikparuchuri/marker works
Main Components
• Input/Providers – Components that ingest documents in various formats. Look at the “marker/providers” directory which includes providers for PDF, image, EPUB, HTML, etc.
• Builders – These components (in “marker/builders”) generate the initial document blocks using extracted text and layouts.
• Processors – Located in “marker/processors,” these refine blocks. They handle tasks like formatting tables, handling equations, inline math, references, and OCR.
• Renderers – In “marker/renderers,” these convert processed blocks into final output formats (markdown, HTML, JSON).
• Converters – Found in “marker/converters,” the converters (e.g., PdfConverter and TableConverter) integrate the whole pipeline by orchestrating providers, builders, processors, and renderers.
• Schema – In “marker/schema,” this defines and structures the blocks and components used across the pipeline.
• LLM Services Integration – Marker supports various external LLM services (see “marker/services”) to enhance conversion accuracy when the “--use_llm” flag is enabled.
• API Server & Interactive App – Marker exposes functionality through a FastAPI-based server (marker_server.py) and an interactive Streamlit app (via marker_app.py) for real-time document conversion.
• Utility and Configuration – The project uses configuration (e.g., marker/config and pyproject.toml), logging, and utility scripts for tasks such as file conversion and benchmarking.
• Testing & Benchmarks – The “tests” and “benchmarks” directories contain unit tests and benchmark scripts that validate performance and conversion quality.
Relationships and Data Flow
• A top-level “Document Conversion Pipeline.” Start with the “Input” layer (Providers) where documents are ingested.
• Flow from Providers to Builders: the provider extracts raw content (text, images, layout details) that the builder then organizes into a structured document model (using schema definitions).
• Finally, Renderers take the processed document blocks and output them in different formats.
• Optionally – include branches for the LLM service: an additional feedback loop where, if “--use_llm” is specified, the document or specific blocks are sent to an LLM service (Gemini, Claude, Vertex, Ollama, or OpenAI) to enhance accuracy.
• The API Server and Interactive App are depicted as front-end interfaces that accept files and invoke the conversion pipeline.
Architectural Patterns and Design Principles
• Modular Pipeline Architecture – Each stage of processing (providers, builders, processors, renderers) is encapsulated in its own module, which makes the design extensible.
• Separation of Concerns – The directory structure clearly separates functionality (conversion logic from API serving, testing, and benchmarking).
• Configuration Driven – Using configuration files and parsers (e.g., marker/config/parser.py) for runtime behavior.
• Integration with External Services – Incorporate LLM services integration as an optional enhancement layer.
Technologies and Frameworks
• Python 3.10+, PyTorch – For deep learning models and backend inference.
• Streamlit and FastAPI – For the interactive app and server respectively.
• GitHub Actions – Indicated by the .github/workflows directory for CI/CD processes.
• Testing frameworks (pytest) are used for validating functionality.
• Various internal utilities for logging, file conversion (e.g., marker/scripts), and benchmark processing.