How datalab-to/chandra works
1. What this repository is
- chandra is a Python OCR/document-intelligence package for turning images and PDFs into structured text outputs: Markdown, HTML, and JSON.
- The core architectural idea is a document-to-structure pipeline, not just plain OCR: it preserves layout, tables, math, forms, images, and reading order.
2. Primary execution modes
- There are two inference backends, and they define the main runtime boundary:
- HuggingFace local inference for in-process model execution.
- vLLM remote/server inference for faster or production-style deployment.
- The README makes vLLM the default path, with HuggingFace as an alternative backend.
3. Core library boundary
- The reusable logic lives under chandra/.
- This is where the repository separates concerns into:
- input handling,
- prompt construction,
- model backend adapters,
- output formatting,
- settings/configuration,
- utility helpers.
- That separation is important for a diagram: CLI, app, and server entrypoints are thin wrappers around this core package.
4. Input subsystem
- chandra/input.py is the likely front door for files and document sources.
- Based on the README, it supports single files and directories, PDF page ranges, batching, and optional extraction of embedded images.
- This subsystem is responsible for normalizing document sources into per-page work units before inference.
5. Prompting and task shaping
- chandra/prompts.py is the layer that converts a page or image into the instruction format expected by the model.
- This is where document-intelligence specifics live: asking the model to produce structured HTML/Markdown/JSON and preserve layout relationships.
- In architecture terms, this sits between normalized input pages and the model backend.
6. Model abstraction layer
- chandra/model/ is the main backend abstraction boundary.
- hf.py and vllm.py implement the two inference paths, while schema.py and util.py support request/response shaping and backend-specific helpers.
- This layer is what keeps the rest of the system backend-agnostic: the higher layers should not care whether inference is local or served over an API.
7. Output subsystem
- chandra/output.py is where inference results become durable artifacts.
- The README indicates each processed document emits a directory containing:
- Markdown output,
- HTML output,
- metadata JSON,
- and extracted images.
- This is the persistence boundary of the pipeline: model outputs are transformed into user-facing files and metadata.
8. Configuration and runtime settings
- chandra/settings.py and environment variables define runtime configuration such as model checkpoint, max output tokens, and vLLM endpoint details.
- The README also mentions optional local.env support.
- In a diagram, this is the control plane for deployment-specific behavior rather than document content flow.
9. User-facing entrypoints
- chandra/scripts/ contains the executable surfaces:
- cli.py for command-line document processing,
- app.py and run_app.py for the Streamlit app,
- screenshot_app.py plus the HTML template for visual capture/debugging,
- vllm.py for launching the vLLM server/container workflow.
- These are orchestration layers over the core library, not the core OCR logic itself.
10. Server/deployment path
- The vllm script indicates a deployment boundary where inference can be served as an API rather than embedded locally.
- The README points to VLLM_API_BASE, VLLM_MODEL_NAME, and VLLM_GPUS, so the remote inference mode is explicitly configured and isolated from the CLI/app frontends.
- This is the main scaling boundary for production or batch workloads.
11. Data flow through the system
- Typical flow is:
- user provides a PDF/image or a directory,
- input is normalized into pages,
- prompts are built for each page,
- a backend runs inference,
- outputs are parsed/structured,
- final files are written per source document.
- Optional branches include image extraction and inclusion/exclusion of headers and footers.
12. Testing and validation
- tests/integration/test_image_inference.py shows at least one integration test path focused on end-to-end image inference.
- This suggests the repository values backend-level correctness, not only unit tests.
- tests/conftest.py likely centralizes fixtures for document/image inputs and backend setup.
13. CI/CD and release flow
- .github/workflows/integration.yml and publish.yml indicate automated validation and package publishing.
- The presence of pyproject.toml, uv.lock, and .python-version suggests a modern Python packaging/runtime workflow centered on reproducible installs.
14. External dependencies that matter architecturally
- The repo’s main external integration points are HuggingFace Transformers and vLLM.
- Those are not incidental libraries; they define the two primary inference architectures and directly affect latency, hardware requirements, and deployment shape.
15. How to draw the architecture
- A good diagram would show:
- CLI/Streamlit/vLLM launcher on the left,
- the shared chandra core in the middle,
- input/prompt/model/output/config as internal stages,
- and two interchangeable backend boxes for HF and vLLM.
- Output artifacts should be shown as a sink on the right: Markdown, HTML, JSON metadata, and extracted images.
Main entry points: __init__.py Model API · input.py Input · vllm.py vLLM · test_image_inference.py Integration test
Components 18
Entrypoints
User-facing command, app, and server launch surfaces.
- CLI · Command-line document processing entrypoint.
chandra/scripts/cli.py - App · Streamlit UI entrypoint.
chandra/scripts/app.py - Run app · Helper that starts the Streamlit application.
chandra/scripts/run_app.py - Screenshot · Visual capture and debugging surface for the app.
chandra/scripts/screenshot_app.py - VLLM script · Launches or wires up the vLLM server workflow.
chandra/scripts/vllm.py
Core pipeline
Document normalization, prompting, output, and runtime settings.
- Input · Normalizes files, directories, page ranges, and embedded images into page work units.
chandra/input.py - Prompts · Builds page-level instructions for structured document output.
chandra/prompts.py - Output · Writes Markdown, HTML, metadata JSON, and extracted images.
chandra/output.py - Settings · Runtime configuration and environment-driven defaults.
chandra/settings.py - Util · Shared utility helpers used across the package.
chandra/util.py
Model layer
Backend abstraction and inference implementations.
- Model API · Backend-agnostic interface for model execution.
chandra/model/__init__.py - HuggingFace · In-process inference path using Transformers/HuggingFace models.
chandra/model/hf.py - vLLM · Server-based inference path for production or scaled execution.
chandra/model/vllm.py - Schema · Shared request and response shaping for backend calls.
chandra/model/schema.py - Model util · Backend-specific helpers and normalization logic.
chandra/model/util.py
Quality & release
Tests and automation that validate and ship the package.
- Integration test · End-to-end image inference coverage.
tests/integration/test_image_inference.py - Fixtures · Shared testing fixtures for inputs and backend setup.
tests/conftest.py - Workflows · Automated integration validation and package publishing.
.github/workflows/integration.yml
Connections 22
- CLI → Input: loads docs · CLI begins by normalizing document sources.
- App → Input: loads docs · Streamlit app feeds the same input pipeline.
- Run app → App: starts · Launcher hands control to the app UI.
- Screenshot → App: captures · Screenshot tooling wraps the app for visual debugging.
- VLLM script → vLLM: starts server · Deployment script exposes the vLLM backend.
- Input → Prompts: pages · Normalized pages are converted into model instructions.
- Prompts → Model API: requests · Prompted page content enters the backend abstraction.
- Model API → HuggingFace: local path · Facade can dispatch to local HuggingFace inference.
- Model API → vLLM: remote path · Facade can dispatch to the server-backed vLLM path.
- HuggingFace → Schema: uses · Local backend shares request/response schema.
- vLLM → Schema: uses · Remote backend shares request/response schema.
- HuggingFace → Model util: helpers · Local backend relies on backend-specific utilities.
- vLLM → Model util: helpers · Remote backend relies on backend-specific utilities.
- Model API → Output: structured text · Backend results are persisted as user-facing artifacts.
- Settings → Model API: configures · Runtime settings select backend and inference behavior.
- Settings → VLLM script: endpoint config · Server launcher is driven by vLLM deployment settings.
- Util → Input: supports · Shared utilities assist document handling.
- Util → Output: supports · Shared utilities assist artifact writing.
- Integration test → Input: validates · Integration test exercises document ingestion and page handling.
- Integration test → Model API: validates · Integration test reaches the inference boundary.
- Fixtures → Integration test: fixtures · Shared fixtures support integration coverage.
- Workflows → Integration test: runs · CI workflow executes the integration test path.