# datalab-to/lift architecture

> Architecture diagram of the GitHub repository datalab-to/lift, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/datalab-to/lift
- Repository: https://github.com/datalab-to/lift

## Overview

lift is a document-intelligence library and toolset that turns PDF or image pages into schema-shaped JSON. The main workflow resolves and validates a user schema, renders document pages as images, dispatches generation through either the local HuggingFace model or an OpenAI-compatible vLLM endpoint, and parses model output. The CLI and Schema Studio provide distinct ways to invoke extraction; Studio also builds and saves schemas and presents test-run results. Edges are limited to calls visible in excerpts, the source index, or README; several CLI and app implementation details were not sampled.

Main entry points: [`lift/model/vllm.py`](https://github.com/datalab-to/lift/blob/master/lift/model/vllm.py) (vLLM backend), [`lift/extract.py`](https://github.com/datalab-to/lift/blob/master/lift/extract.py) (Extraction API), [`lift/model/__init__.py`](https://github.com/datalab-to/lift/blob/master/lift/model/__init__.py) (Inference dispatch), [`lift/model/hf.py`](https://github.com/datalab-to/lift/blob/master/lift/model/hf.py) (HuggingFace backend)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User entry points"]
  node_cli["Extraction CLI<br/>[extract_cli.py]"]
  node_studio["Schema Studio<br/>[app.py]"]
end

subgraph group_schema["Schema authoring"]
  node_schema_resolve["Schema resolution<br/>[extract.py]"]
  node_schema_builder["Schema builder<br/>[schema_builder.py]"]
  node_schema_library[("Saved schemas")]
  node_schema_files["Bundled schemas"]
end

subgraph group_runtime["Extraction runtime"]
  node_extract["Extraction API<br/>[extract.py]"]
  node_input["Document rendering<br/>[input.py]"]
  node_batch_types["Batch contracts<br/>[schema.py]"]
  node_settings["Runtime settings<br/>[settings.py]"]
  node_output_parser["Output parsing<br/>[output.py]"]
  node_results["Extraction result<br/>[schema.py]"]
end

subgraph group_inference["Inference backends"]
  node_inference_manager["Inference dispatch<br/>[__init__.py]"]
  node_hf_backend["HuggingFace backend<br/>[hf.py]"]
  node_vllm_backend["vLLM backend<br/>[vllm.py]"]
  node_image_utils["Image preparation<br/>[util.py]"]
  node_prompt_templates["Prompt templates<br/>[prompts.py]"]
end

node_user(("Developer"))
node_remote_server["vLLM server"]

node_user -->|"runs"| node_cli
node_user -->|"opens"| node_studio
node_cli -->|"invokes"| node_extract
node_studio -->|"tests extraction"| node_extract
node_studio -->|"loads pages"| node_input
node_studio -->|"builds schemas"| node_schema_builder
node_extract -->|"loads document"| node_input
node_extract -->|"resolves schema"| node_schema_resolve
node_schema_resolve -->|"validates and loads"| node_schema_builder
node_schema_builder -->|"reads and writes"| node_schema_library
node_extract -->|"creates batch"| node_batch_types
node_extract -->|"generates"| node_inference_manager
node_inference_manager -->|"dispatches"| node_hf_backend
node_inference_manager -->|"dispatches"| node_vllm_backend
node_hf_backend -->|"consumes batch"| node_batch_types
node_vllm_backend -->|"consumes batch"| node_batch_types
node_hf_backend -->|"scales images"| node_image_utils
node_vllm_backend -->|"scales images"| node_image_utils
node_hf_backend -->|"uses prompts"| node_prompt_templates
node_vllm_backend -->|"uses prompts"| node_prompt_templates
node_hf_backend -->|"loads configuration"| node_settings
node_vllm_backend -->|"loads configuration"| node_settings
node_vllm_backend -->|"sends requests"| node_remote_server
node_inference_manager -->|"parses raw output"| node_output_parser
node_inference_manager -->|"returns results"| node_results
node_schema_library -.->|"includes"| node_schema_files

click node_cli "https://github.com/datalab-to/lift/blob/master/lift/scripts/extract_cli.py"
click node_studio "https://github.com/datalab-to/lift/blob/master/lift/scripts/app.py"
click node_extract "https://github.com/datalab-to/lift/blob/master/lift/extract.py"
click node_input "https://github.com/datalab-to/lift/blob/master/lift/input.py"
click node_schema_resolve "https://github.com/datalab-to/lift/blob/master/lift/extract.py"
click node_schema_builder "https://github.com/datalab-to/lift/blob/master/lift/schema_builder.py"
click node_schema_library "https://github.com/datalab-to/lift/tree/master/lift/schemas"
click node_batch_types "https://github.com/datalab-to/lift/blob/master/lift/model/schema.py"
click node_inference_manager "https://github.com/datalab-to/lift/blob/master/lift/model/__init__.py"
click node_hf_backend "https://github.com/datalab-to/lift/blob/master/lift/model/hf.py"
click node_vllm_backend "https://github.com/datalab-to/lift/blob/master/lift/model/vllm.py"
click node_image_utils "https://github.com/datalab-to/lift/blob/master/lift/model/util.py"
click node_prompt_templates "https://github.com/datalab-to/lift/blob/master/lift/prompts.py"
click node_settings "https://github.com/datalab-to/lift/blob/master/lift/settings.py"
click node_output_parser "https://github.com/datalab-to/lift/blob/master/lift/output.py"
click node_results "https://github.com/datalab-to/lift/blob/master/lift/model/schema.py"
click node_schema_files "https://github.com/datalab-to/lift/tree/master/lift/schemas"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_studio toneBlue
class node_schema_resolve,node_schema_builder,node_schema_library,node_schema_files toneAmber
class node_extract,node_input,node_batch_types,node_settings,node_output_parser,node_results,node_remote_server toneMint
class node_inference_manager,node_hf_backend,node_vllm_backend,node_image_utils,node_prompt_templates toneRose
class node_user toneIndigo
```

## Components

### User entry points

- [Extraction CLI](https://github.com/datalab-to/lift/blob/master/lift/scripts/extract_cli.py): `lift/scripts/extract_cli.py`
- [Schema Studio](https://github.com/datalab-to/lift/blob/master/lift/scripts/app.py): `lift/scripts/app.py`

### Schema authoring

- [Schema resolution](https://github.com/datalab-to/lift/blob/master/lift/extract.py): `lift/extract.py`
- [Schema builder](https://github.com/datalab-to/lift/blob/master/lift/schema_builder.py): `lift/schema_builder.py`
- [Saved schemas](https://github.com/datalab-to/lift/tree/master/lift/schemas): `lift/schemas`
- [Bundled schemas](https://github.com/datalab-to/lift/tree/master/lift/schemas): `lift/schemas`

### Extraction runtime

- [Extraction API](https://github.com/datalab-to/lift/blob/master/lift/extract.py): `lift/extract.py`
- [Document rendering](https://github.com/datalab-to/lift/blob/master/lift/input.py): `lift/input.py`
- [Batch contracts](https://github.com/datalab-to/lift/blob/master/lift/model/schema.py): `lift/model/schema.py`
- [Runtime settings](https://github.com/datalab-to/lift/blob/master/lift/settings.py): `lift/settings.py`
- [Output parsing](https://github.com/datalab-to/lift/blob/master/lift/output.py): `lift/output.py`
- [Extraction result](https://github.com/datalab-to/lift/blob/master/lift/model/schema.py): `lift/model/schema.py`

### Inference backends

- [Inference dispatch](https://github.com/datalab-to/lift/blob/master/lift/model/__init__.py): `lift/model/__init__.py`
- [HuggingFace backend](https://github.com/datalab-to/lift/blob/master/lift/model/hf.py): `lift/model/hf.py`
- [vLLM backend](https://github.com/datalab-to/lift/blob/master/lift/model/vllm.py): `lift/model/vllm.py`
- [Image preparation](https://github.com/datalab-to/lift/blob/master/lift/model/util.py): `lift/model/util.py`
- [Prompt templates](https://github.com/datalab-to/lift/blob/master/lift/prompts.py): `lift/prompts.py`

### Other components

- **Developer**
- **vLLM server**

## Connections

- Developer → Extraction CLI: runs (evidence: `README.md`)
- Developer → Schema Studio: opens (evidence: `README.md`)
- Extraction CLI → Extraction API: invokes (evidence: `lift/scripts/extract_cli.py`)
- Schema Studio → Extraction API: tests extraction (evidence: `lift/scripts/app.py`)
- Schema Studio → Document rendering: loads pages (evidence: `lift/scripts/app.py`)
- Schema Studio → Schema builder: builds schemas (evidence: `lift/scripts/app.py`)
- Extraction API → Document rendering: loads document (evidence: `lift/extract.py`)
- Extraction API → Schema resolution: resolves schema (evidence: `lift/extract.py`)
- Schema resolution → Schema builder: validates and loads (evidence: `lift/extract.py`)
- Schema builder → Saved schemas: reads and writes (evidence: `lift/schema_builder.py`)
- Extraction API → Batch contracts: creates batch (evidence: `lift/extract.py`)
- Extraction API → Inference dispatch: generates (evidence: `lift/extract.py`)
- Inference dispatch → HuggingFace backend: dispatches (evidence: `lift/model/__init__.py`)
- Inference dispatch → vLLM backend: dispatches (evidence: `lift/model/__init__.py`)
- HuggingFace backend → Batch contracts: consumes batch (evidence: `lift/model/hf.py`)
- vLLM backend → Batch contracts: consumes batch (evidence: `lift/model/vllm.py`)
- HuggingFace backend → Image preparation: scales images (evidence: `lift/model/hf.py`)
- vLLM backend → Image preparation: scales images (evidence: `lift/model/vllm.py`)
- HuggingFace backend → Prompt templates: uses prompts (evidence: `lift/model/hf.py`)
- vLLM backend → Prompt templates: uses prompts (evidence: `lift/model/vllm.py`)
- HuggingFace backend → Runtime settings: loads configuration (evidence: `lift/model/hf.py`)
- vLLM backend → Runtime settings: loads configuration (evidence: `lift/model/vllm.py`)
- vLLM backend → vLLM server: sends requests (evidence: `lift/model/vllm.py`)
- Inference dispatch → Output parsing: parses raw output (evidence: `lift/model/__init__.py`)
- Inference dispatch → Extraction result: returns results (evidence: `lift/model/__init__.py`)
- Saved schemas → Bundled schemas: includes (evidence: `README.md`)
