How datalab-to/lift works
lift is a document-intelligence library and toolset that turns PDF or image pages into schema-shaped JSON. The main workflow resolves and validates a user schema, renders document pages as images, dispatches generation through either the local HuggingFace model or an OpenAI-compatible vLLM endpoint, and parses model output. The CLI and Schema Studio provide distinct ways to invoke extraction; Studio also builds and saves schemas and presents test-run results. Edges are limited to calls visible in excerpts, the source index, or README; several CLI and app implementation details were not sampled.
Main entry points: vllm.py vLLM backend · extract.py Extraction API · __init__.py Inference dispatch · hf.py HuggingFace backend
Components 19
User entry points
- Extraction CLI
lift/scripts/extract_cli.py - Schema Studio
lift/scripts/app.py
Schema authoring
- Schema resolution
lift/extract.py - Schema builder
lift/schema_builder.py - Saved schemas
lift/schemas - Bundled schemas
lift/schemas
Extraction runtime
- Extraction API
lift/extract.py - Document rendering
lift/input.py - Batch contracts
lift/model/schema.py - Runtime settings
lift/settings.py - Output parsing
lift/output.py - Extraction result
lift/model/schema.py
Inference backends
- Inference dispatch
lift/model/__init__.py - HuggingFace backend
lift/model/hf.py - vLLM backend
lift/model/vllm.py - Image preparation
lift/model/util.py - Prompt templates
lift/prompts.py
Other components
- Developer
- vLLM server
Connections 26
- Developer → Extraction CLI: runs
README.md - Developer → Schema Studio: opens
README.md - Extraction CLI → Extraction API: invokes
lift/scripts/extract_cli.py - Schema Studio → Extraction API: tests extraction
lift/scripts/app.py - Schema Studio → Document rendering: loads pages
lift/scripts/app.py - Schema Studio → Schema builder: builds schemas
lift/scripts/app.py - Extraction API → Document rendering: loads document
lift/extract.py - Extraction API → Schema resolution: resolves schema
lift/extract.py - Schema resolution → Schema builder: validates and loads
lift/extract.py - Schema builder → Saved schemas: reads and writes
lift/schema_builder.py - Extraction API → Batch contracts: creates batch
lift/extract.py - Extraction API → Inference dispatch: generates
lift/extract.py - Inference dispatch → HuggingFace backend: dispatches
lift/model/__init__.py - Inference dispatch → vLLM backend: dispatches
lift/model/__init__.py - HuggingFace backend → Batch contracts: consumes batch
lift/model/hf.py - vLLM backend → Batch contracts: consumes batch
lift/model/vllm.py - HuggingFace backend → Image preparation: scales images
lift/model/hf.py - vLLM backend → Image preparation: scales images
lift/model/vllm.py - HuggingFace backend → Prompt templates: uses prompts
lift/model/hf.py - vLLM backend → Prompt templates: uses prompts
lift/model/vllm.py - HuggingFace backend → Runtime settings: loads configuration
lift/model/hf.py - vLLM backend → Runtime settings: loads configuration
lift/model/vllm.py - vLLM backend → vLLM server: sends requests
lift/model/vllm.py - Inference dispatch → Output parsing: parses raw output
lift/model/__init__.py - Inference dispatch → Extraction result: returns results
lift/model/__init__.py - Saved schemas → Bundled schemas: includes
README.md