How katanaml/sparrow works
Main Components
• Sparrow Data:
– Contains two primary submodules: OCR (sparrow-data/ocr) and Parse (sparrow-data/parse).
– OCR service handles optical character recognition tasks.
– Parse module implements the Sparrow Parse pipeline, offering support for both local and cloud-based vision-language (VL) model inferencing (via vLLM, Hugging Face backends, MLX, etc.), document pre-processing (cropping), text extraction, table processors, and even PDF page segmentation.
• Sparrow ML:
– Encompasses the LLM engine and Agents.
– The “llm” submodule (sparrow-ml/llm) contains the core pipelines (for example, instructor pipelines and pipelined integrations with Sparrow Parse) as well as configuration, database pool management, and other runtime utilities.
– The “agents” submodule (sparrow-ml/agents) implements the agent framework that orchestrates and manages workflows using Prefect (for visual monitoring), routing through different document processing steps (e.g., medical prescriptions, trading).
• Sparrow UI:
– Provides the dashboard and shell scripts that act as a front end to the Sparrow services.
– Contains UI assets, FastAPI-based API endpoints (e.g., the API in sparrow-data/ocr/api.py and the FastAPI endpoint instructions in the README) which enable interactive HTTP access to the Sparrow LLM and data extraction pipelines.
• Configuration & Integration Components:
– Various configuration files (config.properties, requirements files, Dockerfile, etc.) indicate how each module is deployed and how they interact.
– The overall API design supports both command-line interface (CLI) and HTTP endpoints for running inference pipelines and data extraction flows.
Relationships and Data Flows
• Clients (or external systems) initiate requests via:
– CLI scripts (using sparrow.sh) that pass JSON queries and file-paths.
– HTTP API endpoints (running FastAPI) that receive multipart forms containing queries, pipelines, options, and documents.
• Requests are then routed to the Sparrow Parse pipeline (or other pipelines like “instructor” or “stocks”) located in sparrow-data/parse.
• Depending on the request, the document may first pass through the OCR service (sparrow-data/ocr) before further processing.
• The processed data is then forwarded to the Sparrow ML LLM components (in sparrow-ml/llm), which may invoke different pipelines and run further LLM-based operations.
• For orchestrated workflows (using the agent framework), the Sparrow Agents component (sparrow-ml/agents) coordinates additional tasks like document classification, custom data extractions, or even trading/medical use cases.
• Finally, results are returned to the client via the API, or made visible on the Sparrow UI dashboard.
Architectural Patterns and Technologies
• Modularity & Pluggable Architecture:
– Each pipeline (OCR, Parse, ML LLM, Agents) is self-contained and can be developed, deployed, or scaled independently.
– The pluggable design lets you replace or extend components easily (e.g., swap out a local MLX inference engine for a cloud GPU backend).
• Microservices-Like Separation (even if deployed monolithically):
– Separation between data ingestion (OCR/Parse) and processing (LLM/Agents), with clear boundaries.
• API-first Approach:
– The FastAPI endpoint provides a RESTful interface to the pipelines, enhancing integration flexibility.
• Extensibility via Agents:
– The agent system using Prefect for orchestration enables complex workflows that integrate multiple services.
Final Recommendations for the Principal Software Engineer
– Another could zoom in on the Agents framework and how it orchestrates workflows with Prefect.
• Incorporate any relevant technology notes (e.g., Python 3.10+, FastAPI, Docker, and specific libraries required by MLX or Hugging Face backends) to give context to the infrastructure.