# katanaml/sparrow architecture

> Architecture diagram of the GitHub repository katanaml/sparrow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/katanaml/sparrow
- Repository: https://github.com/katanaml/sparrow

## Overview

##### Main Components
- Sparrow Data:
– Contains two primary submodules: OCR (sparrow-data/ocr) and Parse (sparrow-data/parse).
– OCR service handles optical character recognition tasks.
– Parse module implements the Sparrow Parse pipeline, offering support for both local and cloud-based vision-language (VL) model inferencing (via vLLM, Hugging Face backends, MLX, etc.), document pre-processing (cropping), text extraction, table processors, and even PDF page segmentation.

- Sparrow ML:
– Encompasses the LLM engine and Agents.
– The “llm” submodule (sparrow-ml/llm) contains the core pipelines (for example, instructor pipelines and pipelined integrations with Sparrow Parse) as well as configuration, database pool management, and other runtime utilities.
– The “agents” submodule (sparrow-ml/agents) implements the agent framework that orchestrates and manages workflows using Prefect (for visual monitoring), routing through different document processing steps (e.g., medical prescriptions, trading).

- Sparrow UI:
– Provides the dashboard and shell scripts that act as a front end to the Sparrow services.
– Contains UI assets, FastAPI-based API endpoints (e.g., the API in sparrow-data/ocr/api.py and the FastAPI endpoint instructions in the README) which enable interactive HTTP access to the Sparrow LLM and data extraction pipelines.

- Configuration & Integration Components:
– Various configuration files (config.properties, requirements files, Dockerfile, etc.) indicate how each module is deployed and how they interact.
– The overall API design supports both command-line interface (CLI) and HTTP endpoints for running inference pipelines and data extraction flows.

##### Relationships and Data Flows
- Clients (or external systems) initiate requests via:
– CLI scripts (using sparrow.sh) that pass JSON queries and file-paths.
– HTTP API endpoints (running FastAPI) that receive multipart forms containing queries, pipelines, options, and documents.
- Requests are then routed to the Sparrow Parse pipeline (or other pipelines like “instructor” or “stocks”) located in sparrow-data/parse.
- Depending on the request, the document may first pass through the OCR service (sparrow-data/ocr) before further processing.
- The processed data is then forwarded to the Sparrow ML LLM components (in sparrow-ml/llm), which may invoke different pipelines and run further LLM-based operations.
- For orchestrated workflows (using the agent framework), the Sparrow Agents component (sparrow-ml/agents) coordinates additional tasks like document classification, custom data extractions, or even trading/medical use cases.
- Finally, results are returned to the client via the API, or made visible on the Sparrow UI dashboard.

##### Architectural Patterns and Technologies
- Modularity & Pluggable Architecture:
– Each pipeline (OCR, Parse, ML LLM, Agents) is self-contained and can be developed, deployed, or scaled independently.
– The pluggable design lets you replace or extend components easily (e.g., swap out a local MLX inference engine for a cloud GPU backend).

- Microservices-Like Separation (even if deployed monolithically):
– Separation between data ingestion (OCR/Parse) and processing (LLM/Agents), with clear boundaries.

- API-first Approach:
– The FastAPI endpoint provides a RESTful interface to the pipelines, enhancing integration flexibility.

- Extensibility via Agents:
– The agent system using Prefect for orchestration enables complex workflows that integrate multiple services.

##### Final Recommendations for the Principal Software Engineer
– Another could zoom in on the Agents framework and how it orchestrates workflows with Prefect.
- Incorporate any relevant technology notes (e.g., Python 3.10+, FastAPI, Docker, and specific libraries required by MLX or Hugging Face backends) to give context to the infrastructure.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Clients
    subgraph "Clients"
        Client["Client: CLI/HTTP Request"]:::client
    end

    %% Sparrow UI & API
    subgraph "Sparrow UI & API"
        UI["Dashboard & FastAPI Endpoints (sparrow-ui)"]:::ui
        UI_Shell["Shell Scripts (sparrow-ui/shell)"]:::ui
        UI --> UI_Shell
    end

    %% Sparrow Data
    subgraph "Sparrow Data"
        %% OCR Service Subgroup
        subgraph "OCR Service"
            OCR["OCR Service (sparrow-data/ocr)"]:::data
            OCR_API["API Endpoint (sparrow-data/ocr/api.py)"]:::data
            OCR_Router["OCR Routers (sparrow-data/ocr/routers/ocr.py)"]:::data
            OCR --> OCR_API
            OCR --> OCR_Router
        end
        %% Document Parsing Subgroup
        subgraph "Parse Pipeline"
            Parse["Document Parsing (sparrow-data/parse)"]:::data
        end
    end

    %% Sparrow ML
    subgraph "Sparrow ML"
        %% LLM Pipeline Subgroup
        subgraph "LLM Pipeline"
            LLM["LLM Pipelines (sparrow-ml/llm)"]:::ml
            LLM_Engine["Engine (sparrow-ml/llm/engine.py)"]:::ml
            LLM_Pipelines["Pipelines Directory (sparrow-ml/llm/pipelines)"]:::ml
            LLM_Config["LLM Config (sparrow-ml/llm/config.properties)"]:::ml
            LLM --> LLM_Engine
            LLM --> LLM_Pipelines
            LLM --> LLM_Config
        end
        %% Agents Framework Subgroup
        subgraph "Agents Framework"
            Agents["Agent Workflows (sparrow-ml/agents)"]:::ml
            Agents_API["Agents API (sparrow-ml/agents/api.py)"]:::ml
            Agents_Base["Base Module (sparrow-ml/agents/base.py)"]:::ml
            Agents_Specific["Use-cases: Medical/Trading"]:::ml
            Agents --> Agents_API
            Agents --> Agents_Base
            Agents --> Agents_Specific
        end
    end

    %% Configuration & Integration
    subgraph "Configuration & Integration"
        Config["Core Config & Deployment Scripts"]:::config
        Dockerfile["Dockerfile (sparrow-ml/llm/Dockerfile)"]:::config
        Agents_Config["Agents Config (sparrow-ml/agents/config.properties)"]:::config
        UI_Config["UI Config (sparrow-ui/shell/config.properties)"]:::config
        Config --> Dockerfile
        Config --> Agents_Config
        Config --> UI_Config
    end

    %% Data Flow Connections
    Client -->|"initiates"| UI
    UI -->|"submits documents"| OCR
    UI -->|"submits documents"| Parse
    OCR -->|"extracts text"| Parse
    Parse -->|"processes data"| LLM
    LLM -->|"returns inference"| UI
    LLM -->|"triggers workflows"| Agents
    Agents -->|"orchestrates tasks"| LLM
    UI -->|"delivers output"| Client

    %% Configuration Reads
    UI ---|"reads"| Config
    OCR ---|"reads"| Config
    Parse ---|"reads"| Config
    LLM ---|"reads"| Config
    Agents ---|"reads"| Config

    %% Click Events
    click OCR "https://github.com/katanaml/sparrow/tree/main/sparrow-data/ocr"
    click OCR_API "https://github.com/katanaml/sparrow/blob/main/sparrow-data/ocr/api.py"
    click OCR_Router "https://github.com/katanaml/sparrow/blob/main/sparrow-data/ocr/routers/ocr.py"
    click Parse "https://github.com/katanaml/sparrow/tree/main/sparrow-data/parse"
    click LLM "https://github.com/katanaml/sparrow/tree/main/sparrow-ml/llm"
    click LLM_Engine "https://github.com/katanaml/sparrow/blob/main/sparrow-ml/llm/engine.py"
    click LLM_Pipelines "https://github.com/katanaml/sparrow/tree/main/sparrow-ml/llm/pipelines"
    click LLM_Config "https://github.com/katanaml/sparrow/blob/main/sparrow-ml/llm/config.properties"
    click Agents "https://github.com/katanaml/sparrow/tree/main/sparrow-ml/agents"
    click Agents_API "https://github.com/katanaml/sparrow/blob/main/sparrow-ml/agents/api.py"
    click Agents_Base "https://github.com/katanaml/sparrow/blob/main/sparrow-ml/agents/base.py"
    click UI "https://github.com/katanaml/sparrow/tree/main/sparrow-ui"
    click UI_Shell "https://github.com/katanaml/sparrow/blob/main/sparrow-ui/shell/app.py"
    click Dockerfile "https://github.com/katanaml/sparrow/tree/main/sparrow-ml/llm/Dockerfile"
    click Agents_Config "https://github.com/katanaml/sparrow/blob/main/sparrow-ml/agents/config.properties"
    click UI_Config "https://github.com/katanaml/sparrow/blob/main/sparrow-ui/shell/config.properties"

    %% Styling Classes
    classDef client fill:#F4D03F,stroke:#A04000,stroke-width:2px;
    classDef ui fill:#85C1E9,stroke:#1B4F72,stroke-width:2px;
    classDef data fill:#82E0AA,stroke:#196F3D,stroke-width:2px;
    classDef ml fill:#F7DC6F,stroke:#B9770E,stroke-width:2px;
    classDef config fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
```
