# sroussey/ellmers architecture

> Architecture diagram of the GitHub repository sroussey/ellmers, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sroussey/ellmers
- Repository: https://github.com/sroussey/ellmers

## Overview

##### Main System Components

A. Frontend Layer
- Web Interface (packages/web)
- CLI Interface (packages/cli)

B. Core Layer (packages/core)
- Task Management System
- Storage Systems
- Model Management
- Provider System
- Job Queue System

##### Key Architectural Elements to Highlight

a) Task System (Central to the Architecture):
- It to various task types (Text Generation, Embedding, QA, etc.)
- TaskGraph and TaskGraphRunner components that orchestrate execution

b) Storage Layer:
- Multiple storage backends:
- In-Memory
- SQLite
- PostgreSQL
- IndexedDB
- File System
- Separate repositories for:
- Task Graphs
- Task Outputs
- Key-Value Storage

c) Model Management:
- Supported model types:
- HuggingFace Models
- MediaPipe Models
- GGML Local Models
- Model storage and loading mechanisms

d) Job Queue System:
- Job queue implementations
- Rate limiters and concurrency controls
- To task execution flow

**┌─ User Interfaces ─────────────────────┐**
- Web UI          CLI                  │
- ┬────────────────────┘
- ↓
**┌─ Core Services ──────────────────────────────────┐**
- ┌─ Task Management ─┐  ┌─ Model Management ─┐   │
- Task Registry  │  │  - HuggingFace     │   │
- Task Graph     │  │  - MediaPipe       │   │
- Task Runner    │  │  - GGML Local      │   │
- ┘  └──────────────────────┘  │
- ┌─ Job Queue ───────┐  ┌─ Storage ──────────┐   │
- Rate Limiting  │  │  - Graph Store     │   │
- Concurrency    │  │  - Output Store    │   │
- ┘  │  - KV Store        │   │
- ┘   │
- ┘

##### Data Flow Indicators

Show:
- User input flow from UI to core
- Task execution flow through the system
- Storage operations
- Model loading and usage
- Job queue processing

##### Important Details

- Storage backend options for each repository type
- Supported model types and their providers
- Task types and their relationships
- Job queue implementation variants
- System extension points

##### Specific Implementation Notes

- How the task graph orchestration works
- Relationship between tasks and jobs
- Demonstrate how different storage backends can be swapped
- Document processing pipeline

Remember to:
- Keep components grouped logically
- Clear boundaries between subsystems
- Notes about scalability and extension points

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles
    classDef ui fill:#3498db,stroke:#2980b9,color:white
    classDef core fill:#2ecc71,stroke:#27ae60,color:white
    classDef storage fill:#f1c40f,stroke:#f39c12,color:black
    classDef model fill:#9b59b6,stroke:#8e44ad,color:white
    classDef queue fill:#e74c3c,stroke:#c0392b,color:white
    classDef provider fill:#1abc9c,stroke:#16a085,color:white
    classDef document fill:#34495e,stroke:#2c3e50,color:white

    %% User Interfaces
    subgraph UI
        WebUI["Web Interface"]:::ui
        CLI["CLI Interface"]:::ui
    end

    %% Core Layer
    subgraph CoreLayer["Core Services"]
        %% Task Management
        subgraph TaskManagement["Task Management"]
            TaskRegistry["Task Registry"]:::core
            TaskGraph["Task Graph"]:::core
            TaskRunner["Task Graph Runner"]:::core
        end

        %% Model Management
        subgraph ModelManagement["Model Management"]
            HFModel["HuggingFace Models"]:::model
            MPModel["MediaPipe Models"]:::model
            GGMLModel["GGML Local Models"]:::model
        end

        %% Provider System
        subgraph ProviderSystem["Provider System"]
            ProviderRegistry["Provider Registry"]:::provider
            HFProvider["HuggingFace Provider"]:::provider
            MPProvider["MediaPipe Provider"]:::provider
        end

        %% Storage System
        subgraph StorageSystem["Storage System"]
            KVStore["Key-Value Storage"]:::storage
            GraphStore["Task Graph Storage"]:::storage
            OutputStore["Task Output Storage"]:::storage
        end

        %% Job Queue System
        subgraph JobSystem["Job Queue System"]
            InMemQueue["In-Memory Queue"]:::queue
            PGQueue["PostgreSQL Queue"]:::queue
            SQLiteQueue["SQLite Queue"]:::queue
            RateLimiters["Rate Limiters"]:::queue
        end

        %% Document Processing
        subgraph DocSystem["Document Processing"]
            Document["Document System"]:::document
            DocConverters["Document Converters"]:::document
        end
    end

    %% Relationships
    WebUI --> TaskRegistry
    CLI --> TaskRegistry
    TaskRegistry --> TaskGraph
    TaskGraph --> TaskRunner
    TaskRunner --> JobSystem
    JobSystem --> ProviderSystem
    ProviderSystem --> ModelManagement
    TaskRunner --> StorageSystem
    Document --> DocConverters
    DocConverters --> TaskRegistry

    %% Click Events
    click WebUI "https://github.com/sroussey/ellmers/tree/main/packages/web"
    click CLI "https://github.com/sroussey/ellmers/tree/main/packages/cli"
    click TaskRegistry "https://github.com/sroussey/ellmers/blob/main/packages/core/src/task/base/TaskRegistry.ts"
    click TaskGraph "https://github.com/sroussey/ellmers/blob/main/packages/core/src/task/base/TaskGraph.ts"
    click TaskRunner "https://github.com/sroussey/ellmers/blob/main/packages/core/src/task/base/TaskGraphRunner.ts"
    click KVStore "https://github.com/sroussey/ellmers/tree/main/packages/core/src/storage/base"
    click GraphStore "https://github.com/sroussey/ellmers/tree/main/packages/core/src/storage/taskgraph"
    click OutputStore "https://github.com/sroussey/ellmers/tree/main/packages/core/src/storage/taskoutput"
    click HFModel "https://github.com/sroussey/ellmers/blob/main/packages/core/src/model/HuggingFaceModel.ts"
    click MPModel "https://github.com/sroussey/ellmers/blob/main/packages/core/src/model/MediaPipeModel.ts"
    click GGMLModel "https://github.com/sroussey/ellmers/blob/main/packages/core/src/model/GgmlLocalModel.ts"
    click ProviderRegistry "https://github.com/sroussey/ellmers/blob/main/packages/core/src/provider/ProviderRegistry.ts"
    click HFProvider "https://github.com/sroussey/ellmers/tree/main/packages/core/src/provider/local-hugging-face"
    click MPProvider "https://github.com/sroussey/ellmers/tree/main/packages/core/src/provider/local-media-pipe"
    click InMemQueue "https://github.com/sroussey/ellmers/blob/main/packages/core/src/job/InMemoryJobQueue.ts"
    click PGQueue "https://github.com/sroussey/ellmers/blob/main/packages/core/src/job/PostgreSqlJobQueue.ts"
    click SQLiteQueue "https://github.com/sroussey/ellmers/blob/main/packages/core/src/job/SqliteJobQueue.ts"
    click RateLimiters "https://github.com/sroussey/ellmers/blob/main/packages/core/src/job/InMemoryRateLimiter.ts"
    click Document "https://github.com/sroussey/ellmers/blob/main/packages/core/src/source/Document.ts"
    click DocConverters "https://github.com/sroussey/ellmers/blob/main/packages/core/src/source/DocumentConverterMarkdown.ts"
```
