# synw/locallm architecture

> Architecture diagram of the GitHub repository synw/locallm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/synw/locallm
- Repository: https://github.com/synw/locallm

## Overview

##### Main Components

a. Core Packages (represented as main modules):
- @locallm/types (shared data types)
- @locallm/api (main API interface)
- @locallm/browser (browser-specific implementation)
- @locallm/evaluate (evaluation tools)
- @locallm/evaluations (evaluation implementations)

b. External Providers (represented as external services):
- Llama.cpp Server
- Koboldcpp Server
- Ollama Server
- Wllama (Browser-based)

**┌─ Client Layer ─────────────────────────────────────┐**

- Applications / Examples                           │

- API Layer ──────────────────────────────────────┐ │

- @locallm/api                @locallm/browser    │ │

- Provider Layer ───────────────────────────────┐ │ │

- LlamacppProvider  KoboldcppProvider  etc.    │ │ │

- External Services Layer ────────────────────┐ │ │ │

- Llama.cpp  Koboldcpp  Ollama  Wllama       │ │ │ │

- ┘ │ │ │
- ┘ │ │
- ┘ │
- ┘

##### Key Relationships

a. Data Flow:
- Client → API Layer → Provider Layer → External Services
- Response flow in reverse direction

b. Dependencies:
- All packages depending on @locallm/types
- Provider-specific implementations connecting to their respective servers

##### Implementation Details

Use these symbols and conventions:
- Diamonds: Entry points
- Rectangles: Modules/packages

##### Additional Elements

Annotations for:
- API endpoints
- Configuration options
- Type definitions
- Evaluation paths

##### The evaluation system's integration

Keep the abstraction level focused on the architectural components while maintaining enough detail to show the distinct provider implementations and their relationships to external services.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Client["Client Layer"]
        App[Applications / Examples]
    end

    subgraph API["API Layer"]
        Types["@locallm/types"]:::core
        click Types "https://github.com/synw/locallm/blob/main/packages/types/src/interfaces.ts"
        
        APIMain["@locallm/api"]:::core
        click APIMain "https://github.com/synw/locallm/blob/main/packages/api/src/main.ts"
        
        Browser["@locallm/browser"]:::core
        click Browser "https://github.com/synw/locallm/blob/main/packages/browser/src/main.ts"
        
        Stats[Statistics Module]:::util
        click Stats "https://github.com/synw/locallm/blob/main/packages/api/src/stats.ts"
        
        BrowserInt[Browser Interfaces]:::util
        click BrowserInt "https://github.com/synw/locallm/blob/main/packages/browser/src/interfaces.ts"
    end

    subgraph Providers["Provider Layer"]
        ProviderUtils[Provider Utilities]:::util
        click ProviderUtils "https://github.com/synw/locallm/blob/main/packages/api/src/providers/utils.ts"
        
        LlamaCpp[LlamacppProvider]:::provider
        click LlamaCpp "https://github.com/synw/locallm/blob/main/packages/api/src/providers/llamacpp.ts"
        
        KoboldCpp[KoboldcppProvider]:::provider
        click KoboldCpp "https://github.com/synw/locallm/blob/main/packages/api/src/providers/koboldcpp.ts"
        
        Ollama[OllamaProvider]:::provider
        click Ollama "https://github.com/synw/locallm/blob/main/packages/api/src/providers/ollama.ts"
        
        Wllama[WllamaProvider]:::provider
        click Wllama "https://github.com/synw/locallm/blob/main/packages/browser/src/wllama.ts"
    end

    subgraph External["External Services Layer"]
        LlamaSrv[Llama.cpp Server]:::external
        KoboldSrv[Koboldcpp Server]:::external
        OllamaSrv[Ollama Server]:::external
        WllamaSrv[Wllama]:::external
    end

    subgraph Evaluation["Evaluation System"]
        Evaluate["@locallm/evaluate"]:::core
        click Evaluate "https://github.com/synw/locallm/blob/main/packages/evaluate/src/main.ts"
        
        Evaluations["@locallm/evaluations"]:::core
        click Evaluations "https://github.com/synw/locallm/tree/main/packages/evaluations/src"
        
        TestRunner[Test Runner]:::eval
        click TestRunner "https://github.com/synw/locallm/blob/main/packages/evaluate/src/testrunner.ts"
        
        TestCases[Test Cases]:::eval
        click TestCases "https://github.com/synw/locallm/blob/main/packages/evaluate/src/testcase.ts"
        
        CodeEval[Code Evaluators]:::eval
        click CodeEval "https://github.com/synw/locallm/tree/main/packages/evaluate/src/evaluators/code"
        
        TextEval[Text Evaluators]:::eval
        click TextEval "https://github.com/synw/locallm/tree/main/packages/evaluate/src/evaluators/text"
    end

    %% Relationships
    App --> APIMain
    App --> Browser
    
    APIMain --> Types
    Browser --> Types
    Evaluate --> Types
    
    APIMain --> ProviderUtils
    ProviderUtils --> LlamaCpp & KoboldCpp & Ollama
    Browser --> Wllama
    
    LlamaCpp --> LlamaSrv
    KoboldCpp --> KoboldSrv
    Ollama --> OllamaSrv
    Wllama --> WllamaSrv
    
    Evaluate --> TestRunner
    TestRunner --> TestCases
    TestCases --> CodeEval & TextEval
    Evaluations --> Evaluate

    %% Styles
    classDef core fill:#3498db,stroke:#2980b9,color:white
    classDef provider fill:#2ecc71,stroke:#27ae60,color:white
    classDef external fill:#e67e22,stroke:#d35400,color:white
    classDef util fill:#95a5a6,stroke:#7f8c8d,color:white
    classDef eval fill:#9b59b6,stroke:#8e44ad,color:white
```
