# kagisearch/pyllms architecture

> Architecture diagram of the GitHub repository kagisearch/pyllms, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kagisearch/pyllms
- Repository: https://github.com/kagisearch/pyllms

## Overview

##### Project type and purpose
- PyLLMs is an open-source Python library (not a full-stack app) whose goal is to provide a unified, minimal interface for interacting with many LLM providers and running benchmarks.
- It exposes a simple API (llms.init, .complete, .benchmark) that hides the details of each provider’s HTTP/API calls.

##### File structure
- Top-level: setup.py, requirements.txt, README.md – standard Python package.
- llms/llms.py: the core factory and façade exposing init(), complete(), acomplete(), benchmark().
- llms/providers/: a plugin directory of provider modules.
- base_provider.py defines the Provider interface/abstract base.
- concrete modules (openai.py, anthropic.py, cohere.py, etc.) implement that interface.
- llms/results/result.py: standardizes response metadata (tokens, cost, latency).

##### README for additional insights
- Documentation describes sync/async, streaming, multi-model, chat-history, token counting, benchmarks.

##### Elements
- Clear labels: “Client Code,” “PyLLMs Core,” “Provider Factory,” “Provider (e.g., OpenAIProvider),” “Result Normalizer,” “External LLM Service.”
- From Client → Core → Factory → Provider → External Service → Provider → Result Normalizer → Core → Client

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    Client["Client Code"]:::external

    %% Configuration
    Config["Configuration\n(env vars / init args)"]:::config
    Setup["setup.py"]:::config
    Req["requirements.txt"]:::config
    Doc["README.md"]:::doc

    %% Core Facade & Factory
    subgraph "PyLLMs Core"
        direction TB
        Core["Facade & Factory\nllms/llms.py"]:::internal
        Factory["Provider Factory & Registry"]:::internal
        ResultNorm["Result Normalizer\nllms/results/result.py"]:::result
    end

    %% Providers
    subgraph "Provider Implementations\n(llms/providers)" 
        direction TB
        BaseProv["BaseProvider Interface\nllms/providers/base_provider.py"]:::provider
        OpAI["OpenAIProvider"]:::provider
        Anthro["AnthropicProvider"]:::provider
        Cohere["CohereProvider"]:::provider
        AI21["AI21Provider"]:::provider
        Aleph["AlephProvider"]:::provider
        BedAnthro["BedrockAnthropicProvider"]:::provider
        Deepseek["DeepseekProvider"]:::provider
        Google["GoogleProvider"]:::provider
        GoogleGen["GoogleGenAIProvider"]:::provider
        Groq["GroqProvider"]:::provider
        HF["HuggingFaceProvider"]:::provider
        Mistral["MistralProvider"]:::provider
        Ollama["OllamaProvider"]:::provider
        OpenRouter["OpenRouterProvider"]:::provider
        Reka["RekaProvider"]:::provider
        Together["TogetherProvider"]:::provider
    end

    %% External Services
    subgraph "External LLM Services"
        direction TB
        ExtOpenAI["OpenAI API"]:::external
        ExtAnthro["Anthropic API"]:::external
        ExtCohere["Cohere API"]:::external
        ExtAI21["AI21 API"]:::external
        ExtAleph["Aleph API"]:::external
        ExtBedAnthro["Bedrock Anthropic API"]:::external
        ExtDeepseek["Deepseek API"]:::external
        ExtGoogle["Google API"]:::external
        ExtGoogleGen["Google GenAI API"]:::external
        ExtGroq["Groq API"]:::external
        ExtHF["HuggingFace API"]:::external
        ExtMistral["Mistral API"]:::external
        ExtOllama["Ollama API"]:::external
        ExtOpenRouter["OpenRouter API"]:::external
        ExtReka["Reka API"]:::external
        ExtTogether["Together API"]:::external
    end

    %% Flows
    Client -->|"init()"| Config
    Config -->|"loads into"| Core
    Core --> Factory
    Factory --> BaseProv
    BaseProv --> OpAI
    BaseProv --> Anthro
    BaseProv --> Cohere
    BaseProv --> AI21
    BaseProv --> Aleph
    BaseProv --> BedAnthro
    BaseProv --> Deepseek
    BaseProv --> Google
    BaseProv --> GoogleGen
    BaseProv --> Groq
    BaseProv --> HF
    BaseProv --> Mistral
    BaseProv --> Ollama
    BaseProv --> OpenRouter
    BaseProv --> Reka
    BaseProv --> Together

    Client -->|"complete()"| Core
    Core -->|"calls complete()"| BaseProv
    BaseProv -->|"HTTP call"| ExtOpenAI
    ExtOpenAI -->|"response"| BaseProv
    BaseProv --> ResultNorm
    ResultNorm --> Core
    Core --> Client

    %% Benchmark Parallel Flow
    Client -->|"benchmark()"| Core
    Core -->|"fan-out"| OpAI
    Core --> Cohere
    Core --> Anthro
    Core --> Google
    Cohere --> ResultNorm
    Anthro --> ResultNorm
    OpAI --> ResultNorm
    Google --> ResultNorm
    ResultNorm --> Core
    Core --> Client

    %% Click Events
    click Core "https://github.com/kagisearch/pyllms/blob/main/llms/llms.py"
    click BaseProv "https://github.com/kagisearch/pyllms/blob/main/llms/providers/base_provider.py"
    click OpAI "https://github.com/kagisearch/pyllms/blob/main/llms/providers/openai.py"
    click Anthro "https://github.com/kagisearch/pyllms/blob/main/llms/providers/anthropic.py"
    click Cohere "https://github.com/kagisearch/pyllms/blob/main/llms/providers/cohere.py"
    click AI21 "https://github.com/kagisearch/pyllms/blob/main/llms/providers/ai21.py"
    click Aleph "https://github.com/kagisearch/pyllms/blob/main/llms/providers/aleph.py"
    click BedAnthro "https://github.com/kagisearch/pyllms/blob/main/llms/providers/bedrock_anthropic.py"
    click Deepseek "https://github.com/kagisearch/pyllms/blob/main/llms/providers/deepseek.py"
    click Google "https://github.com/kagisearch/pyllms/blob/main/llms/providers/google.py"
    click GoogleGen "https://github.com/kagisearch/pyllms/blob/main/llms/providers/google_genai.py"
    click Groq "https://github.com/kagisearch/pyllms/blob/main/llms/providers/groq.py"
    click HF "https://github.com/kagisearch/pyllms/blob/main/llms/providers/huggingface.py"
    click Mistral "https://github.com/kagisearch/pyllms/blob/main/llms/providers/mistral.py"
    click Ollama "https://github.com/kagisearch/pyllms/blob/main/llms/providers/ollama.py"
    click OpenRouter "https://github.com/kagisearch/pyllms/blob/main/llms/providers/openrouter.py"
    click Reka "https://github.com/kagisearch/pyllms/blob/main/llms/providers/reka.py"
    click Together "https://github.com/kagisearch/pyllms/blob/main/llms/providers/together.py"
    click ResultNorm "https://github.com/kagisearch/pyllms/blob/main/llms/results/result.py"
    click Setup "https://github.com/kagisearch/pyllms/blob/main/setup.py"
    click Req "https://github.com/kagisearch/pyllms/blob/main/requirements.txt"
    click Doc "https://github.com/kagisearch/pyllms/blob/main/README.md"

    %% Styles
    classDef internal fill:#D6EAF8,stroke:#2980B9,stroke-width:1px
    classDef provider fill:#EBF5FB,stroke:#5DADE2,stroke-width:1px
    classDef external fill:#FADBD8,stroke:#C0392B,stroke-width:1px
    classDef result fill:#EBF5FB,stroke:#28B463,stroke-width:1px
    classDef config fill:#FCF3CF,stroke:#B7950B,stroke-width:1px
    classDef doc fill:#E8DAEF,stroke:#8E44AD,stroke-width:1px
```
