# mensterra/karo architecture

> Architecture diagram of the GitHub repository mensterra/karo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mensterra/karo
- Repository: https://github.com/mensterra/karo

## Overview

##### Project type and purpose
- Karo is a Python framework/library for building AI agents, not a monolithic application.
- It provides reusable building blocks—agent core, memory, provider abstraction, tools, session management, CLI and serving API.
- The README emphasizes modularity, schema-driven interactions, persistent memory, pluggable LLM providers, and extensibility.

##### File structure
- Top-level: pyproject.toml, Dockerfile, README.md, docs/, examples/, tests/
- karo/ package subdivided into:
– cli/ (CLI entry points)
– core/ (BaseAgent & config)
– memory/ (conversation history, MemoryManager, vector store service)
– prompts/ (system prompt builder)
– providers/ (abstract BaseProvider + OpenAI, Anthropic, Gemini, Ollama)
– sessions/ (session lifecycle and persistence)
– serving/ (REST server, auth, config, models)
– tools/ (base tool, built-in tools like calculator, CSV/excel/document readers)
– utils/ (logging, ingestion scripts)
- Supporting docs in docs/ and examples/ illustrate common usage patterns.
- Tests mirror package structure.

##### Key architectural insights from README
- Agent orchestration is pure Python, driven by Pydantic schemas.
- Memory uses a vector store (ChromaDBService).
- CLI (“karo serve”, “karo token”) and an HTTP serving API export agent functionality.
- Providers are pluggable implementations behind a common interface.
- Tools are discoverable via BaseTool interface and injected into agents.

##### Main components
A. Client Interfaces
– CLI (karo command)
– HTTP REST Server (karo.serving.server)
B. Agent Core
– BaseAgent (or custom subclass)
– BaseAgentConfig
C. Provider Layer
– Abstract BaseProvider
– Concrete providers (OpenAIProvider, AnthropicProvider, etc.)
D. Memory Subsystem
– MemoryManager
– ConversationHistory
– Vector Store Service (ChromaDBService)
E. Tooling Layer
– BaseTool
– Built-in tools (calculator, document reader, etc.)
– Custom tools (in examples/)
F. Session Management
– SessionService, Session, Event, DatabaseService
– Persistent session store (e.g., SQLite or other via SQLAlchemy)
G. External Dependencies
– LLM APIs (OpenAI, Anthropic, Gemini…)
– ChromaDB vector store
– Database backend for sessions

##### Relationships and data flows
1. User invokes CLI or sends HTTP request → CLI/Server parses input and invokes Agent Core.
2. Agent Core receives a typed Pydantic InputSchema.
3. Agent Core consults MemoryManager → interacts with ChromaDBService to fetch/store vectors.
4. Agent Core may call one or more Tools → each tool implements BaseTool interface and returns results.
5. Agent Core issues prompts to LLM via the configured Provider.
6. Provider calls external LLM API → returns model response.
7. Agent Core wraps the response in Pydantic OutputSchema.
8. SessionService logs events and persists session state.
9. CLI/Server returns the OutputSchema response to the user.

##### Architectural patterns & principles
- Modular layering: UI (CLI/HTTP) → Core → Memory/Tools/Providers → External Services
- Dependency inversion: Agents depend on abstract providers/tools, not concrete implementations.
- Schema-driven I/O: Pydantic classes enforce data contracts.
- Separation of concerns: Memory, session persistence, serving, and core logic live in separate packages.
- Extensible plugin points: providers/, tools/, memory/services.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Interfaces
    subgraph "Client Interfaces"
        CLI_Main["main.py"]:::client
        CLI_Serve["serve_command.py"]:::client
        CLI_Token["token_command.py"]:::client
        HTTP_Server["server.py"]:::client
        HTTP_Auth["auth.py"]:::client
        HTTP_Config["config.py"]:::client
        HTTP_Models["models.py"]:::client
    end

    %% Core Framework
    subgraph "Agent Core"
        BaseAgent["base_agent.py"]:::core
    end

    %% Memory Subsystem
    subgraph "Memory Subsystem"
        MemoryManager["memory_manager.py"]:::plugin
        ConversationHistory["conversation_history.py"]:::plugin
        MemoryModels["memory_models.py"]:::plugin
        ChromaDBService["chromadb_service.py"]:::plugin
    end

    %% Provider Layer
    subgraph "Providers (Extension Points)"
        BaseProvider["base_provider.py"]:::plugin
        OpenAIProvider["openai_provider.py"]:::plugin
        AnthropicProvider["anthropic_provider.py"]:::plugin
        GeminiProvider["gemini_provider.py"]:::plugin
        OllamaProvider["ollama_provider.py"]:::plugin
    end

    %% Tooling Layer
    subgraph "Tools (Extension Points)"
        BaseTool["base_tool.py"]:::plugin
        CalculatorTool["calculator_tool.py"]:::plugin
        CSVReaderTool["csv_reader_tool.py"]:::plugin
        DocumentReaderTool["document_reader_tool.py"]:::plugin
        ExcelReaderTool["excel_reader_tool.py"]:::plugin
    end

    %% Session Management
    subgraph "Session Management"
        SessionService["service.py"]:::core
        SessionEntity["session.py"]:::core
        EventModel["event.py"]:::core
        SessionDBService["database_service.py"]:::core
    end

    %% External Dependencies
    subgraph "External Services"
        LLM_APIs["LLM APIs"]:::external
        ChromaDB_External["ChromaDB Store"]:::external
        SQL_DB["Session DB"]:::external
    end

    %% Data and Control Flows
    User["User"]:::externalUser
    User -->|CLI invoke| CLI_Main
    CLI_Main --> CLI_Serve
    CLI_Main --> CLI_Token
    User -->|HTTP request| HTTP_Server
    HTTP_Server --> HTTP_Auth
    HTTP_Server --> HTTP_Config
    HTTP_Server --> HTTP_Models
    CLI_Serve --> BaseAgent
    CLI_Token --> BaseAgent
    HTTP_Server --> BaseAgent
    BaseAgent -->|fetch/store| MemoryManager
    MemoryManager -->|uses| ChromaDBService
    ChromaDBService -->|persist/retrieve| ChromaDB_External
    BaseAgent -->|access| ConversationHistory
    BaseAgent -->|call| BaseTool
    BaseTool -->|implements| CalculatorTool
    BaseTool -->|implements| CSVReaderTool
    BaseTool -->|implements| DocumentReaderTool
    BaseTool -->|implements| ExcelReaderTool
    BaseAgent -->|invoke| BaseProvider
    BaseProvider -->|implements| OpenAIProvider
    BaseProvider -->|implements| AnthropicProvider
    BaseProvider -->|implements| GeminiProvider
    BaseProvider -->|implements| OllamaProvider
    OpenAIProvider -->|API call| LLM_APIs
    AnthropicProvider -->|API call| LLM_APIs
    GeminiProvider -->|API call| LLM_APIs
    OllamaProvider -->|API call| LLM_APIs
    BaseAgent -->|wrap response| SessionService
    SessionService --> SessionEntity
    SessionService --> EventModel
    SessionService --> SessionDBService
    SessionDBService -->|persist| SQL_DB
    SessionService -->|log event| EventModel
    SessionService -->|store session| SessionEntity
    SessionService -->|interact| SQL_DB

    %% Click Events
    click CLI_Main "https://github.com/mensterra/karo/blob/main/karo/cli/main.py"
    click CLI_Serve "https://github.com/mensterra/karo/blob/main/karo/cli/serve_command.py"
    click CLI_Token "https://github.com/mensterra/karo/blob/main/karo/cli/token_command.py"
    click HTTP_Server "https://github.com/mensterra/karo/blob/main/karo/serving/server.py"
    click HTTP_Auth "https://github.com/mensterra/karo/blob/main/karo/serving/auth.py"
    click HTTP_Config "https://github.com/mensterra/karo/blob/main/karo/serving/config.py"
    click HTTP_Models "https://github.com/mensterra/karo/blob/main/karo/serving/models.py"
    click BaseAgent "https://github.com/mensterra/karo/blob/main/karo/core/base_agent.py"
    click BaseProvider "https://github.com/mensterra/karo/blob/main/karo/providers/base_provider.py"
    click OpenAIProvider "https://github.com/mensterra/karo/blob/main/karo/providers/openai_provider.py"
    click AnthropicProvider "https://github.com/mensterra/karo/blob/main/karo/providers/anthropic_provider.py"
    click GeminiProvider "https://github.com/mensterra/karo/blob/main/karo/providers/gemini_provider.py"
    click OllamaProvider "https://github.com/mensterra/karo/blob/main/karo/providers/ollama_provider.py"
    click MemoryManager "https://github.com/mensterra/karo/blob/main/karo/memory/memory_manager.py"
    click ConversationHistory "https://github.com/mensterra/karo/blob/main/karo/memory/conversation_history.py"
    click MemoryModels "https://github.com/mensterra/karo/blob/main/karo/memory/memory_models.py"
    click ChromaDBService "https://github.com/mensterra/karo/blob/main/karo/memory/services/chromadb_service.py"
    click BaseTool "https://github.com/mensterra/karo/blob/main/karo/tools/base_tool.py"
    click CalculatorTool "https://github.com/mensterra/karo/blob/main/karo/tools/calculator_tool.py"
    click CSVReaderTool "https://github.com/mensterra/karo/blob/main/karo/tools/csv_reader_tool.py"
    click DocumentReaderTool "https://github.com/mensterra/karo/blob/main/karo/tools/document_reader_tool.py"
    click ExcelReaderTool "https://github.com/mensterra/karo/blob/main/karo/tools/excel_reader_tool.py"
    click SessionService "https://github.com/mensterra/karo/blob/main/karo/sessions/service.py"
    click SessionEntity "https://github.com/mensterra/karo/blob/main/karo/sessions/session.py"
    click EventModel "https://github.com/mensterra/karo/blob/main/karo/sessions/event.py"
    click SessionDBService "https://github.com/mensterra/karo/blob/main/karo/sessions/database_service.py"

    %% Styles
    classDef client fill:#cccccc,stroke:#333,stroke-width:1px
    classDef core fill:#cce5ff,stroke:#339,stroke-width:1px
    classDef plugin fill:#d4edda,stroke:#3a3,stroke-width:1px
    classDef external fill:#ffe5b4,stroke:#e68a00,stroke-width:1px
    classDef externalUser fill:#f0f0f0,stroke:#666,stroke-width:1px,stroke-dasharray: 5 5
```
