# dartmouth/openwebui-token-tracking architecture

> Architecture diagram of the GitHub repository dartmouth/openwebui-token-tracking, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dartmouth/openwebui-token-tracking
- Repository: https://github.com/dartmouth/openwebui-token-tracking

## Overview

##### Project Type and Purpose
- This is a Python library designed to plug into Open WebUI in order to track and limit token usage. It implements “token tracking” by wrapping model calls in tracked pipes (for providers like Anthropic, OpenAI, etc.) and managing user allowances (both basic credit groups and sponsored allowances).
- The README emphasizes token accounting using “credits” rather than raw token counts and integrates with Open WebUI’s pipe mechanism. This makes it a specialized middleware component for token management.

##### Analyzing the File Structure
- cli (command-line interface commands to initialize, manage credit groups, and find users),
- db (database-related modules including migrations, table definitions, and the actual database operations for tracking token usage),
- pipes (implementations of tracked pipes for various model providers such as Anthropic, OpenAI, Google GenAI, and Mistral).
- There is also a migrations directory using Alembic for database schema management, indicating the library’s reliance on database changes for token credit tables.
- Documentation is thorough, residing in the docs folder, with additional configuration (pyproject.toml, MANIFEST.in) and automated workflows (GitHub workflows under .github).

##### Key Components
a. Core Library Components:
- Token Tracker Core: Central logic that checks token quotas, applies credit conversions, and charges token credits after model responses.
- CLI Module: Processes commands for initializing the system, managing credit groups, and configuring sponsored allowances.
- Database Module: Contains the ORM (or raw SQL) interactions and migration scripts to set up tables (like token_tracking_base_settings, credit groups, user tables, etc.).
- Pipes Module: Implements individual tracked pipes. These are wrappers around API calls to various model providers (Anthropic, Google, etc.) that inject token tracking code.
- Resource Module: Contains JSON resources for models or pricing definitions.

b. External Systems and Interactions:
- Open WebUI: The web interface/system that calls these pipes. It is the integration point where the token tracker intercepts model calls.
- Database: A relational database for storing user allowances, token usage logs, and configuration (initialized via migration scripts).
- API Endpoints/Model Providers: External services (such as OpenAI and Anthropic) that are accessed through the tracked pipes.

##### Relationships and Interactions
- When a user sends a request via Open WebUI, the system first routes the request through the appropriate tracked pipe.
- The tracked pipe (an implementation from the pipes module) then consults the token tracker logic to verify if the user’s allowance is sufficient.
- If allowed, the pipe passes the request to the external language model API.
- On receiving the response, the token tracker updates the database (through the db module) by adjusting the user’s token/credit consumption.
- All configuration and initialization (like base allowances and credit groups) is managed via the CLI commands.

##### Architectural Patterns and Design Principles
- Layered Architecture: Clear separation between the user interface (Open WebUI integration), business logic (token tracking, credit conversion), and persistence (database modules with migrations).
- Plugin/Extension Pattern: Tracks different model providers using a common pipe interface but with provider-specific implementations.
- Command-Line Interface: A self-contained CLI tool ensures ease of setup and management of credit groups, sponsorship allowances, and database migrations.
- Integration with External Systems: The library acts as middleware between Open WebUI and external LLM APIs, enhancing them with token tracking capabilities.

##### Elements
- Open WebUI (external system that initiates requests)
- Token Tracker Core / Business Logic
- CLI Module (for configuration and management)
- Database Module (which includes migration setups and tables for token tracking, allowances, etc.)
- External LLM Providers (as external services)
- From Open WebUI to the respective pipe, then token check, then to the API, and back with a token usage update.
- Internal library modules (e.g., pipes, CLI, db) versus external systems (Open WebUI, LLM APIs, Database as an external persistent store).
- Annotations that provide configuration details (like the credit conversion rate, typical allowance values, etc.) as needed.

## Diagram (Mermaid)

```mermaid
graph TD
    %% External Systems
    subgraph "External Systems"
        opweb("Open WebUI"):::external
        extLLM("External LLM Providers"):::external
    end

    %% Token Tracking System
    subgraph "Token Tracking System"
        %% Pipes Module
        subgraph "Pipes Module"
            direction TB
            p1("Anthropic Pipe"):::internal
            p2("OpenAI Pipe"):::internal
            p3("Google GenAI Pipe"):::internal
            p4("Mistral Pipe"):::internal
            p5("Base Tracked Pipe"):::internal
        end

        core("Token Tracker Core"):::internal

        %% Database Module
        subgraph "Database Module"
            direction TB
            db1("DB Base"):::internal
            db2("DB Credit Group"):::internal
            db3("DB Operations"):::internal
            db4("Database Migrations"):::internal
        end

        %% CLI Module
        subgraph "CLI Module"
            direction TB
            cli_main("CLI Main"):::internal
            subgraph "CLI Commands"
                direction TB
                cmd1("credit_group.py"):::internal
                cmd2("database.py"):::internal
                cmd3("pricing.py"):::internal
                cmd4("settings.py"):::internal
                cmd5("sponsored.py"):::internal
                cmd6("user.py"):::internal
            end
        end

        %% Resource Module
        subgraph "Resource Module"
            direction TB
            res("models.json"):::internal
        end
    end

    %% Data Flow Connections
    opweb -->|"request"| p1
    opweb -->|"request"| p2
    opweb -->|"request"| p3
    opweb -->|"request"| p4
    opweb -->|"request"| p5

    p1 -->|"token-validation"| core
    p2 -->|"token-validation"| core
    p3 -->|"token-validation"| core
    p4 -->|"token-validation"| core
    p5 -->|"token-validation"| core

    core -->|"API_call"| extLLM
    extLLM -->|"response"| core
    core -->|"update_usage"| db3

    cli_main -->|"configure/init"| db4
    cli_main -->|"configure"| core

    res -->|"load_resources"| core

    %% Styles
    classDef external fill:#ffe6e6,stroke:#ff0000,stroke-width:2px;
    classDef internal fill:#e6f7ff,stroke:#007acc,stroke-width:2px;

    %% Click Events
    click core "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/tracking.py"
    click cli_main "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/cli/main.py"
    click cmd1 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/cli/commands/credit_group.py"
    click cmd2 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/cli/commands/database.py"
    click cmd3 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/cli/commands/pricing.py"
    click cmd4 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/cli/commands/settings.py"
    click cmd5 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/cli/commands/sponsored.py"
    click cmd6 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/cli/commands/user.py"
    click db1 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/db/base.py"
    click db2 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/db/credit_group.py"
    click db3 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/db/db.py"
    click db4 "https://github.com/dartmouth/openwebui-token-tracking/tree/main/src/openwebui_token_tracking/migrations"
    click p1 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/pipes/anthropic.py"
    click p2 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/pipes/openai.py"
    click p3 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/pipes/google_genai.py"
    click p4 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/pipes/mistral.py"
    click p5 "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/pipes/base_tracked_pipe.py"
    click res "https://github.com/dartmouth/openwebui-token-tracking/blob/main/src/openwebui_token_tracking/resources/models.json"
```
