# yym68686/uni-api architecture

> Architecture diagram of the GitHub repository yym68686/uni-api, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yym68686/uni-api
- Repository: https://github.com/yym68686/uni-api

## Overview

##### Main Components
- API Gateway: The central module (main.py) receives HTTP requests (e.g., /v1/chat/completions, /v1/images/generations, etc.) and serves as the entry point.
- Request & Response Handlers: Modules like request.py, response.py, and models.py encapsulate and normalize incoming requests and outgoing responses.
- Provider Management: Within the components directory (e.g., provider_table.py) and through the configuration file (api.yaml), the system manages a list of backend service providers (OpenAI, Anthropic, Gemini, Vertex, Azure, Cloudflare, etc.) along with model mappings, API keys, and scheduling strategies.
- Load Balancing & Routing Engine: Implementing various scheduling algorithms (fixed_priority, round_robin, weighted_round_robin, lottery) that handle selecting the proper backend channel for each request. This module is tightly coupled with configuration processing (see the api_keys and preferences in the configuration file).
- Error Handling & Retry Logic: Auto-retry logic, channel cooling, and fine-grained model timeout settings ensure robust handling when a provider channel fails or times out.
- Configuration and Environment Management: The use of either a locally mounted api.yaml or a configuration URL (via the CONFIG_URL environment variable), along with environment variables (TIMEOUT, DISABLE_DATABASE, etc.), drives the runtime behavior.
- Logging and Monitoring: The file log_config.py (and related Docker logs) indicate that logging is a significant concern for debugging and monitoring the API’s operations.
- Tests: A comprehensive test suite (located under the test/ directory) ensures functionality and facilitates continuous integration.

##### Relationships and Data Flow
- Inbound API Request: An external client sends a request (e.g., a chat completion request) to the API Gateway.
- Request Parsing & Normalization: The request handler module validates and normalizes the incoming request.
- Configuration Lookup & Routing: The system reads the configuration file to identify which provider channels support the requested model. The routing engine then selects one (or more) providers based on scheduling and load-balancing policies.
- Outbound Calls: The API Gateway forwards the normalized request to the chosen backend service (each backend is an external provider with its specified base_url and API key).
- Response Aggregation & Transformation: Once a cloud provider responds, the system transforms (if needed) the response into the unified OpenAI format before returning it to the client.
- Error Handling: In case of errors (e.g., timeouts or rate limits), the retry mechanism automatically resends the request following the predefined policies and may trigger channel cooling.
- Monitoring and Logging: All requests and error scenarios are logged and potentially stored (if a database is enabled).

##### Architectural Patterns and Technologies
- Configuration-driven design: The entire behavior (provider selection, load balancing, rate limits, timeouts, scheduling algorithms) is controlled via external YAML configuration.
- Modular design: Separation of concerns across modules (provider management, request handling, logging, and error recovery).
- Resilience patterns: Automatic retry, load balancing (across multiple channels and keys), and error-triggered cooldown ensure robustness.
- Use of containerization: Deployment scripts (Dockerfile, docker-compose.yml, vercel.json, wrangler.toml) indicate the design is cloud-ready and supports multiple deployment environments.
- Testing emphasis: Comprehensive test coverage in the test/ directory ensures that changes to one component won’t break others.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Client
    Client["External Client"]:::external

    %% Core Services Subgraph
    subgraph "Core Services"
        A["API Gateway (/v1 endpoints)"]:::core
        B["Configuration Manager (api.yaml)"]:::core
        C["Provider Manager & Model Router"]:::core
        D["Load Balancer / Scheduler"]:::core
        E["Logging & Error Handling"]:::core
        F["Utilities"]:::core
    end

    %% LLM Providers Subgraph
    subgraph "LLM Providers"
        OA["OpenAI"]:::external
        AT["Anthropic"]:::external
        GE["Gemini"]:::external
        VE["Vertex"]:::external
        AZ["Azure"]:::external
    end

    %% Deployment & Packaging Subgraph
    subgraph "Deployment Artifacts"
        H["Deployment & Packaging"]:::deployment
    end

    %% Testing & CI/CD Subgraph
    subgraph "Testing & CI/CD"
        G["Testing & CI/CD"]:::testing
    end

    %% Relationships
    Client -->|"sends request"| A
    A -->|"auth/validate"| B
    B -->|"load config"| C
    C -->|"map request"| D
    D -->|"dispatch"| OA
    D -->|"dispatch"| AT
    D -->|"dispatch"| GE
    D -->|"dispatch"| VE
    D -->|"dispatch"| AZ
    OA -->|"error/retry"| D
    AT -->|"error/retry"| D
    GE -->|"error/retry"| D
    VE -->|"error/retry"| D
    AZ -->|"error/retry"| D
    D -->|"log event"| E
    A -->|"log event"| E

    %% Optional utility relationship (support functions)
    A ---|"uses"| F

    %% Styles
    classDef core fill:#BBDEFB,stroke:#0D47A1,stroke-width:2px;
    classDef external fill:#C8E6C9,stroke:#1B5E20,stroke-width:2px;
    classDef deployment fill:#FFCCBC,stroke:#BF360C,stroke-width:2px;
    classDef testing fill:#D1C4E9,stroke:#4527A0,stroke-width:2px;

    %% Click Events for Component Mapping
    click A "https://github.com/yym68686/uni-api/blob/main/main.py"
    click C "https://github.com/yym68686/uni-api/blob/main/components/provider_table.py"
    click C "https://github.com/yym68686/uni-api/blob/main/models.py"
    click C "https://github.com/yym68686/uni-api/blob/main/request.py"
    click C "https://github.com/yym68686/uni-api/blob/main/response.py"
    click E "https://github.com/yym68686/uni-api/blob/main/log_config.py"
    click F "https://github.com/yym68686/uni-api/blob/main/utils.py"
    click G "https://github.com/yym68686/uni-api/tree/main/test/"
    click G "https://github.com/yym68686/uni-api/tree/main/.github/workflows/"
    click H "https://github.com/yym68686/uni-api/tree/main/Dockerfile"
    click H "https://github.com/yym68686/uni-api/blob/main/docker-compose.yml"
    click H "https://github.com/yym68686/uni-api/blob/main/vercel.json"
    click H "https://github.com/yym68686/uni-api/blob/main/wrangler.toml"
```
