# bricks-cloud/bricksllm architecture

> Architecture diagram of the GitHub repository bricks-cloud/bricksllm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bricks-cloud/bricksllm
- Repository: https://github.com/bricks-cloud/bricksllm

## Overview

##### Project Type and Purpose
- BricksLLM is a backend API gateway built in Go that integrates with several large language model (LLM) providers (e.g., OpenAI, Anthropic, Azure OpenAI, vLLM).
- Its purpose is to provide enterprise features around rate limiting, cost management, PII redaction, caching, and failover for LLM usage.
- Use cases include controlled API key distribution, failovers, analytics, logging with privacy control, and integration with external model providers.

##### File Structure to Identify Architectural Components
- The top-level structure shows a clear Go project layout with “cmd”, “internal”, and configuration files. The main application entry point is in cmd/bricksllm.
- The “internal” folder is broken down into submodules such as authentication, cache, config, event, logger, manager, provider, server, storage, telemetry, and more. This indicates a layered, modular design.
- Notice the “internal/server/web” directory, which is split into “admin” and “proxy” subdirectories:
- The Admin component handles management, reporting, and policy configuration.
- The Proxy component is the entry point for handling client API requests (that are eventually forwarded to various LLM providers).
- There are also directories for providers (OpenAI, Anthropic, Azure, vLLM, etc.) that encapsulate external API interactions and cost/token handling.
- The existence of Dockerfiles, docker-compose.yml, and Kubernetes Helm Charts reveals that containerization and orchestration are supported.

##### Key Components and Their Relationships
a. Main Components to Include:
- Clients/Users: External clients, SDKs, or applications consuming the API.
- API Gateway (Proxy Server): The entry point for client requests, handling routing, rate limiting, caching, and failover.
- Admin Server: Provides Swagger documentation endpoints, management, and reporting functionalities.
- Internal Business Logic: Modules/services that implement authentication, access control, cost/rate management, key management, encryption, event handling, and PII detection.
- Providers: Components interfacing with external LLM providers (OpenAI, Anthropic, Azure, vLLM, Deepinfra, and custom integrations).
- Storage Layer:
- Postgresql for persistent storage (configuration, rate limits, user key data, etc.)
- Redis for caching and quick-access storage needs.
- Auxiliary Services: Logging (using Zap), telemetry (Prometheus, StatsD/Datadog), and messaging (internal event bus).

b. Relationships and Interactions:
- Client requests enter through the Proxy API server, with the admin server separately available for management.
- The API gateway will validate requests using authenticator modules and then route them to appropriate provider modules based on the target (e.g., OpenAI or Anthropic).
- Before forwarding, modules for rate limiting, cost check, and PII detection intervene.
- All key configurations and events are stored in Postgresql, while caching (such as API key data and temporary responses) utilizes Redis.
- Internal logging and telemetry modules collect metrics and send events to external monitoring systems (like Datadog or Prometheus).
- The project uses Docker, Kubernetes (via Helm charts), and standard CI/CD workflows (as seen in .github/workflows) to manage deployments.

##### Important Architectural Patterns and Design Principles
- Modular design: Separation between internal business logic (managers, providers, and validators) and API server layers.
- Separation of Concerns: Distinct roles managed by the Admin Server versus the Proxy Server.
- Extensibility: Provider modules are isolated, enabling ease of adding or modifying support for new LLM providers.
- Resiliency: Built-in caching, retry, and failover mechanisms to enhance stability and performance.
- Infrastructure as Code: Docker-compose and Kubernetes Helm charts for deployment consistency.

##### Elements
- Components should have clear labels (e.g., “Proxy API Server”, “Admin Server”, “Rate Limiter”, “PII Detection Module”, “OpenAI Provider”, etc.).
- From Clients -> Proxy API Layer.
- From Proxy -> Pipeline modules (Authentication, Rate Limit, Failover/Caching, etc.).
- From internal components -> External LLM providers.
- Feedback loops from internal logging/telemetry back to monitoring systems.
- Rectangular blocks for servers and services.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Global Entities
    Clients["Clients / Users"]:::client

    %% API Gateways
    Proxy["Proxy API Server"]:::server
    Admin["Admin Server"]:::server

    %% API Processing Modules (inside Proxy)
    subgraph "API Processing Modules"
        Auth["Authentication Module"]:::module
        Cache["Caching Module"]:::module
        Rate["Rate Limiting & Cost Management"]:::module
        PII["PII Detection Module"]:::module
        Route["API Routing"]:::module
        Encrypt["Encryption Services"]:::module
        Key["Key Management"]:::module
    end

    %% LLM Providers (External)
    subgraph "LLM Providers"
        OpenAI["OpenAI Provider"]:::provider
        Anthropic["Anthropic Provider"]:::provider
        Azure["Azure OpenAI Provider"]:::provider
        vLLM["vLLM Provider"]:::provider
        Deepinfra["Deepinfra Provider"]:::provider
        Custom["Custom Provider"]:::provider
    end

    %% Auxiliary Services and Storage
    subgraph "Auxiliary Services & Storage"
        Log["Logging Service"]:::module
        Telemetry["Telemetry & Monitoring"]:::module
        EventBus["Event Messaging"]:::module
        PG["Postgresql"]:::database
        Redis["Redis Storage"]:::database
        Config["Administration & Configuration"]:::module
    end

    %% Containerization & Orchestration
    Infra["Containerization & Orchestration"]:::infra

    %% Connections between Components
    Clients -->|"sends_requests"| Proxy
    Clients -->|"admin_access"| Admin

    %% Proxy Internal Pipeline
    Proxy --> Auth
    Auth --> Cache
    Cache --> Rate
    Rate --> PII
    PII --> Route
    Route --> Encrypt
    Encrypt --> Key

    %% Routing to External Providers
    Key -->|"routes_to"| OpenAI
    Key -->|"routes_to"| Anthropic
    Key -->|"routes_to"| Azure
    Key -->|"routes_to"| vLLM
    Key -->|"routes_to"| Deepinfra
    Key -->|"routes_to"| Custom

    %% Internal Service Interactions
    Proxy --- Log
    Proxy --- Telemetry
    Proxy --- EventBus

    Admin --> Config
    Config --- PG
    Config --- Redis
    Admin --- Log
    Admin --- Telemetry

    %% Storage involvement for Internal Modules
    Auth --- PG
    Cache --- Redis
    Rate --- PG
    PII --- PG

    %% Logging and Telemetry external feedback
    Telemetry -->|"metrics"| Infra
    Log -->|"logs"| Infra

    %% Infrastructure supporting API Gateways
    Infra --- Proxy
    Infra --- Admin

    %% Click Events for Component Mappings
    click Proxy "https://github.com/bricks-cloud/bricksllm/tree/main/internal/server/web/proxy"
    click Admin "https://github.com/bricks-cloud/bricksllm/tree/main/internal/server/web/admin"
    click Auth "https://github.com/bricks-cloud/bricksllm/tree/main/internal/authenticator"
    click Cache "https://github.com/bricks-cloud/bricksllm/tree/main/internal/cache"
    click Rate "https://github.com/bricks-cloud/bricksllm/blob/main/internal/manager/rate_limit.go"
    click Rate "https://github.com/bricks-cloud/bricksllm/blob/main/internal/manager/provider_setting.go"
    click PII "https://github.com/bricks-cloud/bricksllm/tree/main/internal/pii"
    click Log "https://github.com/bricks-cloud/bricksllm/tree/main/internal/logger"
    click Log "https://github.com/bricks-cloud/bricksllm/tree/main/internal/logger/zap"
    click Telemetry "https://github.com/bricks-cloud/bricksllm/tree/main/internal/telemetry"
    click Telemetry "https://github.com/bricks-cloud/bricksllm/tree/main/internal/telemetry/prometheus"
    click Telemetry "https://github.com/bricks-cloud/bricksllm/tree/main/internal/telemetry/stats"
    click EventBus "https://github.com/bricks-cloud/bricksllm/tree/main/internal/message"
    click OpenAI "https://github.com/bricks-cloud/bricksllm/tree/main/internal/provider/openai"
    click Anthropic "https://github.com/bricks-cloud/bricksllm/tree/main/internal/provider/anthropic"
    click Azure "https://github.com/bricks-cloud/bricksllm/tree/main/internal/provider/azure"
    click vLLM "https://github.com/bricks-cloud/bricksllm/tree/main/internal/provider/vllm"
    click Deepinfra "https://github.com/bricks-cloud/bricksllm/tree/main/internal/provider/deepinfra"
    click Custom "https://github.com/bricks-cloud/bricksllm/tree/main/internal/provider/custom"
    click PG "https://github.com/bricks-cloud/bricksllm/tree/main/internal/storage/postgresql"
    click Redis "https://github.com/bricks-cloud/bricksllm/tree/main/internal/storage/redis"
    click Config "https://github.com/bricks-cloud/bricksllm/tree/main/internal/config"
    click Config "https://github.com/bricks-cloud/bricksllm/tree/main/internal/policy"
    click Config "https://github.com/bricks-cloud/bricksllm/tree/main/internal/policy/custom"
    click Route "https://github.com/bricks-cloud/bricksllm/tree/main/internal/route"
    click Route "https://github.com/bricks-cloud/bricksllm/blob/main/internal/manager/route.go"
    click Encrypt "https://github.com/bricks-cloud/bricksllm/tree/main/internal/encryptor"
    click Key "https://github.com/bricks-cloud/bricksllm/tree/main/internal/key"
    click Key "https://github.com/bricks-cloud/bricksllm/blob/main/internal/manager/key.go"
    click Infra "https://github.com/bricks-cloud/bricksllm/blob/main/docker-compose.yml"

    %% Styles
    classDef client fill:#ffcccc,stroke:#333,stroke-width:2px;
    classDef server fill:#cce5ff,stroke:#333,stroke-width:2px;
    classDef module fill:#d5f5e3,stroke:#333,stroke-width:2px;
    classDef provider fill:#f9e79f,stroke:#333,stroke-width:2px;
    classDef database fill:#fadbd8,stroke:#333,stroke-width:2px;
    classDef infra fill:#d6eaf8,stroke:#333,stroke-width:2px;
```
