# tbphp/gpt-load architecture

> Architecture diagram of the GitHub repository tbphp/gpt-load, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tbphp/gpt-load
- Repository: https://github.com/tbphp/gpt-load

## Overview

1. **What this repo is**
   GPT-Load is a single Go service that combines two major surfaces: a **management API + web console** and a **transparent AI proxy**. The backend serves both the admin UI and the proxy endpoint, while the Vue 3 frontend is a separate SPA built under `web/`.

2. **Runtime and bootstrap path**
   `main.go` boots the app through `internal/app` and wiring in `internal/container`. That composition layer assembles config, database, store/cache, HTTP client manager, router, middleware, and services into one runtime. The app is designed to run standalone, via Docker, or in clustered mode.

3. **Configuration model**
   The architecture uses **two configuration tiers**:
   - **Static env-based config** for server, auth, DB DSN, Redis DSN, logging, CORS, proxy env vars, etc.
   - **Dynamic DB-backed settings** for system settings and group overrides.
   Group settings can override system settings for request timeouts, retries, key validation, and proxy behavior. This hot-reload behavior is central to the design.

4. **Persistence and migration layer**
   `internal/db` and `internal/db/migrations` manage the schema and migrations. The supported persistence backends are SQLite, MySQL, and PostgreSQL. API keys can be stored encrypted using `internal/encryption`, and there is a dedicated migration command (`internal/commands/migrate.go`) for encrypting, decrypting, or rekeying stored secrets.

5. **Management API boundary**
   `internal/router` routes requests into `internal/handler` modules for dashboards, keys, groups, settings, logs, integrations, and tasks. These handlers are thin HTTP boundaries: they translate API requests into service calls and return standardized responses from `internal/response`. The management surface is authenticated separately from the proxy surface.

6. **Business logic layer**
   The core app logic sits in `internal/services`. This is where key import/delete, manual validation, group aggregation, subgroup management, log ingestion/cleanup, and task execution happen. These services own the domain rules; handlers do not contain the main business logic.

7. **Proxy request flow**
   Requests to `/proxy/{group}/{original_path}` go through `internal/proxy`. The proxy layer preserves native upstream API formats and authentication style, so clients only swap endpoint and use a proxy key. It supports OpenAI-style, Gemini, Anthropic, and OpenAI Responses flows without forcing a new client contract.

8. **Provider adapters**
   `internal/channel` contains provider-specific channel implementations:
   - `openai_channel.go`
   - `openai_response_channel.go`
   - `gemini_channel.go`
   - `anthropic_channel.go`
   These adapters encapsulate provider-specific request/response handling and are chosen by `factory.go`. This is the main boundary where generic proxy logic becomes provider-specific.

9. **Load balancing, retries, and failover**
   Key selection and resilience are handled by `internal/keypool`, `internal/failover`, and `internal/httpclient`. The system rotates through keys, validates them in the background, blacklists failing keys after thresholds, and retries across different keys when configured. Failure classification is partly status-code based, so upstream errors drive key health decisions.

10. **Cluster and cache coordination**
   The repo supports distributed deployment with leader/follower behavior. `internal/store` abstracts cache/state storage across memory and Redis, and `internal/syncer/cache_syncer.go` suggests cache/state propagation across nodes. Redis is optional for single-node use but required for reliable multi-node coordination.

11. **Logging, monitoring, and background maintenance**
   Request logs are first-class data, with services for request logging and cleanup plus scheduled tasks. The app also exposes dashboard data for operational visibility. Background maintenance includes cron-based key checks and log retention enforcement, which keeps runtime state aligned with DB-backed policy.

12. **Cross-cutting infrastructure**
   `internal/middleware` and `internal/i18n` provide auth, request handling, and localization. `internal/errors` centralizes error parsing and classification, including ignorable and uncounted errors that affect retry/blacklist logic. `internal/utils` contains support functions for headers, compression, logging, passwords, config parsing, and strings.

13. **Frontend architecture**
   The Vue app under `web/` is an SPA with routes for Dashboard, Keys, Logs, Settings, and Login. It talks to the Go backend through typed API clients in `web/src/api` and uses shared state/utilities for theme, auth/session state, clipboard, and display formatting. The frontend is organized around management workflows, not around the proxy traffic itself.

14. **How to draw the main architecture diagram**
   The key boxes are:
   - **Browser / admin user** → Vue SPA → management handlers/services → DB/Redis
   - **API client** → `/proxy/...` → proxy/router/channel adapters → upstream AI providers
   - **Background workers/tasks** → key validation, log cleanup, cache sync
   - **Persistent stores** → SQL DB for config/keys/logs, Redis or memory for coordination/cache
   This repo is fundamentally a multi-surface control plane plus data plane proxy, with provider adapters and key-management logic at the center.

Main entry points: [`internal/container/container.go`](https://github.com/tbphp/gpt-load/blob/main/internal/container/container.go) (Container), [`web/src/api/dashboard.ts`](https://github.com/tbphp/gpt-load/blob/main/web/src/api/dashboard.ts) (Web API), [`internal/services/key_service.go`](https://github.com/tbphp/gpt-load/blob/main/internal/services/key_service.go) (Services), [`internal/proxy/server.go`](https://github.com/tbphp/gpt-load/blob/main/internal/proxy/server.go) (Proxy)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_backend["Go backend"]
  node_main["main<br/>entrypoint<br/>[main.go]"]
  node_app["App<br/>bootstrap<br/>[app.go]"]
  node_container["Container<br/>composition<br/>[container.go]"]
  node_config["Config<br/>settings<br/>[manager.go]"]
  node_router["Router<br/>http routing<br/>[router.go]"]
  node_handlers["Handlers<br/>api boundary<br/>[handler.go]"]
  node_services["Services<br/>domain logic<br/>[key_service.go]"]
  node_proxy["Proxy<br/>data plane<br/>[server.go]"]
  node_channels["Channels<br/>provider adapters<br/>[factory.go]"]
end

subgraph group_frontend["Vue SPA"]
  node_dashboard["Dashboard<br/>ui feature<br/>[Dashboard.vue]"]
  node_keysui["Keys UI<br/>ui feature<br/>[Keys.vue]"]
  node_logsui["Logs UI<br/>ui feature<br/>[Logs.vue]"]
  node_settingsui["Settings UI<br/>ui feature<br/>[Settings.vue]"]
  node_api["Web API<br/>typed client<br/>[dashboard.ts]"]
end

subgraph group_data["Data layer"]
  node_db[("Database<br/>persistence<br/>[database.go]")]
  node_migrations["Migrations<br/>schema<br/>[migration.go]"]
  node_encryption["Encryption<br/>secret crypto<br/>[encryption.go]"]
  node_store[("Store<br/>cache/state<br/>[store.go]")]
end

subgraph group_ops["Background ops"]
  node_syncer["Cache sync<br/>cluster sync<br/>[cache_syncer.go]"]
  node_keypool["Key pool<br/>routing/resilience<br/>[provider.go]"]
  node_failover["Failover<br/>resilience"]
  node_httpclient["HTTP client<br/>upstream client<br/>[manager.go]"]
end

node_main -->|"starts"| node_app
node_app -->|"builds"| node_container
node_container -->|"loads"| node_config
node_container -->|"connects"| node_db
node_container -->|"creates"| node_store
node_db -->|"applies"| node_migrations
node_container -->|"uses"| node_encryption
node_container -->|"exposes"| node_router
node_router -->|"routes admin"| node_handlers
node_handlers -->|"calls"| node_services
node_services -->|"reads/writes"| node_db
node_services -->|"shares state"| node_store
node_store -->|"syncs"| node_syncer
node_router -->|"routes proxy"| node_proxy
node_proxy -->|"adapts"| node_channels
node_proxy -->|"selects keys"| node_keypool
node_keypool -->|"classifies"| node_failover
node_proxy -->|"egresses"| node_httpclient
node_dashboard -->|"uses"| node_api
node_keysui -->|"uses"| node_api
node_logsui -->|"uses"| node_api
node_settingsui -->|"uses"| node_api
node_api -->|"calls"| node_handlers
node_services -->|"drives"| node_keypool

click node_main "https://github.com/tbphp/gpt-load/blob/main/main.go"
click node_app "https://github.com/tbphp/gpt-load/blob/main/internal/app/app.go"
click node_container "https://github.com/tbphp/gpt-load/blob/main/internal/container/container.go"
click node_config "https://github.com/tbphp/gpt-load/blob/main/internal/config/manager.go"
click node_db "https://github.com/tbphp/gpt-load/blob/main/internal/db/database.go"
click node_migrations "https://github.com/tbphp/gpt-load/blob/main/internal/db/migrations/migration.go"
click node_encryption "https://github.com/tbphp/gpt-load/blob/main/internal/encryption/encryption.go"
click node_store "https://github.com/tbphp/gpt-load/blob/main/internal/store/store.go"
click node_syncer "https://github.com/tbphp/gpt-load/blob/main/internal/syncer/cache_syncer.go"
click node_router "https://github.com/tbphp/gpt-load/blob/main/internal/router/router.go"
click node_handlers "https://github.com/tbphp/gpt-load/blob/main/internal/handler/handler.go"
click node_services "https://github.com/tbphp/gpt-load/blob/main/internal/services/key_service.go"
click node_proxy "https://github.com/tbphp/gpt-load/blob/main/internal/proxy/server.go"
click node_channels "https://github.com/tbphp/gpt-load/blob/main/internal/channel/factory.go"
click node_keypool "https://github.com/tbphp/gpt-load/blob/main/internal/keypool/provider.go"
click node_failover "https://github.com/tbphp/gpt-load/blob/main/internal/failover/status_code_matcher.go"
click node_httpclient "https://github.com/tbphp/gpt-load/blob/main/internal/httpclient/manager.go"
click node_dashboard "https://github.com/tbphp/gpt-load/blob/main/web/src/views/Dashboard.vue"
click node_keysui "https://github.com/tbphp/gpt-load/blob/main/web/src/views/Keys.vue"
click node_logsui "https://github.com/tbphp/gpt-load/blob/main/web/src/views/Logs.vue"
click node_settingsui "https://github.com/tbphp/gpt-load/blob/main/web/src/views/Settings.vue"
click node_api "https://github.com/tbphp/gpt-load/blob/main/web/src/api/dashboard.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_app,node_container,node_config,node_router,node_handlers,node_services,node_proxy,node_channels toneBlue
class node_dashboard,node_keysui,node_logsui,node_settingsui,node_api toneAmber
class node_db,node_migrations,node_encryption,node_store toneMint
class node_syncer,node_keypool,node_failover,node_httpclient toneRose
```

## Components

### Go backend

Single Go service providing management API and proxy runtime

- [main](https://github.com/tbphp/gpt-load/blob/main/main.go): entrypoint · Process bootstrap · `main.go`
- [App](https://github.com/tbphp/gpt-load/blob/main/internal/app/app.go): bootstrap · Wires the runtime and service composition · `internal/app/app.go`
- [Container](https://github.com/tbphp/gpt-load/blob/main/internal/container/container.go): composition · Assembles config, DB, store, client manager, router, and services · `internal/container/container.go`
- [Config](https://github.com/tbphp/gpt-load/blob/main/internal/config/manager.go): settings · Static environment config and dynamic system settings · `internal/config/manager.go`
- [Router](https://github.com/tbphp/gpt-load/blob/main/internal/router/router.go): http routing · Dispatches management and proxy requests · `internal/router/router.go`
- [Handlers](https://github.com/tbphp/gpt-load/blob/main/internal/handler/handler.go): api boundary · Dashboard, keys, groups, settings, logs, integrations, and tasks endpoints · `internal/handler/handler.go`
- [Services](https://github.com/tbphp/gpt-load/blob/main/internal/services/key_service.go): domain logic · Core business rules for keys, groups, logs, and tasks · `internal/services/key_service.go`
- [Proxy](https://github.com/tbphp/gpt-load/blob/main/internal/proxy/server.go): data plane · Transparent /proxy/{group}/{path} request flow · `internal/proxy/server.go`
- [Channels](https://github.com/tbphp/gpt-load/blob/main/internal/channel/factory.go): provider adapters · OpenAI, Responses, Gemini, and Anthropic adapters · `internal/channel/factory.go`

### Vue SPA

Admin console shipped separately under web/

- [Dashboard](https://github.com/tbphp/gpt-load/blob/main/web/src/views/Dashboard.vue): ui feature · Operational overview in the web console · `web/src/views/Dashboard.vue`
- [Keys UI](https://github.com/tbphp/gpt-load/blob/main/web/src/views/Keys.vue): ui feature · Key and group management workflows · `web/src/views/Keys.vue`
- [Logs UI](https://github.com/tbphp/gpt-load/blob/main/web/src/views/Logs.vue): ui feature · Request log review and retention workflows · `web/src/views/Logs.vue`
- [Settings UI](https://github.com/tbphp/gpt-load/blob/main/web/src/views/Settings.vue): ui feature · System settings and admin configuration · `web/src/views/Settings.vue`
- [Web API](https://github.com/tbphp/gpt-load/blob/main/web/src/api/dashboard.ts): typed client · Frontend client layer for backend management endpoints · `web/src/api/dashboard.ts`

### Data layer

Database, cache, and encryption boundaries

- [Database](https://github.com/tbphp/gpt-load/blob/main/internal/db/database.go): persistence · SQL database access and migrations · `internal/db/database.go`
- [Migrations](https://github.com/tbphp/gpt-load/blob/main/internal/db/migrations/migration.go): schema · Schema evolution and migration commands · `internal/db/migrations/migration.go`
- [Encryption](https://github.com/tbphp/gpt-load/blob/main/internal/encryption/encryption.go): secret crypto · Encrypts and rekeys stored secrets · `internal/encryption/encryption.go`
- [Store](https://github.com/tbphp/gpt-load/blob/main/internal/store/store.go): cache/state · Memory and Redis-backed shared state · `internal/store/store.go`

### Background ops

Validation, cleanup, and synchronization work

- [Cache sync](https://github.com/tbphp/gpt-load/blob/main/internal/syncer/cache_syncer.go): cluster sync · Propagates cache/state across nodes · `internal/syncer/cache_syncer.go`
- [Key pool](https://github.com/tbphp/gpt-load/blob/main/internal/keypool/provider.go): routing/resilience · Key rotation, validation, blacklisting, and retry selection · `internal/keypool/provider.go`
- [Failover](https://github.com/tbphp/gpt-load/blob/main/internal/failover/status_code_matcher.go): resilience · Classifies upstream failures for retry and blacklist decisions · `internal/failover/status_code_matcher.go`
- [HTTP client](https://github.com/tbphp/gpt-load/blob/main/internal/httpclient/manager.go): upstream client · Managed outbound HTTP client pool · `internal/httpclient/manager.go`

## Connections

- main → App: starts · main boots the application
- App → Container: builds · application initialization composes the runtime
- Container → Config: loads · runtime receives static and dynamic settings
- Container → Database: connects · persistent state is wired into the runtime
- Container → Store: creates · shared cache/state backend is selected
- Database → Migrations: applies · schema versioning is managed separately
- Container → Encryption: uses · secret handling is available to persistence and commands
- Container → Router: exposes · HTTP surface is mounted from the composition root
- Router → Handlers: routes admin · management endpoints are dispatched to handlers
- Handlers → Services: calls · handlers delegate business rules
- Services → Database: reads/writes · domain state is persisted in SQL
- Services → Store: shares state · runtime coordination and cached state flow through store
- Store → Cache sync: syncs · distributed mode propagates cache/state changes
- Router → Proxy: routes proxy · proxy surface is separated from admin API
- Proxy → Channels: adapts · provider-specific request/response handling
- Proxy → Key pool: selects keys · proxy requests are assigned to upstream keys
- Key pool → Failover: classifies · upstream failures affect retry and blacklist decisions
- Proxy → HTTP client: egresses · outbound calls use managed HTTP clients
- Dashboard → Web API: uses · console talks to backend APIs
- Keys UI → Web API: uses · key management UI consumes typed clients
- Logs UI → Web API: uses · log views consume typed clients
- Settings UI → Web API: uses · settings views consume typed clients
- Web API → Handlers: calls · frontend clients target management endpoints
- Services → Key pool: drives · validation and import flows influence key rotation state
