How tbphp/gpt-load works
1. What this repo is
GPT-Load is a single Go service that combines two major surfaces: a management API + web console and a transparent AI proxy. The backend serves both the admin UI and the proxy endpoint, while the Vue 3 frontend is a separate SPA built under web/.
2. Runtime and bootstrap path
main.go boots the app through internal/app and wiring in internal/container. That composition layer assembles config, database, store/cache, HTTP client manager, router, middleware, and services into one runtime. The app is designed to run standalone, via Docker, or in clustered mode.
3. Configuration model
The architecture uses two configuration tiers:
- Static env-based config for server, auth, DB DSN, Redis DSN, logging, CORS, proxy env vars, etc.
- Dynamic DB-backed settings for system settings and group overrides.
Group settings can override system settings for request timeouts, retries, key validation, and proxy behavior. This hot-reload behavior is central to the design.
4. Persistence and migration layer
internal/db and internal/db/migrations manage the schema and migrations. The supported persistence backends are SQLite, MySQL, and PostgreSQL. API keys can be stored encrypted using internal/encryption, and there is a dedicated migration command (internal/commands/migrate.go) for encrypting, decrypting, or rekeying stored secrets.
5. Management API boundary
internal/router routes requests into internal/handler modules for dashboards, keys, groups, settings, logs, integrations, and tasks. These handlers are thin HTTP boundaries: they translate API requests into service calls and return standardized responses from internal/response. The management surface is authenticated separately from the proxy surface.
6. Business logic layer
The core app logic sits in internal/services. This is where key import/delete, manual validation, group aggregation, subgroup management, log ingestion/cleanup, and task execution happen. These services own the domain rules; handlers do not contain the main business logic.
7. Proxy request flow
Requests to /proxy/{group}/{original_path} go through internal/proxy. The proxy layer preserves native upstream API formats and authentication style, so clients only swap endpoint and use a proxy key. It supports OpenAI-style, Gemini, Anthropic, and OpenAI Responses flows without forcing a new client contract.
8. Provider adapters
internal/channel contains provider-specific channel implementations:
- openai_channel.go
- openai_response_channel.go
- gemini_channel.go
- anthropic_channel.go
These adapters encapsulate provider-specific request/response handling and are chosen by factory.go. This is the main boundary where generic proxy logic becomes provider-specific.
9. Load balancing, retries, and failover
Key selection and resilience are handled by internal/keypool, internal/failover, and internal/httpclient. The system rotates through keys, validates them in the background, blacklists failing keys after thresholds, and retries across different keys when configured. Failure classification is partly status-code based, so upstream errors drive key health decisions.
10. Cluster and cache coordination
The repo supports distributed deployment with leader/follower behavior. internal/store abstracts cache/state storage across memory and Redis, and internal/syncer/cache_syncer.go suggests cache/state propagation across nodes. Redis is optional for single-node use but required for reliable multi-node coordination.
11. Logging, monitoring, and background maintenance
Request logs are first-class data, with services for request logging and cleanup plus scheduled tasks. The app also exposes dashboard data for operational visibility. Background maintenance includes cron-based key checks and log retention enforcement, which keeps runtime state aligned with DB-backed policy.
12. Cross-cutting infrastructure
internal/middleware and internal/i18n provide auth, request handling, and localization. internal/errors centralizes error parsing and classification, including ignorable and uncounted errors that affect retry/blacklist logic. internal/utils contains support functions for headers, compression, logging, passwords, config parsing, and strings.
13. Frontend architecture
The Vue app under web/ is an SPA with routes for Dashboard, Keys, Logs, Settings, and Login. It talks to the Go backend through typed API clients in web/src/api and uses shared state/utilities for theme, auth/session state, clipboard, and display formatting. The frontend is organized around management workflows, not around the proxy traffic itself.
14. How to draw the main architecture diagram
The key boxes are:
- Browser / admin user → Vue SPA → management handlers/services → DB/Redis
- API client → /proxy/... → proxy/router/channel adapters → upstream AI providers
- Background workers/tasks → key validation, log cleanup, cache sync
- Persistent stores → SQL DB for config/keys/logs, Redis or memory for coordination/cache
This repo is fundamentally a multi-surface control plane plus data plane proxy, with provider adapters and key-management logic at the center.
Main entry points: container.go Container · dashboard.ts Web API · key_service.go Services · server.go Proxy
Components 22
Go backend
Single Go service providing management API and proxy runtime
- main · Process bootstrap
main.go - App · Wires the runtime and service composition
internal/app/app.go - Container · Assembles config, DB, store, client manager, router, and services
internal/container/container.go - Config · Static environment config and dynamic system settings
internal/config/manager.go - Router · Dispatches management and proxy requests
internal/router/router.go - Handlers · Dashboard, keys, groups, settings, logs, integrations, and tasks endpoints
internal/handler/handler.go - Services · Core business rules for keys, groups, logs, and tasks
internal/services/key_service.go - Proxy · Transparent /proxy/{group}/{path} request flow
internal/proxy/server.go - Channels · OpenAI, Responses, Gemini, and Anthropic adapters
internal/channel/factory.go
Vue SPA
Admin console shipped separately under web/
- Dashboard · Operational overview in the web console
web/src/views/Dashboard.vue - Keys UI · Key and group management workflows
web/src/views/Keys.vue - Logs UI · Request log review and retention workflows
web/src/views/Logs.vue - Settings UI · System settings and admin configuration
web/src/views/Settings.vue - Web API · Frontend client layer for backend management endpoints
web/src/api/dashboard.ts
Data layer
Database, cache, and encryption boundaries
- Database · SQL database access and migrations
internal/db/database.go - Migrations · Schema evolution and migration commands
internal/db/migrations/migration.go - Encryption · Encrypts and rekeys stored secrets
internal/encryption/encryption.go - Store · Memory and Redis-backed shared state
internal/store/store.go
Background ops
Validation, cleanup, and synchronization work
- Cache sync · Propagates cache/state across nodes
internal/syncer/cache_syncer.go - Key pool · Key rotation, validation, blacklisting, and retry selection
internal/keypool/provider.go - Failover · Classifies upstream failures for retry and blacklist decisions
internal/failover/status_code_matcher.go - HTTP client · Managed outbound HTTP client pool
internal/httpclient/manager.go
Connections 24
- main → App: starts · main boots the application
- App → Container: builds · application initialization composes the runtime
- Container → Config: loads · runtime receives static and dynamic settings
- Container → Database: connects · persistent state is wired into the runtime
- Container → Store: creates · shared cache/state backend is selected
- Database → Migrations: applies · schema versioning is managed separately
- Container → Encryption: uses · secret handling is available to persistence and commands
- Container → Router: exposes · HTTP surface is mounted from the composition root
- Router → Handlers: routes admin · management endpoints are dispatched to handlers
- Handlers → Services: calls · handlers delegate business rules
- Services → Database: reads/writes · domain state is persisted in SQL
- Services → Store: shares state · runtime coordination and cached state flow through store
- Store → Cache sync: syncs · distributed mode propagates cache/state changes
- Router → Proxy: routes proxy · proxy surface is separated from admin API
- Proxy → Channels: adapts · provider-specific request/response handling
- Proxy → Key pool: selects keys · proxy requests are assigned to upstream keys
- Key pool → Failover: classifies · upstream failures affect retry and blacklist decisions
- Proxy → HTTP client: egresses · outbound calls use managed HTTP clients
- Dashboard → Web API: uses · console talks to backend APIs
- Keys UI → Web API: uses · key management UI consumes typed clients
- Logs UI → Web API: uses · log views consume typed clients
- Settings UI → Web API: uses · settings views consume typed clients
- Web API → Handlers: calls · frontend clients target management endpoints
- Services → Key pool: drives · validation and import flows influence key rotation state