# ss-vip/api-gateway architecture

> Architecture diagram of the GitHub repository ss-vip/api-gateway, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ss-vip/api-gateway
- Repository: https://github.com/ss-vip/api-gateway

## Overview

1. **Deployment/runtime boundary**
   - This is a **Cloudflare Workers** service, not a traditional server.
   - `src/worker.js` is the runtime entrypoint and wires together the API gateway, admin dashboard, and maintenance endpoints.
   - Persistence is split between **D1** for durable state and **in-memory globals** for short-lived caching and rate buffers.

2. **Primary API surface**
   - The main product is an **OpenAI-compatible proxy gateway**.
   - Clients call endpoints like `/v1/chat/completions`, and the worker forwards requests to upstream providers after routing decisions and policy checks.
   - The README indicates the gateway accepts standard OpenAI-style request bodies, including streaming.

3. **Request orchestration layer**
   - `src/gateway.js` is the central control plane for API traffic.
   - It is responsible for authentication/token checks, model-aware channel selection, forwarding requests, and applying response handling rules.
   - This is where the “smart routing” behavior lives, rather than in the provider implementation.

4. **Channel/provider abstraction**
   - `src/lib/providers/openai.js` is the concrete provider adapter for OpenAI-compatible upstreams.
   - The architecture implies multiple “channels” or upstream endpoints can be managed, each with independent health, capability, and cooldown state.
   - The gateway treats providers as pluggable backends behind a common interface.

5. **Routing and failure management**
   - Channel selection is described as **weighted random with dynamic degradation**.
   - Weights are influenced by model matching, latency, RPM/RPD availability, and error history.
   - Error handling includes **exponential backoff cooldowns**, excluding a channel after repeated failures, and gradual recovery via sliding-window success tracking.

6. **Capability learning**
   - The system remembers per-channel support for features like **vision, tools, and streaming**.
   - Capability detection happens during testing or live calls, and those observations feed back into routing decisions.
   - This creates a feedback loop between request execution and future selection quality.

7. **State and persistence**
   - `src/lib/db.js` is the D1 database access layer.
   - `src/lib/schema.js` centralizes the schema definition; the README says the worker can auto-update schema at startup.
   - The architecture intentionally uses `SELECT *` internally so schema additions do not require broad query rewrites, while the dashboard uses explicit columns for stability.

8. **Caching and operational counters**
   - The gateway keeps **channel status cached for 60 seconds** to reduce D1 reads.
   - RPM/RPD tracking is held in an in-memory `rateBuffer` and periodically persisted.
   - This means operational metrics are split between fast ephemeral state in the worker instance and durable state in D1.

9. **Maintenance and health flow**
   - `src/routes/maintenance.js` provides the `/health` endpoint.
   - This endpoint is not just a status check: it also performs cleanup and recovery tasks such as expiring cooldowns, flushing rate buffers, and reporting channel statistics.
   - The README recommends calling it every 5 minutes via a scheduled job, so it acts like a maintenance tick for the system.

10. **Admin/operator interface**
   - `src/dashboard/index.js`, `src/dashboard/portal.html`, and `src/dashboard/resources.js` implement the `/admin` management UI.
   - The dashboard covers channel CRUD, JSON import/export backups, initial password setup, client token management, and health monitoring.
   - This is a separate operator boundary from the API gateway: humans manage configuration here, while clients only interact with the OpenAI-compatible endpoints.

11. **Response policy layer**
   - The gateway also supports **content filtering** by keyword.
   - Filtering can either remove matched content or truncate the response after a match, depending on configuration.
   - This happens after upstream execution, so it is part of the outbound response path rather than pre-routing.

12. **Overall data flow**
   - External client request → worker entrypoint → gateway auth/routing → cached/D1 channel lookup → upstream provider call → error/capability/latency observation → optional filtering → client response.
   - Separately, operator actions flow through the dashboard into D1, and `/health` performs periodic maintenance that keeps routing state fresh.

13. **Key architectural boundaries to draw**
   - **Client traffic vs admin traffic**: API proxy endpoints are distinct from `/admin`.
   - **Ephemeral state vs durable state**: cache/rate buffers live in Worker memory; authoritative configuration and history live in D1.
   - **Control plane vs data plane**: dashboard, schema, and health tasks manage the system; gateway request forwarding carries user traffic.

Main entry points: [`src/gateway.js`](https://github.com/ss-vip/api-gateway/blob/main/src/gateway.js) (Gateway), [`src/lib/db.js`](https://github.com/ss-vip/api-gateway/blob/main/src/lib/db.js) (D1 Layer), [`src/dashboard/index.js`](https://github.com/ss-vip/api-gateway/blob/main/src/dashboard/index.js) (Admin UI), [`src/routes/maintenance.js`](https://github.com/ss-vip/api-gateway/blob/main/src/routes/maintenance.js) (Health)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_worker["Worker<br/>entrypoint<br/>[worker.js]"]
  node_gateway["Gateway<br/>control plane<br/>[gateway.js]"]
  node_provider_openai["OpenAI Adapter<br/>provider adapter<br/>[openai.js]"]
  node_constants["Constants<br/>shared config<br/>[constants.js]"]
  node_rate_buffer["Rate Buffer<br/>ephemeral state"]
  node_cache(("Status Cache<br/>ephemeral cache"))
  node_routing{{"Routing<br/>selection logic"}}
  node_capability_learning(("Capability Learning<br/>feedback loop"))
  node_filtering{{"Filtering<br/>response policy"}}
end

subgraph group_admin["Admin"]
  node_maintenance["Health<br/>maintenance route<br/>[maintenance.js]"]
  node_dashboard_index["Admin UI<br/>dashboard app<br/>[index.js]"]
  node_portal_html["Portal<br/>ui shell<br/>[portal.html]"]
  node_resources_js["Resources<br/>ui logic<br/>[resources.js]"]
end

subgraph group_data["Data"]
  node_db[("D1 Layer<br/>database access<br/>[db.js]")]
  node_schema["Schema<br/>[schema.js]"]
  node_channel_state[("Channel State<br/>durable state")]
end

node_worker -->|"routes API"| node_gateway
node_worker -->|"serves admin"| node_dashboard_index
node_worker -->|"exposes health"| node_maintenance
node_gateway -->|"uses limits"| node_constants
node_gateway -->|"reads status"| node_cache
node_gateway -->|"reads/writes"| node_db
node_gateway -->|"selects channel"| node_routing
node_routing -->|"calls upstream"| node_provider_openai
node_provider_openai -->|"observes support"| node_capability_learning
node_provider_openai -->|"updates counters"| node_rate_buffer
node_rate_buffer -->|"flushes"| node_db
node_db -->|"stores state"| node_channel_state
node_schema -->|"defines tables"| node_db
node_maintenance -->|"repairs state"| node_db
node_maintenance -->|"flushes buffers"| node_rate_buffer
node_maintenance -->|"expires cache"| node_cache
node_dashboard_index -->|"renders shell"| node_portal_html
node_dashboard_index -->|"loads UI"| node_resources_js
node_dashboard_index -->|"manages config"| node_db
node_dashboard_index -->|"edits channels"| node_channel_state
node_filtering -->|"post-processes"| node_gateway
node_gateway -->|"applies policy"| node_filtering
node_capability_learning -->|"improves choices"| node_routing

click node_worker "https://github.com/ss-vip/api-gateway/blob/main/src/worker.js"
click node_gateway "https://github.com/ss-vip/api-gateway/blob/main/src/gateway.js"
click node_provider_openai "https://github.com/ss-vip/api-gateway/blob/main/src/lib/providers/openai.js"
click node_constants "https://github.com/ss-vip/api-gateway/blob/main/src/lib/constants.js"
click node_maintenance "https://github.com/ss-vip/api-gateway/blob/main/src/routes/maintenance.js"
click node_dashboard_index "https://github.com/ss-vip/api-gateway/blob/main/src/dashboard/index.js"
click node_portal_html "https://github.com/ss-vip/api-gateway/blob/main/src/dashboard/portal.html"
click node_resources_js "https://github.com/ss-vip/api-gateway/blob/main/src/dashboard/resources.js"
click node_db "https://github.com/ss-vip/api-gateway/blob/main/src/lib/db.js"
click node_schema "https://github.com/ss-vip/api-gateway/blob/main/src/lib/schema.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_worker,node_gateway,node_provider_openai,node_constants,node_rate_buffer,node_cache,node_routing,node_capability_learning,node_filtering toneBlue
class node_maintenance,node_dashboard_index,node_portal_html,node_resources_js toneAmber
class node_db,node_schema,node_channel_state toneMint
```

## Components

### Runtime

Worker execution and request handling

- [Worker](https://github.com/ss-vip/api-gateway/blob/main/src/worker.js): entrypoint · Cloudflare Workers entrypoint wiring API, admin, and maintenance surfaces · `src/worker.js`
- [Gateway](https://github.com/ss-vip/api-gateway/blob/main/src/gateway.js): control plane · Central request orchestrator for authentication, routing, forwarding, and response policy · `src/gateway.js`
- [OpenAI Adapter](https://github.com/ss-vip/api-gateway/blob/main/src/lib/providers/openai.js): provider adapter · Concrete upstream adapter for OpenAI-compatible providers · `src/lib/providers/openai.js`
- [Constants](https://github.com/ss-vip/api-gateway/blob/main/src/lib/constants.js): shared config · Shared limits, labels, and runtime constants · `src/lib/constants.js`
- **Rate Buffer**: ephemeral state · In-memory RPM/RPD tracking and short-lived operational counters
- **Status Cache**: ephemeral cache · Short-lived cached channel status to reduce D1 reads
- **Routing**: selection logic · Weighted channel selection with degradation, cooldowns, and capability awareness
- **Capability Learning**: feedback loop · Learns per-channel support for vision, tools, and streaming from tests and live traffic
- **Filtering**: response policy · Post-upstream content filtering that removes or truncates matched content

### Admin

Operator UI and maintenance endpoints

- [Health](https://github.com/ss-vip/api-gateway/blob/main/src/routes/maintenance.js): maintenance route · Health endpoint that also expires cooldowns, flushes buffers, and reports channel stats · `src/routes/maintenance.js`
- [Admin UI](https://github.com/ss-vip/api-gateway/blob/main/src/dashboard/index.js): dashboard app · Admin portal entrypoint for channel and token management · `src/dashboard/index.js`
- [Portal](https://github.com/ss-vip/api-gateway/blob/main/src/dashboard/portal.html): ui shell · Admin dashboard HTML shell · `src/dashboard/portal.html`
- [Resources](https://github.com/ss-vip/api-gateway/blob/main/src/dashboard/resources.js): ui logic · Admin dashboard client-side resources and interactions · `src/dashboard/resources.js`

### Data

Durable state and persistence layer

- [D1 Layer](https://github.com/ss-vip/api-gateway/blob/main/src/lib/db.js): database access · Database access layer for durable state and operational records · `src/lib/db.js`
- [Schema](https://github.com/ss-vip/api-gateway/blob/main/src/lib/schema.js): schema definition · Central schema definition with startup migration/update support · `src/lib/schema.js`
- **Channel State**: durable state · Stored channel configuration, health, and capability history

## Connections

- Worker → Gateway: routes API · Worker hands client API traffic to the gateway
- Worker → Admin UI: serves admin · Worker exposes the operator dashboard under /admin
- Worker → Health: exposes health · Worker mounts the maintenance and health endpoint
- Gateway → Constants: uses limits · Gateway reads shared configuration and operational constants
- Gateway → Status Cache: reads status · Gateway consults cached channel state before hitting D1
- Gateway → D1 Layer: reads/writes · Gateway persists and loads routing and operational state through D1
- Gateway → Routing: selects channel · Gateway applies weighted routing and failure-aware selection
- Routing → OpenAI Adapter: calls upstream · Selected channel is executed through the provider adapter
- OpenAI Adapter → Capability Learning: observes support · Adapter surfaces capability and runtime observations back into learning
- OpenAI Adapter → Rate Buffer: updates counters · Provider execution contributes to in-memory rate tracking
- Rate Buffer → D1 Layer: flushes · Buffered counters are periodically persisted to D1
- D1 Layer → Channel State: stores state · D1 holds channel configuration, health, and history
- Schema → D1 Layer: defines tables · Schema governs the database structure used by the persistence layer
- Health → D1 Layer: repairs state · Health flow performs cleanup and recovery work against durable state
- Health → Rate Buffer: flushes buffers · Maintenance tick persists and clears buffered operational counters
- Health → Status Cache: expires cache · Health flow helps refresh short-lived cached routing state
- Admin UI → Portal: renders shell · Admin app serves the portal HTML
- Admin UI → Resources: loads UI · Admin app uses dashboard resources for interactions
- Admin UI → D1 Layer: manages config · Dashboard creates, updates, exports, and imports operator state in D1
- Admin UI → Channel State: edits channels · Dashboard manages channel records and related operator data
- Filtering → Gateway: post-processes · Gateway applies response filtering after upstream execution
- Gateway → Filtering: applies policy · Outbound response path can remove or truncate matched content
- Capability Learning → Routing: improves choices · Learned capabilities feed future channel selection
