# mnfst/llm-gateway architecture

> Architecture diagram of the GitHub repository mnfst/llm-gateway, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mnfst/llm-gateway
- Repository: https://github.com/mnfst/llm-gateway

## Overview

Manifest is an OpenAI-compatible LLM gateway: users connect provider credentials or subscriptions, configure model routing, then send requests through one endpoint to a selected model. The gateway supports fallback and self-healing, records request bodies, and exposes analytics and usage controls. Its dashboard and CLI manage gateway configuration; MCP offers a separate agent-facing interface. The README documents these capabilities, while sampled backend excerpts confirm provider discovery, routing configuration, proxy response handling, and analytics queries. Unsampled subsystem internals are represented at their existing directory paths.

Main entry points: [`packages/backend/src/entities`](https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/entities) (Application database), [`packages/backend/src/routing/provider.controller.ts`](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/provider.controller.ts) (Provider management API), [`packages/backend/src/routing/proxy/proxy.controller.ts`](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/proxy/proxy.controller.ts) (OpenAI-compatible proxy), [`packages/frontend/src/pages/Routing.tsx`](https://github.com/mnfst/llm-gateway/blob/main/packages/frontend/src/pages/Routing.tsx) (Web dashboard)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_access["User access"]
  node_dashboard["Web dashboard<br/>[Routing.tsx]"]
  node_cli["CLI configuration<br/>[routing.ts]"]
  node_mcp["MCP interface<br/>[mcp.controller.ts]"]
  node_auth["Authentication"]
end

subgraph group_routing["Provider and routing"]
  node_provider_api["Provider management API"]
  node_provider_service["Provider connections"]
  node_discovery["Model discovery"]
  node_oauth["Subscription sign-in"]
  node_route_config["Routing configuration"]
end

subgraph group_gateway["Gateway runtime"]
  node_proxy["OpenAI-compatible proxy"]
  node_provider_clients["Provider adapters<br/>[provider-client.ts]"]
  node_fallback["Fallback and healing"]
  node_recording["Request recording"]
end

subgraph group_operations["Observability and control"]
  node_database[("Application database")]
  node_analytics["Analytics queries"]
  node_usage_controls["Usage and notifications"]
  node_dashboard_events["Live change events"]
end

node_provider_models{{"Provider model APIs"}}
node_llm_providers{{"LLM providers"}}

node_dashboard -.->|"configures providers"| node_provider_api
node_cli -.->|"configures routes"| node_provider_api
node_dashboard -.->|"receives updates"| node_dashboard_events
node_mcp -->|"authenticates"| node_auth
node_mcp -->|"queries tenant data"| node_database
node_provider_api -->|"manages connections"| node_provider_service
node_provider_api -->|"requests models"| node_discovery
node_discovery -->|"fetches catalogs"| node_provider_models
node_oauth -.->|"authorizes subscriptions"| node_llm_providers
node_provider_service -->|"reads and writes"| node_database
node_route_config -.->|"stores routes"| node_database
node_proxy -->|"forwards requests"| node_provider_clients
node_provider_clients -->|"sends model requests"| node_llm_providers
node_proxy -.->|"recovers failures"| node_fallback
node_proxy -.->|"records exchanges"| node_recording
node_recording -.->|"persists records"| node_database
node_analytics -->|"queries messages"| node_database
node_usage_controls -.->|"tracks usage"| node_database
node_dashboard_events -.->|"notifies changes"| node_dashboard

click node_dashboard "https://github.com/mnfst/llm-gateway/blob/main/packages/frontend/src/pages/Routing.tsx"
click node_cli "https://github.com/mnfst/llm-gateway/blob/main/packages/cli/src/commands/routing.ts"
click node_mcp "https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/mcp/mcp.controller.ts"
click node_auth "https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/auth"
click node_provider_api "https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/provider.controller.ts"
click node_provider_service "https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/routing-core/provider.service.ts"
click node_discovery "https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/model-discovery/model-discovery.service.ts"
click node_oauth "https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/routing/oauth"
click node_route_config "https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/routing"
click node_proxy "https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/proxy/proxy.controller.ts"
click node_provider_clients "https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/proxy/provider-client.ts"
click node_fallback "https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/routing/autofix"
click node_recording "https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/common/services/request-recording-storage.service.ts"
click node_database "https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/entities"
click node_analytics "https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/analytics/services/messages-query.service.ts"
click node_usage_controls "https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/billing"
click node_dashboard_events "https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/common/services/ingest-event-bus.service.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard,node_cli,node_mcp,node_auth toneBlue
class node_provider_api,node_provider_service,node_discovery,node_oauth,node_route_config toneAmber
class node_proxy,node_provider_clients,node_fallback,node_recording,node_provider_models toneMint
class node_database,node_analytics,node_usage_controls,node_dashboard_events toneRose
class node_llm_providers toneIndigo
```

## Components

### User access

- [Web dashboard](https://github.com/mnfst/llm-gateway/blob/main/packages/frontend/src/pages/Routing.tsx): `packages/frontend/src/pages/Routing.tsx`
- [CLI configuration](https://github.com/mnfst/llm-gateway/blob/main/packages/cli/src/commands/routing.ts): `packages/cli/src/commands/routing.ts`
- [MCP interface](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/mcp/mcp.controller.ts): `packages/backend/src/mcp/mcp.controller.ts`
- [Authentication](https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/auth): `packages/backend/src/auth`

### Provider and routing

- [Provider management API](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/provider.controller.ts): `packages/backend/src/routing/provider.controller.ts`
- [Provider connections](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/routing-core/provider.service.ts): `packages/backend/src/routing/routing-core/provider.service.ts`
- [Model discovery](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/model-discovery/model-discovery.service.ts): `packages/backend/src/model-discovery/model-discovery.service.ts`
- [Subscription sign-in](https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/routing/oauth): `packages/backend/src/routing/oauth`
- [Routing configuration](https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/routing): `packages/backend/src/routing`

### Gateway runtime

- [OpenAI-compatible proxy](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/proxy/proxy.controller.ts): `packages/backend/src/routing/proxy/proxy.controller.ts`
- [Provider adapters](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/routing/proxy/provider-client.ts): `packages/backend/src/routing/proxy/provider-client.ts`
- [Fallback and healing](https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/routing/autofix): `packages/backend/src/routing/autofix`
- [Request recording](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/common/services/request-recording-storage.service.ts): `packages/backend/src/common/services/request-recording-storage.service.ts`

### Observability and control

- [Application database](https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/entities): `packages/backend/src/entities`
- [Analytics queries](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/analytics/services/messages-query.service.ts): `packages/backend/src/analytics/services/messages-query.service.ts`
- [Usage and notifications](https://github.com/mnfst/llm-gateway/tree/main/packages/backend/src/billing): `packages/backend/src/billing`
- [Live change events](https://github.com/mnfst/llm-gateway/blob/main/packages/backend/src/common/services/ingest-event-bus.service.ts): `packages/backend/src/common/services/ingest-event-bus.service.ts`

### Other components

- **Provider model APIs**
- **LLM providers**

## Connections

- Web dashboard → Provider management API: configures providers
- CLI configuration → Provider management API: configures routes
- Web dashboard → Live change events: receives updates
- MCP interface → Authentication: authenticates (evidence: `packages/backend/src/mcp/mcp.controller.ts`)
- MCP interface → Application database: queries tenant data (evidence: `packages/backend/src/mcp/mcp.controller.ts`)
- Provider management API → Provider connections: manages connections (evidence: `packages/backend/src/routing/provider.controller.ts`)
- Provider management API → Model discovery: requests models (evidence: `packages/backend/src/routing/provider.controller.ts`)
- Model discovery → Provider model APIs: fetches catalogs (evidence: `packages/backend/src/model-discovery/model-discovery.service.ts`)
- Subscription sign-in → LLM providers: authorizes subscriptions
- Provider connections → Application database: reads and writes (evidence: `packages/backend/src/routing/routing-core/provider.service.ts`)
- Routing configuration → Application database: stores routes (evidence: `packages/backend/src/routing/routing-core/provider.service.ts`)
- OpenAI-compatible proxy → Provider adapters: forwards requests (evidence: `packages/backend/src/routing/proxy/proxy.controller.ts`)
- Provider adapters → LLM providers: sends model requests
- OpenAI-compatible proxy → Fallback and healing: recovers failures (evidence: `packages/backend/src/routing/proxy/proxy.controller.ts`)
- OpenAI-compatible proxy → Request recording: records exchanges
- Request recording → Application database: persists records
- Analytics queries → Application database: queries messages (evidence: `packages/backend/src/analytics/services/messages-query.service.ts`)
- Usage and notifications → Application database: tracks usage
- Live change events → Web dashboard: notifies changes
