How mnfst/llm-gateway works
Manifest is an OpenAI-compatible LLM gateway: users connect provider credentials or subscriptions, configure model routing, then send requests through one endpoint to a selected model. The gateway supports fallback and self-healing, records request bodies, and exposes analytics and usage controls. Its dashboard and CLI manage gateway configuration; MCP offers a separate agent-facing interface. The README documents these capabilities, while sampled backend excerpts confirm provider discovery, routing configuration, proxy response handling, and analytics queries. Unsampled subsystem internals are represented at their existing directory paths.
Main entry points: entities Application database · provider.controller.ts Provider management API · proxy.controller.ts OpenAI-compatible proxy · Routing.tsx Web dashboard
Components 19
User access
- Web dashboard
packages/frontend/src/pages/Routing.tsx - CLI configuration
packages/cli/src/commands/routing.ts - MCP interface
packages/backend/src/mcp/mcp.controller.ts - Authentication
packages/backend/src/auth
Provider and routing
- Provider management API
packages/backend/src/routing/provider.controller.ts - Provider connections
packages/backend/src/routing/routing-core/provider.service.ts - Model discovery
packages/backend/src/model-discovery/model-discovery.service.ts - Subscription sign-in
packages/backend/src/routing/oauth - Routing configuration
packages/backend/src/routing
Gateway runtime
- OpenAI-compatible proxy
packages/backend/src/routing/proxy/proxy.controller.ts - Provider adapters
packages/backend/src/routing/proxy/provider-client.ts - Fallback and healing
packages/backend/src/routing/autofix - Request recording
packages/backend/src/common/services/request-recording-storage.service.ts
Observability and control
- Application database
packages/backend/src/entities - Analytics queries
packages/backend/src/analytics/services/messages-query.service.ts - Usage and notifications
packages/backend/src/billing - Live change events
packages/backend/src/common/services/ingest-event-bus.service.ts
Other components
- Provider model APIs
- LLM providers
Connections 19
- Web dashboard → Provider management API: configures providers
- CLI configuration → Provider management API: configures routes
- Web dashboard → Live change events: receives updates
- MCP interface → Authentication: authenticates
packages/backend/src/mcp/mcp.controller.ts - MCP interface → Application database: queries tenant data
packages/backend/src/mcp/mcp.controller.ts - Provider management API → Provider connections: manages connections
packages/backend/src/routing/provider.controller.ts - Provider management API → Model discovery: requests models
packages/backend/src/routing/provider.controller.ts - Model discovery → Provider model APIs: fetches catalogs
packages/backend/src/model-discovery/model-discovery.service.ts - Subscription sign-in → LLM providers: authorizes subscriptions
- Provider connections → Application database: reads and writes
packages/backend/src/routing/routing-core/provider.service.ts - Routing configuration → Application database: stores routes
packages/backend/src/routing/routing-core/provider.service.ts - OpenAI-compatible proxy → Provider adapters: forwards requests
packages/backend/src/routing/proxy/proxy.controller.ts - Provider adapters → LLM providers: sends model requests
- OpenAI-compatible proxy → Fallback and healing: recovers failures
packages/backend/src/routing/proxy/proxy.controller.ts - OpenAI-compatible proxy → Request recording: records exchanges
- Request recording → Application database: persists records
- Analytics queries → Application database: queries messages
packages/backend/src/analytics/services/messages-query.service.ts - Usage and notifications → Application database: tracks usage
- Live change events → Web dashboard: notifies changes