# maximhq/bifrost architecture

> Architecture diagram of the GitHub repository maximhq/bifrost, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/maximhq/bifrost
- Repository: https://github.com/maximhq/bifrost

## Overview

#### Purpose and Entry Points
Bifrost is a Go-based AI gateway that normalizes inference across many LLM and multimodal providers behind OpenAI-compatible and provider-compatible HTTP APIs. It can also be embedded as a Go SDK.
- HTTP server executable: `transports/bifrost-http/main.go`
- Core SDK façade: `core/bifrost.go`
- Interactive/local CLI: `cli/main.go`
- Browser administration UI is a separate TypeScript application served by the gateway: `ui/app/main.tsx`, `transports/bifrost-http/handlers/ui.go`

#### HTTP Gateway and API Compatibility
The HTTP transport owns server initialization, routing, authentication/middleware, request validation, streaming/websocket handling, inference endpoints, and management APIs. It is the boundary between external clients and the core runtime.
- Server composition and lifecycle: `transports/bifrost-http/server/server.go`
- Canonical inference request handlers: `transports/bifrost-http/handlers/inference.go`
- SDK-compatibility routing for OpenAI, Anthropic, Bedrock, GenAI, LangChain, LiteLLM, and others: `transports/bifrost-http/integrations/router.go`, `transports/bifrost-http/integrations/openai.go`
- Public API contract is maintained under `docs/openapi/openapi.yaml` and `docs/openapi/paths/inference/`

#### Core Inference Runtime
`core` defines shared schemas and orchestrates request execution. It transforms normalized requests into provider-specific API calls, selects configured keys, handles errors/streams, and returns normalized responses. Provider modules implement the external-service boundary rather than exposing provider details to transport callers.
- Primary runtime and shared API: `core/bifrost.go`, `core/schemas/bifrost.go`
- Provider interface/schema layer: `core/schemas/provider.go`
- Representative provider adapters: `core/providers/openai/openai.go`, `core/providers/anthropic/anthropic.go`, `core/providers/bedrock/bedrock.go`
- Streaming and provider HTTP utilities: `core/providers/utils/stream.go`, `core/network/http.go`

#### Plugin Pipeline, Routing, and Governance
Plugins are a first-class interception boundary around inference and MCP operations. The core pipeline loads configured plugins; plugins can implement request policy, routing, caching, logging, telemetry, mocking, or compatibility behavior without embedding those concerns in providers.
- Pipeline and plugin interfaces: `core/mcp/pluginpipeline.go`, `core/schemas/plugin.go`
- Dynamic/native plugin loading: `framework/plugins/loader.go`, `framework/plugins/soloader.go`
- Governance applies virtual-key, budget, rate-limit, complexity, and routing policies: `plugins/governance/main.go`, `plugins/governance/routing.go`
- Built-in semantic caching uses vector stores: `plugins/semanticcache/main.go`, `plugins/semanticcache/search.go`

#### MCP Gateway
Bifrost can manage MCP clients/servers, discover and filter tools, execute tools, maintain sessions, and support agent/code modes. MCP credentials are isolated behind credential-store implementations, including per-user headers and OAuth.
- MCP initialization and client orchestration: `core/mcp/init.go`, `core/mcp/clientmanager.go`
- Tool inventory/execution: `core/mcp/toolmanager.go`, `core/mcp/exec.go`
- Agent and code-mode behavior: `core/mcp/agent.go`, `core/mcp/codemode.go`
- HTTP MCP, sessions, and OAuth surfaces: `transports/bifrost-http/handlers/mcp.go`, `transports/bifrost-http/handlers/mcpoauth2.go`

#### Persistence, Catalogs, and Observability
The optional framework layer abstracts operational state from inference execution. It supports SQLite/Postgres configuration and logs, ClickHouse logs, object storage, model/pricing catalogs, and vector-backed semantic cache. This permits zero-config/in-memory-style operation or durable production deployment.
- Config state, migrations, and secret-related persistence: `framework/configstore/store.go`, `framework/configstore/migrations.go`
- Request/usage log storage: `framework/logstore/store.go`, `framework/logstore/clickhouse.go`
- Model catalog and pricing synchronization: `framework/modelcatalog/main.go`, `framework/modelcatalog/datasheet/sync.go`
- Storage integrations: `framework/vectorstore/store.go`, `framework/objectstore/s3.go`, `framework/objectstore/gcs.go`

#### Operations and Deployment
The primary runtime is Go; the admin UI is TypeScript/React-style frontend code. Deployment targets include Docker, Kubernetes/Helm, and Terraform for AWS, Azure, GCP, and generic Kubernetes. External dependencies commonly include LLM provider APIs, PostgreSQL/SQLite, ClickHouse, Redis, Qdrant, Weaviate, Pinecone, S3/GCS, Prometheus, and OpenTelemetry collectors.
- Container build/runtime: `transports/Dockerfile`, `transports/docker-entrypoint.sh`
- Helm deployment plus optional PostgreSQL/Redis/Qdrant/Weaviate: `helm-charts/bifrost/templates/deployment.yaml`, `helm-charts/bifrost/values.yaml`
- Cloud and Kubernetes provisioning modules: `terraform/modules/bifrost/main.tf`

Main entry points: [`core/bifrost.go`](https://github.com/maximhq/bifrost/blob/dev/core/bifrost.go) (Inference orchestration), [`transports/bifrost-http/server/server.go`](https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/server/server.go) (HTTP server), [`transports/bifrost-http/handlers/inference.go`](https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/handlers/inference.go) (Inference handlers), [`core/mcp/pluginpipeline.go`](https://github.com/maximhq/bifrost/blob/dev/core/mcp/pluginpipeline.go) (Plugin pipeline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_clients["Clients"]
  node_admin_ui["Admin UI<br/>TypeScript frontend<br/>[main.tsx]"]
  node_cli["Local CLI<br/>Go CLI<br/>[main.go]"]
end

subgraph group_gateway["HTTP Gateway"]
  node_http_server["HTTP server<br/>Go transport<br/>[server.go]"]
  node_inference_api["Inference handlers<br/>HTTP handlers<br/>[inference.go]"]
  node_compat_router["Compatibility APIs<br/>integration router<br/>[router.go]"]
  node_openapi["OpenAPI contract<br/>API specification<br/>[openapi.yaml]"]
  node_mcp_http["MCP HTTP &amp; OAuth<br/>HTTP handlers<br/>[mcp.go]"]
end

subgraph group_runtime["Core Runtime"]
  node_sdk["Go SDK façade<br/>SDK API<br/>[bifrost.go]"]
  node_core_runtime["Inference orchestration<br/>core runtime<br/>[bifrost.go]"]
  node_provider_contract["Provider contract<br/>provider interface<br/>[provider.go]"]
  node_provider_adapters{{"Provider adapters<br/>external API adapters<br/>[openai.go]"}}
end

subgraph group_extensions["Plugins &amp; MCP"]
  node_plugin_pipeline["Plugin pipeline<br/>interception pipeline<br/>[pluginpipeline.go]"]
  node_plugin_loader["Plugin loader<br/>dynamic loader<br/>[loader.go]"]
  node_governance["Governance plugin<br/>policy plugin<br/>[main.go]"]
  node_semantic_cache["Semantic cache<br/>cache plugin<br/>[main.go]"]
  node_mcp_gateway["MCP orchestration<br/>MCP runtime<br/>[clientmanager.go]"]
end

subgraph group_state["Operational State"]
  node_config_store[("Config store<br/>state persistence<br/>[store.go]")]
  node_log_store[("Usage log store<br/>observability persistence<br/>[store.go]")]
  node_model_catalog[("Model &amp; pricing catalog<br/>catalog sync<br/>[main.go]")]
  node_vector_store[("Vector store<br/>storage integration<br/>[store.go]")]
end

subgraph group_delivery["Delivery"]
  node_deployment["Container &amp; Helm deployment<br/>deployment assets<br/>[deployment.yaml]"]
end

node_admin_ui -->|"served by"| node_http_server
node_cli -->|"uses"| node_sdk
node_http_server -->|"routes requests"| node_inference_api
node_http_server -->|"routes compatible APIs"| node_compat_router
node_openapi -.->|"defines contract"| node_inference_api
node_compat_router -->|"adapts to canonical API"| node_inference_api
node_inference_api -->|"executes inference"| node_core_runtime
node_sdk -->|"invokes"| node_core_runtime
node_core_runtime -->|"intercepts through"| node_plugin_pipeline
node_plugin_loader -->|"loads plugins into"| node_plugin_pipeline
node_plugin_pipeline -->|"applies policy"| node_governance
node_plugin_pipeline -->|"checks cache"| node_semantic_cache
node_semantic_cache -->|"similarity search"| node_vector_store
node_core_runtime -->|"targets"| node_provider_contract
node_provider_contract -->|"implemented by"| node_provider_adapters
node_inference_api -->|"coordinates MCP"| node_mcp_gateway
node_mcp_http -->|"exposes MCP and OAuth"| node_mcp_gateway
node_http_server -->|"routes MCP APIs"| node_mcp_http
node_core_runtime -.->|"reads runtime configuration"| node_config_store
node_core_runtime -.->|"records usage"| node_log_store
node_core_runtime -.->|"uses model metadata"| node_model_catalog
node_deployment -->|"deploys"| node_http_server

click node_admin_ui "https://github.com/maximhq/bifrost/blob/dev/ui/app/main.tsx"
click node_cli "https://github.com/maximhq/bifrost/blob/dev/cli/main.go"
click node_sdk "https://github.com/maximhq/bifrost/blob/dev/core/bifrost.go"
click node_http_server "https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/server/server.go"
click node_inference_api "https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/handlers/inference.go"
click node_compat_router "https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/integrations/router.go"
click node_openapi "https://github.com/maximhq/bifrost/blob/dev/docs/openapi/openapi.yaml"
click node_core_runtime "https://github.com/maximhq/bifrost/blob/dev/core/bifrost.go"
click node_provider_contract "https://github.com/maximhq/bifrost/blob/dev/core/schemas/provider.go"
click node_provider_adapters "https://github.com/maximhq/bifrost/blob/dev/core/providers/openai/openai.go"
click node_plugin_pipeline "https://github.com/maximhq/bifrost/blob/dev/core/mcp/pluginpipeline.go"
click node_plugin_loader "https://github.com/maximhq/bifrost/blob/dev/framework/plugins/loader.go"
click node_governance "https://github.com/maximhq/bifrost/blob/dev/plugins/governance/main.go"
click node_semantic_cache "https://github.com/maximhq/bifrost/blob/dev/plugins/semanticcache/main.go"
click node_mcp_gateway "https://github.com/maximhq/bifrost/blob/dev/core/mcp/clientmanager.go"
click node_mcp_http "https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/handlers/mcp.go"
click node_config_store "https://github.com/maximhq/bifrost/blob/dev/framework/configstore/store.go"
click node_log_store "https://github.com/maximhq/bifrost/blob/dev/framework/logstore/store.go"
click node_model_catalog "https://github.com/maximhq/bifrost/blob/dev/framework/modelcatalog/main.go"
click node_vector_store "https://github.com/maximhq/bifrost/blob/dev/framework/vectorstore/store.go"
click node_deployment "https://github.com/maximhq/bifrost/blob/dev/helm-charts/bifrost/templates/deployment.yaml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_admin_ui,node_cli toneBlue
class node_http_server,node_inference_api,node_compat_router,node_openapi,node_mcp_http toneAmber
class node_sdk,node_core_runtime,node_provider_contract,node_provider_adapters toneMint
class node_plugin_pipeline,node_plugin_loader,node_governance,node_semantic_cache,node_mcp_gateway toneRose
class node_config_store,node_log_store,node_model_catalog,node_vector_store toneIndigo
class node_deployment toneTeal
```

## Components

### Clients

- [Admin UI](https://github.com/maximhq/bifrost/blob/dev/ui/app/main.tsx): TypeScript frontend · Browser administration application served by the gateway. · `ui/app/main.tsx`
- [Local CLI](https://github.com/maximhq/bifrost/blob/dev/cli/main.go): Go CLI · `cli/main.go`

### HTTP Gateway

- [HTTP server](https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/server/server.go): Go transport · Owns lifecycle, routing, middleware, authentication, and API endpoints. · `transports/bifrost-http/server/server.go`
- [Inference handlers](https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/handlers/inference.go): HTTP handlers · Validates canonical requests and manages streaming or websocket responses. · `transports/bifrost-http/handlers/inference.go`
- [Compatibility APIs](https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/integrations/router.go): integration router · Adapts OpenAI and other provider-compatible request formats. · `transports/bifrost-http/integrations/router.go`
- [OpenAPI contract](https://github.com/maximhq/bifrost/blob/dev/docs/openapi/openapi.yaml): API specification · `docs/openapi/openapi.yaml`
- [MCP HTTP & OAuth](https://github.com/maximhq/bifrost/blob/dev/transports/bifrost-http/handlers/mcp.go): HTTP handlers · `transports/bifrost-http/handlers/mcp.go`

### Core Runtime

- [Go SDK façade](https://github.com/maximhq/bifrost/blob/dev/core/bifrost.go): SDK API · Embeddable programmatic entry point. · `core/bifrost.go`
- [Inference orchestration](https://github.com/maximhq/bifrost/blob/dev/core/bifrost.go): core runtime · Normalizes execution, selects keys, invokes providers, and returns normalized results. · `core/bifrost.go`
- [Provider contract](https://github.com/maximhq/bifrost/blob/dev/core/schemas/provider.go): provider interface · `core/schemas/provider.go`
- [Provider adapters](https://github.com/maximhq/bifrost/blob/dev/core/providers/openai/openai.go): external API adapters · Implements OpenAI, Anthropic, Bedrock, and other provider boundaries. · `core/providers/openai/openai.go`

### Plugins & MCP

- [Plugin pipeline](https://github.com/maximhq/bifrost/blob/dev/core/mcp/pluginpipeline.go): interception pipeline · Runs configured hooks around inference and MCP operations. · `core/mcp/pluginpipeline.go`
- [Plugin loader](https://github.com/maximhq/bifrost/blob/dev/framework/plugins/loader.go): dynamic loader · Loads configured native and dynamically linked plugins. · `framework/plugins/loader.go`
- [Governance plugin](https://github.com/maximhq/bifrost/blob/dev/plugins/governance/main.go): policy plugin · Applies virtual-key, budget, rate-limit, complexity, and routing policy. · `plugins/governance/main.go`
- [Semantic cache](https://github.com/maximhq/bifrost/blob/dev/plugins/semanticcache/main.go): cache plugin · Uses vector similarity to serve cached inference results. · `plugins/semanticcache/main.go`
- [MCP orchestration](https://github.com/maximhq/bifrost/blob/dev/core/mcp/clientmanager.go): MCP runtime · Manages clients, tools, sessions, agent/code modes, and credential isolation. · `core/mcp/clientmanager.go`

### Operational State

- [Config store](https://github.com/maximhq/bifrost/blob/dev/framework/configstore/store.go): state persistence · Stores configuration, migrations, and secret-related state in SQLite or Postgres. · `framework/configstore/store.go`
- [Usage log store](https://github.com/maximhq/bifrost/blob/dev/framework/logstore/store.go): observability persistence · Persists request and usage logs, including ClickHouse support. · `framework/logstore/store.go`
- [Model & pricing catalog](https://github.com/maximhq/bifrost/blob/dev/framework/modelcatalog/main.go): catalog sync · `framework/modelcatalog/main.go`
- [Vector store](https://github.com/maximhq/bifrost/blob/dev/framework/vectorstore/store.go): storage integration · Backs semantic cache search. · `framework/vectorstore/store.go`

### Delivery

- [Container & Helm deployment](https://github.com/maximhq/bifrost/blob/dev/helm-charts/bifrost/templates/deployment.yaml): deployment assets · Packages the gateway for Docker and Kubernetes. · `helm-charts/bifrost/templates/deployment.yaml`

## Connections

- Admin UI → HTTP server: served by
- Local CLI → Go SDK façade: uses
- HTTP server → Inference handlers: routes requests
- HTTP server → Compatibility APIs: routes compatible APIs
- OpenAPI contract → Inference handlers: defines contract
- Compatibility APIs → Inference handlers: adapts to canonical API
- Inference handlers → Inference orchestration: executes inference
- Go SDK façade → Inference orchestration: invokes
- Inference orchestration → Plugin pipeline: intercepts through
- Plugin loader → Plugin pipeline: loads plugins into
- Plugin pipeline → Governance plugin: applies policy
- Plugin pipeline → Semantic cache: checks cache
- Semantic cache → Vector store: similarity search
- Inference orchestration → Provider contract: targets
- Provider contract → Provider adapters: implemented by
- Inference handlers → MCP orchestration: coordinates MCP
- MCP HTTP & OAuth → MCP orchestration: exposes MCP and OAuth
- HTTP server → MCP HTTP & OAuth: routes MCP APIs
- Inference orchestration → Config store: reads runtime configuration
- Inference orchestration → Usage log store: records usage
- Inference orchestration → Model & pricing catalog: uses model metadata
- Container & Helm deployment → HTTP server: deploys
