# devansh-365/freellm architecture

> Architecture diagram of the GitHub repository devansh-365/freellm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/devansh-365/freellm
- Repository: https://github.com/devansh-365/freellm

## Overview

FreeLLM is an OpenAI-compatible gateway that accepts chat requests, chooses a provider/model strategy, fails over when needed, and returns normalized responses. Its primary workflow passes through authentication and chat routing into provider adapters; streaming requests use a separate normalization path. The README also documents caching, privacy and strict routing, usage/status reporting, scoped keys, browser tokens, and a dashboard. Some of these subsystems are unsampled, so their connections are shown only where the README or excerpts establish them.

Main entry points: [`apps/gateway/src/routing/router.ts`](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/router.ts) (Model selection and failover), [`apps/gateway/src/app.ts`](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/app.ts) (HTTP application), [`apps/gateway/src/routes/v1/chat.ts`](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routes/v1/chat.ts) (Chat endpoint), [`apps/gateway/src/middleware/auth.ts`](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/middleware/auth.ts) (Authentication)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Client experience"]
  node_dashboard["Dashboard UI<br/>[App.tsx]"]
  node_dashboard_api["Dashboard API hooks<br/>[hooks.ts]"]
end

subgraph group_gateway["Gateway API"]
  node_gateway_app["HTTP application<br/>[app.ts]"]
  node_chat["Chat endpoint<br/>[chat.ts]"]
  node_model_api["Models endpoint<br/>[models.ts]"]
  node_status_api["Status endpoint<br/>[status.ts]"]
  node_token_api["Token issuance<br/>[tokens.ts]"]
  node_json_mode["JSON mode<br/>[json-mode.ts]"]
  node_stream_pipeline["Stream normalization<br/>[pipeline.ts]"]
  node_errors["Error responses<br/>[error-handler.ts]"]
end

subgraph group_routing["Routing and providers"]
  node_router["Model selection and failover<br/>[router.ts]"]
  node_registry["Provider registry<br/>[registry.ts]"]
  node_cache[("Response cache<br/>[cache.ts]")]
  node_privacy["Privacy routing<br/>[privacy.ts]"]
  node_strict["Strict mode<br/>[strict.ts]"]
  node_provider_adapters["Provider adapters<br/>[base.ts]"]
end

subgraph group_operations["Access and observability"]
  node_auth["Authentication<br/>[auth.ts]"]
  node_virtual_keys[("Virtual keys<br/>[virtual-keys.ts]")]
  node_browser_tokens["Browser tokens<br/>[browser-tokens.ts]"]
  node_identifier_limits["Per-user limits"]
  node_metrics[("Usage and request logs<br/>[usage-tracker.ts]")]
end

node_caller(("SDK / app"))
node_provider_services{{"LLM providers"}}

node_caller -->|"sends API requests"| node_gateway_app
node_dashboard -->|"queries gateway data"| node_dashboard_api
node_dashboard_api -.->|"calls API"| node_gateway_app
node_gateway_app -->|"authenticates requests"| node_auth
node_gateway_app -->|"limits identifiers"| node_identifier_limits
node_gateway_app -->|"mounts routes"| node_chat
node_gateway_app -->|"handles errors"| node_errors
node_auth -->|"checks credentials"| node_virtual_keys
node_auth -->|"verifies tokens"| node_browser_tokens
node_chat -->|"routes completion"| node_router
node_chat -->|"translates JSON mode"| node_json_mode
node_chat -->|"normalizes streams"| node_stream_pipeline
node_chat -->|"checks and records caps"| node_virtual_keys
node_router -->|"selects providers"| node_registry
node_router -->|"reads and writes cache"| node_cache
node_router -->|"filters providers"| node_privacy
node_router -->|"enforces strict mode"| node_strict
node_router -->|"dispatches completion"| node_provider_adapters
node_provider_adapters -->|"sends model requests"| node_provider_services
node_router -->|"records outcomes"| node_metrics
node_model_api -.->|"reports available models"| node_registry
node_status_api -.->|"reports usage and health"| node_metrics
node_token_api -.->|"issues tokens"| node_browser_tokens
node_errors -->|"builds retry advice"| node_router

click node_dashboard "https://github.com/devansh-365/freellm/blob/main/apps/dashboard/src/App.tsx"
click node_dashboard_api "https://github.com/devansh-365/freellm/blob/main/apps/dashboard/src/api/hooks.ts"
click node_gateway_app "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/app.ts"
click node_chat "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routes/v1/chat.ts"
click node_model_api "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routes/v1/models.ts"
click node_status_api "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routes/v1/status.ts"
click node_token_api "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routes/v1/tokens.ts"
click node_auth "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/middleware/auth.ts"
click node_virtual_keys "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/features/virtual-keys.ts"
click node_browser_tokens "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/features/browser-tokens.ts"
click node_identifier_limits "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/features/identifier-limiter.ts"
click node_router "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/router.ts"
click node_registry "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/registry.ts"
click node_cache "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/cache.ts"
click node_privacy "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/privacy.ts"
click node_strict "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/strict.ts"
click node_provider_adapters "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/providers/base.ts"
click node_json_mode "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/features/json-mode.ts"
click node_stream_pipeline "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/streaming/pipeline.ts"
click node_metrics "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/observability/usage-tracker.ts"
click node_errors "https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/middleware/error-handler.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard,node_dashboard_api toneBlue
class node_gateway_app,node_chat,node_model_api,node_status_api,node_token_api,node_json_mode,node_stream_pipeline,node_errors toneAmber
class node_router,node_registry,node_cache,node_privacy,node_strict,node_provider_adapters toneMint
class node_auth,node_virtual_keys,node_browser_tokens,node_identifier_limits,node_metrics toneRose
class node_caller,node_provider_services toneIndigo
```

## Components

### Client experience

- [Dashboard UI](https://github.com/devansh-365/freellm/blob/main/apps/dashboard/src/App.tsx): `apps/dashboard/src/App.tsx`
- [Dashboard API hooks](https://github.com/devansh-365/freellm/blob/main/apps/dashboard/src/api/hooks.ts): `apps/dashboard/src/api/hooks.ts`

### Gateway API

- [HTTP application](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/app.ts): `apps/gateway/src/app.ts`
- [Chat endpoint](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routes/v1/chat.ts): `apps/gateway/src/routes/v1/chat.ts`
- [Models endpoint](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routes/v1/models.ts): `apps/gateway/src/routes/v1/models.ts`
- [Status endpoint](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routes/v1/status.ts): `apps/gateway/src/routes/v1/status.ts`
- [Token issuance](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routes/v1/tokens.ts): `apps/gateway/src/routes/v1/tokens.ts`
- [JSON mode](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/features/json-mode.ts): `apps/gateway/src/features/json-mode.ts`
- [Stream normalization](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/streaming/pipeline.ts): `apps/gateway/src/streaming/pipeline.ts`
- [Error responses](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/middleware/error-handler.ts): `apps/gateway/src/middleware/error-handler.ts`

### Routing and providers

- [Model selection and failover](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/router.ts): `apps/gateway/src/routing/router.ts`
- [Provider registry](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/registry.ts): `apps/gateway/src/routing/registry.ts`
- [Response cache](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/cache.ts): `apps/gateway/src/routing/cache.ts`
- [Privacy routing](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/privacy.ts): `apps/gateway/src/routing/privacy.ts`
- [Strict mode](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/routing/strict.ts): `apps/gateway/src/routing/strict.ts`
- [Provider adapters](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/providers/base.ts): `apps/gateway/src/providers/base.ts`

### Access and observability

- [Authentication](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/middleware/auth.ts): `apps/gateway/src/middleware/auth.ts`
- [Virtual keys](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/features/virtual-keys.ts): `apps/gateway/src/features/virtual-keys.ts`
- [Browser tokens](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/features/browser-tokens.ts): `apps/gateway/src/features/browser-tokens.ts`
- [Per-user limits](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/features/identifier-limiter.ts): `apps/gateway/src/features/identifier-limiter.ts`
- [Usage and request logs](https://github.com/devansh-365/freellm/blob/main/apps/gateway/src/observability/usage-tracker.ts): `apps/gateway/src/observability/usage-tracker.ts`

### Other components

- **SDK / app**
- **LLM providers**

## Connections

- SDK / app → HTTP application: sends API requests (evidence: `README.md`)
- Dashboard UI → Dashboard API hooks: queries gateway data (evidence: `apps/dashboard/src/api/hooks.ts`)
- Dashboard API hooks → HTTP application: calls API
- HTTP application → Authentication: authenticates requests (evidence: `apps/gateway/src/app.ts`)
- HTTP application → Per-user limits: limits identifiers (evidence: `apps/gateway/src/app.ts`)
- HTTP application → Chat endpoint: mounts routes (evidence: `apps/gateway/src/app.ts`)
- HTTP application → Error responses: handles errors (evidence: `apps/gateway/src/app.ts`)
- Authentication → Virtual keys: checks credentials (evidence: `apps/gateway/src/middleware/auth.ts`)
- Authentication → Browser tokens: verifies tokens (evidence: `apps/gateway/src/middleware/auth.ts`)
- Chat endpoint → Model selection and failover: routes completion (evidence: `apps/gateway/src/routes/v1/chat.ts`)
- Chat endpoint → JSON mode: translates JSON mode (evidence: `apps/gateway/src/routes/v1/chat.ts`)
- Chat endpoint → Stream normalization: normalizes streams (evidence: `apps/gateway/src/routes/v1/chat.ts`)
- Chat endpoint → Virtual keys: checks and records caps (evidence: `apps/gateway/src/routes/v1/chat.ts`)
- Model selection and failover → Provider registry: selects providers (evidence: `apps/gateway/src/routing/index.ts`)
- Model selection and failover → Response cache: reads and writes cache (evidence: `apps/gateway/src/routing/router.ts`)
- Model selection and failover → Privacy routing: filters providers (evidence: `apps/gateway/src/routing/router.ts`)
- Model selection and failover → Strict mode: enforces strict mode (evidence: `apps/gateway/src/routing/router.ts`)
- Model selection and failover → Provider adapters: dispatches completion (evidence: `apps/gateway/src/routing/router.ts`)
- Provider adapters → LLM providers: sends model requests (evidence: `README.md`)
- Model selection and failover → Usage and request logs: records outcomes (evidence: `apps/gateway/src/routing/router.ts`)
- Models endpoint → Provider registry: reports available models
- Status endpoint → Usage and request logs: reports usage and health
- Token issuance → Browser tokens: issues tokens
- Error responses → Model selection and failover: builds retry advice (evidence: `apps/gateway/src/middleware/error-handler.ts`)
