# kaiser-data/free-llm-proxy-router architecture

> Architecture diagram of the GitHub repository kaiser-data/free-llm-proxy-router, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kaiser-data/free-llm-proxy-router
- Repository: https://github.com/kaiser-data/free-llm-proxy-router

## Overview

This is a Go service that combines free LLM providers behind a local OpenAI-compatible API. The main workflow is: a client sends a request, the proxy selects a strategy and candidate models, then the fallback chain tries providers until one returns a usable answer. A separate scan workflow discovers and updates the model catalog. The graph also shows configuration and persistent reliability/rate-limit state. Some subsystem links come from README documentation; unsampled implementations are represented only where their role is clear.

Main entry points: [`cmd/free-llm-proxy/main.go`](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/cmd/free-llm-proxy/main.go) (Proxy startup), [`cmd/free-llm-scan/main.go`](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/cmd/free-llm-scan/main.go) (Scan commands), [`pkg/proxy/fallback.go`](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/proxy/fallback.go) (Fallback chain), [`pkg/proxy/server.go`](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/proxy/server.go) (HTTP API server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Proxy runtime"]
  node_proxycmd["Proxy startup<br/>[main.go]"]
  node_httpserver["HTTP API server<br/>[server.go]"]
  node_fallback["Fallback chain<br/>[fallback.go]"]
end

subgraph group_discovery["Catalog discovery"]
  node_catalog[("Model catalog<br/>[catalog.go]")]
  node_scancli["Scan commands<br/>[main.go]"]
  node_scanners["Provider scanners"]
  node_probe["Model probing<br/>[prober.go]"]
  node_llmrefresh["LLM catalog refresh<br/>[llm_refresh.go]"]
  node_catalogsync["Catalog sync<br/>[gitsync.go]"]
end

subgraph group_support["Routing support"]
  node_strategies["Routing strategies"]
  node_config["Configuration<br/>[config.go]"]
  node_modelmeta["Model enrichment<br/>[enriched.go]"]
  node_ratelimit[("Rate-limit tracking<br/>[tracker.go]")]
  node_reliability[("Reliability tracking<br/>[tracker.go]")]
end

subgraph group_external["External systems"]
  node_providerapis{{"Provider APIs"}}
  node_refreshllm{{"Refresh LLM provider"}}
  node_catalogfile[("Catalog file")]
end

node_client(("OpenAI client"))

node_client -->|"sends request"| node_httpserver
node_proxycmd -->|"loads settings"| node_config
node_proxycmd -->|"loads catalog"| node_catalog
node_proxycmd -->|"starts server"| node_httpserver
node_proxycmd -->|"builds registry"| node_strategies
node_httpserver -->|"selects strategy"| node_strategies
node_httpserver -->|"runs chain"| node_fallback
node_fallback -->|"calls providers"| node_providerapis
node_fallback -->|"checks cooldowns"| node_ratelimit
node_fallback -->|"updates scores"| node_reliability
node_strategies -->|"ranks models"| node_catalog
node_strategies -->|"uses capabilities"| node_modelmeta
node_scancli -->|"loads settings"| node_config
node_scancli -->|"loads and updates"| node_catalog
node_scancli -->|"runs scans"| node_scanners
node_scanners -->|"discovers models"| node_providerapis
node_scancli -->|"probes pending models"| node_probe
node_scancli -->|"runs refresh"| node_llmrefresh
node_llmrefresh -->|"requests diff"| node_refreshllm
node_proxycmd -->|"starts sync"| node_catalogsync
node_catalogsync -.->|"updates catalog"| node_catalogfile
node_proxycmd -->|"restores state"| node_ratelimit
node_proxycmd -->|"loads state"| node_reliability

click node_proxycmd "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/cmd/free-llm-proxy/main.go"
click node_httpserver "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/proxy/server.go"
click node_fallback "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/proxy/fallback.go"
click node_strategies "https://github.com/kaiser-data/free-llm-proxy-router/tree/main/pkg/strategy"
click node_catalog "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/catalog/catalog.go"
click node_scancli "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/cmd/free-llm-scan/main.go"
click node_scanners "https://github.com/kaiser-data/free-llm-proxy-router/tree/main/pkg/scan"
click node_probe "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/scan/prober.go"
click node_llmrefresh "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/scan/llm_refresh.go"
click node_catalogsync "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/catalog/gitsync.go"
click node_config "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/config/config.go"
click node_modelmeta "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/models/enriched.go"
click node_ratelimit "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/ratelimit/tracker.go"
click node_reliability "https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/reliability/tracker.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_proxycmd,node_httpserver,node_fallback,node_client toneBlue
class node_catalog,node_scancli,node_scanners,node_probe,node_llmrefresh,node_catalogsync toneAmber
class node_strategies,node_config,node_modelmeta,node_ratelimit,node_reliability toneMint
class node_providerapis,node_refreshllm,node_catalogfile toneRose
```

## Components

### Proxy runtime

- [Proxy startup](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/cmd/free-llm-proxy/main.go): `cmd/free-llm-proxy/main.go`
- [HTTP API server](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/proxy/server.go): `pkg/proxy/server.go`
- [Fallback chain](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/proxy/fallback.go): `pkg/proxy/fallback.go`

### Catalog discovery

- [Model catalog](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/catalog/catalog.go): `pkg/catalog/catalog.go`
- [Scan commands](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/cmd/free-llm-scan/main.go): `cmd/free-llm-scan/main.go`
- [Provider scanners](https://github.com/kaiser-data/free-llm-proxy-router/tree/main/pkg/scan): `pkg/scan`
- [Model probing](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/scan/prober.go): `pkg/scan/prober.go`
- [LLM catalog refresh](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/scan/llm_refresh.go): `pkg/scan/llm_refresh.go`
- [Catalog sync](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/catalog/gitsync.go): `pkg/catalog/gitsync.go`

### Routing support

- [Routing strategies](https://github.com/kaiser-data/free-llm-proxy-router/tree/main/pkg/strategy): `pkg/strategy`
- [Configuration](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/config/config.go): `pkg/config/config.go`
- [Model enrichment](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/models/enriched.go): `pkg/models/enriched.go`
- [Rate-limit tracking](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/ratelimit/tracker.go): `pkg/ratelimit/tracker.go`
- [Reliability tracking](https://github.com/kaiser-data/free-llm-proxy-router/blob/main/pkg/reliability/tracker.go): `pkg/reliability/tracker.go`

### External systems

- **Provider APIs**
- **Refresh LLM provider**
- **Catalog file**

### Other components

- **OpenAI client**

## Connections

- OpenAI client → HTTP API server: sends request (evidence: `README.md`)
- Proxy startup → Configuration: loads settings (evidence: `cmd/free-llm-proxy/main.go`)
- Proxy startup → Model catalog: loads catalog (evidence: `cmd/free-llm-proxy/main.go`)
- Proxy startup → HTTP API server: starts server (evidence: `cmd/free-llm-proxy/main.go`)
- Proxy startup → Routing strategies: builds registry (evidence: `cmd/free-llm-proxy/main.go`)
- HTTP API server → Routing strategies: selects strategy (evidence: `pkg/proxy/server.go`)
- HTTP API server → Fallback chain: runs chain (evidence: `README.md`)
- Fallback chain → Provider APIs: calls providers (evidence: `README.md`)
- Fallback chain → Rate-limit tracking: checks cooldowns (evidence: `pkg/proxy/fallback.go`)
- Fallback chain → Reliability tracking: updates scores (evidence: `pkg/proxy/fallback.go`)
- Routing strategies → Model catalog: ranks models (evidence: `pkg/strategy/adaptive.go`)
- Routing strategies → Model enrichment: uses capabilities (evidence: `pkg/strategy/long_context.go`)
- Scan commands → Configuration: loads settings (evidence: `cmd/free-llm-scan/main.go`)
- Scan commands → Model catalog: loads and updates (evidence: `cmd/free-llm-scan/main.go`)
- Scan commands → Provider scanners: runs scans (evidence: `cmd/free-llm-scan/main.go`)
- Provider scanners → Provider APIs: discovers models (evidence: `pkg/scan/openrouter.go`)
- Scan commands → Model probing: probes pending models (evidence: `cmd/free-llm-scan/main.go`)
- Scan commands → LLM catalog refresh: runs refresh (evidence: `cmd/free-llm-scan/main.go`)
- LLM catalog refresh → Refresh LLM provider: requests diff (evidence: `pkg/scan/llm_refresh.go`)
- Proxy startup → Catalog sync: starts sync (evidence: `cmd/free-llm-proxy/main.go`)
- Catalog sync → Catalog file: updates catalog
- Proxy startup → Rate-limit tracking: restores state (evidence: `cmd/free-llm-proxy/main.go`)
- Proxy startup → Reliability tracking: loads state (evidence: `cmd/free-llm-proxy/main.go`)