How kaiser-data/free-llm-proxy-router works
This is a Go service that combines free LLM providers behind a local OpenAI-compatible API. The main workflow is: a client sends a request, the proxy selects a strategy and candidate models, then the fallback chain tries providers until one returns a usable answer. A separate scan workflow discovers and updates the model catalog. The graph also shows configuration and persistent reliability/rate-limit state. Some subsystem links come from README documentation; unsampled implementations are represented only where their role is clear.
Main entry points: free-llm-proxy/main.go Proxy startup · free-llm-scan/main.go Scan commands · fallback.go Fallback chain · server.go HTTP API server
Components 18
Proxy runtime
- Proxy startup
cmd/free-llm-proxy/main.go - HTTP API server
pkg/proxy/server.go - Fallback chain
pkg/proxy/fallback.go
Catalog discovery
- Model catalog
pkg/catalog/catalog.go - Scan commands
cmd/free-llm-scan/main.go - Provider scanners
pkg/scan - Model probing
pkg/scan/prober.go - LLM catalog refresh
pkg/scan/llm_refresh.go - Catalog sync
pkg/catalog/gitsync.go
Routing support
- Routing strategies
pkg/strategy - Configuration
pkg/config/config.go - Model enrichment
pkg/models/enriched.go - Rate-limit tracking
pkg/ratelimit/tracker.go - Reliability tracking
pkg/reliability/tracker.go
External systems
- Provider APIs
- Refresh LLM provider
- Catalog file
Other components
- OpenAI client
Connections 23
- OpenAI client → HTTP API server: sends request
README.md - Proxy startup → Configuration: loads settings
cmd/free-llm-proxy/main.go - Proxy startup → Model catalog: loads catalog
cmd/free-llm-proxy/main.go - Proxy startup → HTTP API server: starts server
cmd/free-llm-proxy/main.go - Proxy startup → Routing strategies: builds registry
cmd/free-llm-proxy/main.go - HTTP API server → Routing strategies: selects strategy
pkg/proxy/server.go - HTTP API server → Fallback chain: runs chain
README.md - Fallback chain → Provider APIs: calls providers
README.md - Fallback chain → Rate-limit tracking: checks cooldowns
pkg/proxy/fallback.go - Fallback chain → Reliability tracking: updates scores
pkg/proxy/fallback.go - Routing strategies → Model catalog: ranks models
pkg/strategy/adaptive.go - Routing strategies → Model enrichment: uses capabilities
pkg/strategy/long_context.go - Scan commands → Configuration: loads settings
cmd/free-llm-scan/main.go - Scan commands → Model catalog: loads and updates
cmd/free-llm-scan/main.go - Scan commands → Provider scanners: runs scans
cmd/free-llm-scan/main.go - Provider scanners → Provider APIs: discovers models
pkg/scan/openrouter.go - Scan commands → Model probing: probes pending models
cmd/free-llm-scan/main.go - Scan commands → LLM catalog refresh: runs refresh
cmd/free-llm-scan/main.go - LLM catalog refresh → Refresh LLM provider: requests diff
pkg/scan/llm_refresh.go - Proxy startup → Catalog sync: starts sync
cmd/free-llm-proxy/main.go - Catalog sync → Catalog file: updates catalog
- Proxy startup → Rate-limit tracking: restores state
cmd/free-llm-proxy/main.go - Proxy startup → Reliability tracking: loads state
cmd/free-llm-proxy/main.go