How s1line1/home-llm-gateway works
home-llm-gateway routes OpenAI-compatible requests from clients through a cloud gateway to models hosted on edge machines. The main flow is HTTP admission and API-key authorization, model-aware agent selection, QUIC/mTLS frame transport, and HTTP/SSE response relay; agents maintain the outbound tunnel and call their local LLM. The gateway also provides key/agent administration, usage and metrics, and a hosted React dashboard. Edges use README statements and shown routing/HTTP code; unsampled subsystem links are omitted where their callers are not evidenced.
Main entry points: proxy/mod.rs Request Proxy · http/mod.rs HTTP Router · auth.rs Key Authorization · client.ts Gateway API Client
Components 26
Gateway HTTP
- HTTP Router
crates/gateway/src/http/mod.rs - Key Authorization
crates/gateway/src/auth.rs - Request Proxy
crates/gateway/src/proxy/mod.rs - OpenAI Semantics
crates/gateway/src/openai.rs - Admin API
crates/gateway/src/admin.rs - Health And Metrics
crates/gateway/src/http/observability.rs
Edge Tunnel
- QUIC Tunnel
crates/gateway/src/proxy/tunnel.rs - Edge Agent
crates/agent/src/lib.rs - Agent Configuration
crates/agent/src/config.rs
Routing And State
- Gateway State
crates/gateway/src/state.rs - Agent Registry
crates/gateway/src/registry.rs - Model Routing
crates/gateway/src/proxy/routing.rs - Key Rate Limits
crates/gateway/src/ratelimit.rs - Key And Usage Store
crates/gateway/src/storage/mod.rs - Key Hashing
crates/gateway/src/storage/hash.rs - Usage Meter
crates/gateway/src/usage_meter.rs
Admin Dashboard
- Dashboard App
web/src/main.tsx - Gateway API Client
web/src/api/client.ts - Admin Token
web/src/hooks/useAdminToken.ts - Metrics History
web/src/hooks/useMetricsHistory.tsx - Metrics Parser
web/src/api/metrics.ts - Query Configuration
web/src/api/queryClient.ts
Tunnel Protocol
- Tunnel Frames
crates/proto/src/frame.rs - Frame IO
crates/proto/src/io.rs
Other components
- API Client
- Local LLM
Connections 32
- API Client → HTTP Router: sends HTTPS requests
- HTTP Router → Key Authorization: authorizes API calls
crates/gateway/src/http/mod.rs - HTTP Router → Request Proxy: routes API requests
crates/gateway/src/http/mod.rs - HTTP Router → Admin API: routes admin calls
crates/gateway/src/http/mod.rs - HTTP Router → Health And Metrics: routes probes
crates/gateway/src/http/mod.rs - Request Proxy → Key Authorization: authenticates requests
crates/gateway/src/proxy/mod.rs - Request Proxy → OpenAI Semantics: parses request facts
crates/gateway/src/proxy/mod.rs - Request Proxy → Model Routing: selects edge
crates/gateway/src/proxy/routing.rs - Model Routing → Agent Registry: acquires agent
crates/gateway/src/proxy/routing.rs - Model Routing → QUIC Tunnel: opens tunnel stream
crates/gateway/src/proxy/routing.rs - Request Proxy → QUIC Tunnel: forwards framed request
crates/gateway/src/proxy/mod.rs - QUIC Tunnel → Tunnel Frames: exchanges frames
- Frame IO → Tunnel Frames: encodes and decodes
crates/proto/src/io.rs - Edge Agent → QUIC Tunnel: connects outbound
- Edge Agent → Local LLM: forwards HTTP requests
- Agent Configuration → Edge Agent: configures agent
crates/agent/src/lib.rs - Admin API → Key And Usage Store: manages API keys
crates/gateway/src/admin.rs - Key Authorization → Key And Usage Store: verifies keys
crates/gateway/src/auth.rs - Key And Usage Store → Key Hashing: hashes credentials
crates/gateway/src/storage/mod.rs - Key Authorization → Key Rate Limits: checks key limits
crates/gateway/src/auth.rs - Gateway State → Agent Registry: shares registry
crates/gateway/src/state.rs - Gateway State → Key And Usage Store: shares key store
crates/gateway/src/state.rs - Request Proxy → Usage Meter: collects usage
crates/gateway/src/proxy/mod.rs - Dashboard App → Gateway API Client: loads gateway data
web/src/main.tsx - Dashboard App → Metrics History: provides metrics history
web/src/main.tsx - Metrics History → Gateway API Client: fetches metrics text
web/src/hooks/useMetricsHistory.tsx - Metrics History → Metrics Parser: parses samples
web/src/hooks/useMetricsHistory.tsx - Gateway API Client → HTTP Router: calls gateway endpoints
- Dashboard App → Admin Token: guards admin pages
- Query Configuration → Admin Token: clears token on 401
web/src/api/queryClient.ts - Gateway API Client → Admin Token: uses bearer token
web/src/hooks/useAdminToken.ts - Health And Metrics → Gateway State: reads shared state
crates/gateway/src/http/observability.rs