How maneeshchaturvedi/resillm works
resillm is an OpenAI-compatible LLM resilience proxy: an application sends chat requests to the proxy, which routes them to configured providers, applies retries and circuit breakers, and returns a response with usage, cost, and provider metadata. The graph preserves budget controls, metrics, health/admin operations, and configuration hot reload. Evidence is limited to the README and sampled/indexed source; unsampled server wiring and provider capabilities are not inferred.
Main entry points: router.go Model router · server.go HTTP server · handlers.go API handlers · collector.go Prometheus metrics
Components 19
Proxy entry
- Service runtime
cmd/resillm/main.go - HTTP server
internal/server/server.go - API handlers
internal/server/handlers.go
Request routing
- Model router
internal/router/router.go - Provider registry
internal/providers/provider.go - Provider adapter
internal/providers/goopenai.go
Resilience and cost
- Retry policy
internal/resilience/retry.go - Circuit breaker
internal/resilience/circuitbreaker.go - Provider concurrency
internal/providers/semaphore.go - Budget tracker
internal/budget/tracker.go - Cost calculator
internal/pricing/calculator.go
Operations
- Configuration
internal/config/config.go - Config hot reload
internal/config/watcher.go - Prometheus metrics
internal/metrics/collector.go - Rate limiting
internal/server/ratelimit.go - Load shedding
internal/server/loadshed.go
Other components
- Client application
- LLM response
- LLM providers
Connections 23
- Client application → HTTP server: sends API request
README.md - Service runtime → Configuration: loads settings
cmd/resillm/main.go - Service runtime → HTTP server: starts service
cmd/resillm/main.go - HTTP server → API handlers: routes requests
- API handlers → Model router: executes chat
internal/server/handlers.go - Model router → Provider registry: looks up provider
internal/router/router.go - Provider registry → Provider adapter: constructs adapter
internal/providers/provider.go - Provider adapter → LLM providers: sends completion
README.md - Model router → Retry policy: retries requests
internal/router/router.go - Model router → Circuit breaker: checks and records
internal/router/router.go - Model router → Provider concurrency: limits concurrency
internal/router/router.go - Model router → Cost calculator: calculates cost
internal/router/router.go - Model router → Prometheus metrics: records outcomes
internal/router/router.go - Circuit breaker → Prometheus metrics: records circuit state
internal/resilience/circuitbreaker.go - API handlers → Budget tracker: checks and records spend
- API handlers → LLM response: returns API response
- LLM providers → LLM response: provides completion
- Config hot reload → Configuration: reloads settings
internal/config/watcher.go - Config hot reload → HTTP server: applies changes
- HTTP server → Prometheus metrics: records HTTP requests
internal/server/server.go - HTTP server → Rate limiting: rate limits clients
- HTTP server → Load shedding: sheds overload
- API handlers → Model router: queries provider health
internal/server/handlers.go