# maneeshchaturvedi/resillm architecture

> Architecture diagram of the GitHub repository maneeshchaturvedi/resillm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/maneeshchaturvedi/resillm
- Repository: https://github.com/maneeshchaturvedi/resillm

## Overview

resillm is an OpenAI-compatible LLM resilience proxy: an application sends chat requests to the proxy, which routes them to configured providers, applies retries and circuit breakers, and returns a response with usage, cost, and provider metadata. The graph preserves budget controls, metrics, health/admin operations, and configuration hot reload. Evidence is limited to the README and sampled/indexed source; unsampled server wiring and provider capabilities are not inferred.

Main entry points: [`internal/router/router.go`](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/router/router.go) (Model router), [`internal/server/server.go`](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/server.go) (HTTP server), [`internal/server/handlers.go`](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/handlers.go) (API handlers), [`internal/metrics/collector.go`](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/metrics/collector.go) (Prometheus metrics)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Proxy entry"]
  node_runtime["Service runtime<br/>[main.go]"]
  node_server["HTTP server<br/>[server.go]"]
  node_handlers["API handlers<br/>[handlers.go]"]
end

subgraph group_routing["Request routing"]
  node_router["Model router<br/>[router.go]"]
  node_registry["Provider registry<br/>[provider.go]"]
  node_provider["Provider adapter<br/>[goopenai.go]"]
end

subgraph group_controls["Resilience and cost"]
  node_retry["Retry policy<br/>[retry.go]"]
  node_breaker["Circuit breaker<br/>[circuitbreaker.go]"]
  node_semaphore["Provider concurrency<br/>[semaphore.go]"]
  node_budget["Budget tracker<br/>[tracker.go]"]
  node_pricing["Cost calculator<br/>[calculator.go]"]
end

subgraph group_operations["Operations"]
  node_configuration["Configuration<br/>[config.go]"]
  node_watcher["Config hot reload<br/>[watcher.go]"]
  node_metrics["Prometheus metrics<br/>[collector.go]"]
  node_rate["Rate limiting<br/>[ratelimit.go]"]
  node_loadshed["Load shedding<br/>[loadshed.go]"]
end

node_app(("Client application"))
node_response["LLM response"]
node_llm(("LLM providers"))

node_app -->|"sends API request"| node_server
node_runtime -->|"loads settings"| node_configuration
node_runtime -->|"starts service"| node_server
node_server -.->|"routes requests"| node_handlers
node_handlers -->|"executes chat"| node_router
node_router -->|"looks up provider"| node_registry
node_registry -->|"constructs adapter"| node_provider
node_provider -->|"sends completion"| node_llm
node_router -->|"retries requests"| node_retry
node_router -->|"checks and records"| node_breaker
node_router -->|"limits concurrency"| node_semaphore
node_router -->|"calculates cost"| node_pricing
node_router -->|"records outcomes"| node_metrics
node_breaker -->|"records circuit state"| node_metrics
node_handlers -.->|"checks and records spend"| node_budget
node_handlers -.->|"returns API response"| node_response
node_llm -.->|"provides completion"| node_response
node_watcher -->|"reloads settings"| node_configuration
node_watcher -.->|"applies changes"| node_server
node_server -->|"records HTTP requests"| node_metrics
node_server -.->|"rate limits clients"| node_rate
node_server -.->|"sheds overload"| node_loadshed
node_handlers -->|"queries provider health"| node_router

click node_runtime "https://github.com/maneeshchaturvedi/resillm/blob/main/cmd/resillm/main.go"
click node_server "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/server.go"
click node_handlers "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/handlers.go"
click node_router "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/router/router.go"
click node_registry "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/providers/provider.go"
click node_provider "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/providers/goopenai.go"
click node_retry "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/resilience/retry.go"
click node_breaker "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/resilience/circuitbreaker.go"
click node_semaphore "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/providers/semaphore.go"
click node_budget "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/budget/tracker.go"
click node_pricing "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/pricing/calculator.go"
click node_configuration "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/config/config.go"
click node_watcher "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/config/watcher.go"
click node_metrics "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/metrics/collector.go"
click node_rate "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/ratelimit.go"
click node_loadshed "https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/loadshed.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_runtime,node_server,node_handlers,node_app toneBlue
class node_router,node_registry,node_provider toneAmber
class node_retry,node_breaker,node_semaphore,node_budget,node_pricing toneMint
class node_configuration,node_watcher,node_metrics,node_rate,node_loadshed toneRose
class node_response,node_llm toneIndigo
```

## Components

### Proxy entry

- [Service runtime](https://github.com/maneeshchaturvedi/resillm/blob/main/cmd/resillm/main.go): `cmd/resillm/main.go`
- [HTTP server](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/server.go): `internal/server/server.go`
- [API handlers](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/handlers.go): `internal/server/handlers.go`

### Request routing

- [Model router](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/router/router.go): `internal/router/router.go`
- [Provider registry](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/providers/provider.go): `internal/providers/provider.go`
- [Provider adapter](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/providers/goopenai.go): `internal/providers/goopenai.go`

### Resilience and cost

- [Retry policy](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/resilience/retry.go): `internal/resilience/retry.go`
- [Circuit breaker](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/resilience/circuitbreaker.go): `internal/resilience/circuitbreaker.go`
- [Provider concurrency](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/providers/semaphore.go): `internal/providers/semaphore.go`
- [Budget tracker](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/budget/tracker.go): `internal/budget/tracker.go`
- [Cost calculator](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/pricing/calculator.go): `internal/pricing/calculator.go`

### Operations

- [Configuration](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/config/config.go): `internal/config/config.go`
- [Config hot reload](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/config/watcher.go): `internal/config/watcher.go`
- [Prometheus metrics](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/metrics/collector.go): `internal/metrics/collector.go`
- [Rate limiting](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/ratelimit.go): `internal/server/ratelimit.go`
- [Load shedding](https://github.com/maneeshchaturvedi/resillm/blob/main/internal/server/loadshed.go): `internal/server/loadshed.go`

### Other components

- **Client application**
- **LLM response**
- **LLM providers**

## Connections

- Client application → HTTP server: sends API request (evidence: `README.md`)
- Service runtime → Configuration: loads settings (evidence: `cmd/resillm/main.go`)
- Service runtime → HTTP server: starts service (evidence: `cmd/resillm/main.go`)
- HTTP server → API handlers: routes requests
- API handlers → Model router: executes chat (evidence: `internal/server/handlers.go`)
- Model router → Provider registry: looks up provider (evidence: `internal/router/router.go`)
- Provider registry → Provider adapter: constructs adapter (evidence: `internal/providers/provider.go`)
- Provider adapter → LLM providers: sends completion (evidence: `README.md`)
- Model router → Retry policy: retries requests (evidence: `internal/router/router.go`)
- Model router → Circuit breaker: checks and records (evidence: `internal/router/router.go`)
- Model router → Provider concurrency: limits concurrency (evidence: `internal/router/router.go`)
- Model router → Cost calculator: calculates cost (evidence: `internal/router/router.go`)
- Model router → Prometheus metrics: records outcomes (evidence: `internal/router/router.go`)
- Circuit breaker → Prometheus metrics: records circuit state (evidence: `internal/resilience/circuitbreaker.go`)
- API handlers → Budget tracker: checks and records spend
- API handlers → LLM response: returns API response
- LLM providers → LLM response: provides completion
- Config hot reload → Configuration: reloads settings (evidence: `internal/config/watcher.go`)
- Config hot reload → HTTP server: applies changes
- HTTP server → Prometheus metrics: records HTTP requests (evidence: `internal/server/server.go`)
- HTTP server → Rate limiting: rate limits clients
- HTTP server → Load shedding: sheds overload
- API handlers → Model router: queries provider health (evidence: `internal/server/handlers.go`)
