# anishakalivarapu/llm-router architecture

> Architecture diagram of the GitHub repository anishakalivarapu/llm-router, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anishakalivarapu/llm-router
- Repository: https://github.com/anishakalivarapu/llm-router

## Overview

Smart LLM Router is a FastAPI service that classifies query complexity, selects a model according to cost/latency/quality preference, and executes an escalating OpenAI/Gemini model chain. The main path is API request → router → classifier and model selection → executor → provider → response, with metrics recorded for successful routes. The README also documents classify, model-list, health, and metrics endpoints. The supplied excerpts show the route, classifier, executor, and provider wiring; some endpoint and OpenAI implementation details are not excerpted, so those links are omitted where unsupported.

Main entry points: [`app/router.py`](https://github.com/anishakalivarapu/llm-router/blob/main/app/router.py) (Query router), [`app/main.py`](https://github.com/anishakalivarapu/llm-router/blob/main/app/main.py) (App and dependency setup), [`app/executor.py`](https://github.com/anishakalivarapu/llm-router/blob/main/app/executor.py) (Retries and escalation), [`app/providers/registry.py`](https://github.com/anishakalivarapu/llm-router/blob/main/app/providers/registry.py) (Provider registry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["HTTP API"]
  node_api["FastAPI endpoints<br/>[api.py]"]
  node_schemas["Request and response schemas<br/>[schemas.py]"]
  node_bootstrap["App and dependency setup<br/>[main.py]"]
end

subgraph group_routing["Classification and Routing"]
  node_router["Query router<br/>[router.py]"]
  node_heuristic["Heuristic classifier<br/>[heuristic.py]"]
  node_hybrid["Hybrid classifier<br/>[hybrid.py]"]
  node_llmclassifier["LLM classifier<br/>[llm.py]"]
end

subgraph group_execution["Model Execution"]
  node_executor["Retries and escalation<br/>[executor.py]"]
  node_breaker["Circuit breaker<br/>[executor.py]"]
end

subgraph group_providers["Provider Adapters"]
  node_providerbase["Provider protocol and HTTP<br/>[base.py]"]
  node_registry["Provider registry<br/>[registry.py]"]
  node_openai["OpenAI adapter<br/>[openai_provider.py]"]
  node_gemini["Gemini adapter<br/>[gemini_provider.py]"]
end

subgraph group_operations["Configuration and Metrics"]
  node_config["Settings and model ladder<br/>[config.py]"]
  node_metrics[("In-memory metrics<br/>[metrics.py]")]
end

node_caller(("API client"))
node_openaiservice{{"OpenAI API"}}
node_geminiservice{{"Gemini API"}}
node_client["HTTPX client"]

node_caller -->|"sends requests"| node_api
node_api -->|"validates and returns"| node_schemas
node_api -->|"dispatches routes"| node_router
node_bootstrap -->|"registers endpoints"| node_api
node_bootstrap -->|"builds router"| node_router
node_bootstrap -->|"builds executor"| node_executor
node_bootstrap -->|"creates metrics"| node_metrics
node_bootstrap -->|"builds providers"| node_registry
node_router -->|"uses fallback"| node_heuristic
node_router -->|"executes chain"| node_executor
node_router -->|"records outcomes"| node_metrics
node_router -->|"selects models"| node_config
node_router -.->|"dispatches classifier"| node_hybrid
node_hybrid -->|"scores first"| node_heuristic
node_hybrid -->|"asks when uncertain"| node_llmclassifier
node_llmclassifier -->|"labels with model"| node_executor
node_executor -->|"uses retry settings"| node_config
node_executor -->|"calls provider protocol"| node_providerbase
node_executor -->|"checks and updates"| node_breaker
node_registry -->|"reads credentials"| node_config
node_registry -->|"registers adapter"| node_openai
node_registry -->|"registers adapter"| node_gemini
node_openai -.->|"posts and maps errors"| node_providerbase
node_gemini -->|"posts and maps errors"| node_providerbase
node_openai -.->|"requests completions"| node_openaiservice
node_gemini -->|"requests completions"| node_geminiservice
node_bootstrap -->|"creates HTTP client"| node_client
node_registry -->|"configures adapters"| node_client
node_metrics -->|"prices baseline"| node_config

click node_api "https://github.com/anishakalivarapu/llm-router/blob/main/app/api.py"
click node_schemas "https://github.com/anishakalivarapu/llm-router/blob/main/app/schemas.py"
click node_bootstrap "https://github.com/anishakalivarapu/llm-router/blob/main/app/main.py"
click node_router "https://github.com/anishakalivarapu/llm-router/blob/main/app/router.py"
click node_heuristic "https://github.com/anishakalivarapu/llm-router/blob/main/app/classifiers/heuristic.py"
click node_hybrid "https://github.com/anishakalivarapu/llm-router/blob/main/app/classifiers/hybrid.py"
click node_llmclassifier "https://github.com/anishakalivarapu/llm-router/blob/main/app/classifiers/llm.py"
click node_config "https://github.com/anishakalivarapu/llm-router/blob/main/app/config.py"
click node_executor "https://github.com/anishakalivarapu/llm-router/blob/main/app/executor.py"
click node_breaker "https://github.com/anishakalivarapu/llm-router/blob/main/app/executor.py"
click node_providerbase "https://github.com/anishakalivarapu/llm-router/blob/main/app/providers/base.py"
click node_registry "https://github.com/anishakalivarapu/llm-router/blob/main/app/providers/registry.py"
click node_openai "https://github.com/anishakalivarapu/llm-router/blob/main/app/providers/openai_provider.py"
click node_gemini "https://github.com/anishakalivarapu/llm-router/blob/main/app/providers/gemini_provider.py"
click node_metrics "https://github.com/anishakalivarapu/llm-router/blob/main/app/metrics.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_schemas,node_bootstrap,node_caller,node_client toneBlue
class node_router,node_heuristic,node_hybrid,node_llmclassifier toneAmber
class node_executor,node_breaker,node_openaiservice,node_geminiservice toneMint
class node_providerbase,node_registry,node_openai,node_gemini toneRose
class node_config,node_metrics toneIndigo
```

## Components

### HTTP API

- [FastAPI endpoints](https://github.com/anishakalivarapu/llm-router/blob/main/app/api.py): `app/api.py`
- [Request and response schemas](https://github.com/anishakalivarapu/llm-router/blob/main/app/schemas.py): `app/schemas.py`
- [App and dependency setup](https://github.com/anishakalivarapu/llm-router/blob/main/app/main.py): `app/main.py`

### Classification and Routing

- [Query router](https://github.com/anishakalivarapu/llm-router/blob/main/app/router.py): `app/router.py`
- [Heuristic classifier](https://github.com/anishakalivarapu/llm-router/blob/main/app/classifiers/heuristic.py): `app/classifiers/heuristic.py`
- [Hybrid classifier](https://github.com/anishakalivarapu/llm-router/blob/main/app/classifiers/hybrid.py): `app/classifiers/hybrid.py`
- [LLM classifier](https://github.com/anishakalivarapu/llm-router/blob/main/app/classifiers/llm.py): `app/classifiers/llm.py`

### Model Execution

- [Retries and escalation](https://github.com/anishakalivarapu/llm-router/blob/main/app/executor.py): `app/executor.py`
- [Circuit breaker](https://github.com/anishakalivarapu/llm-router/blob/main/app/executor.py): `app/executor.py`

### Provider Adapters

- [Provider protocol and HTTP](https://github.com/anishakalivarapu/llm-router/blob/main/app/providers/base.py): `app/providers/base.py`
- [Provider registry](https://github.com/anishakalivarapu/llm-router/blob/main/app/providers/registry.py): `app/providers/registry.py`
- [OpenAI adapter](https://github.com/anishakalivarapu/llm-router/blob/main/app/providers/openai_provider.py): `app/providers/openai_provider.py`
- [Gemini adapter](https://github.com/anishakalivarapu/llm-router/blob/main/app/providers/gemini_provider.py): `app/providers/gemini_provider.py`

### Configuration and Metrics

- [Settings and model ladder](https://github.com/anishakalivarapu/llm-router/blob/main/app/config.py): `app/config.py`
- [In-memory metrics](https://github.com/anishakalivarapu/llm-router/blob/main/app/metrics.py): `app/metrics.py`

### Other components

- **API client**
- **OpenAI API**
- **Gemini API**
- **HTTPX client**

## Connections

- API client → FastAPI endpoints: sends requests (evidence: `README.md`)
- FastAPI endpoints → Request and response schemas: validates and returns (evidence: `app/api.py`)
- FastAPI endpoints → Query router: dispatches routes (evidence: `app/api.py`)
- App and dependency setup → FastAPI endpoints: registers endpoints (evidence: `app/main.py`)
- App and dependency setup → Query router: builds router (evidence: `app/main.py`)
- App and dependency setup → Retries and escalation: builds executor (evidence: `app/main.py`)
- App and dependency setup → In-memory metrics: creates metrics (evidence: `app/main.py`)
- App and dependency setup → Provider registry: builds providers (evidence: `app/main.py`)
- Query router → Heuristic classifier: uses fallback (evidence: `app/router.py`)
- Query router → Retries and escalation: executes chain (evidence: `app/router.py`)
- Query router → In-memory metrics: records outcomes (evidence: `app/router.py`)
- Query router → Settings and model ladder: selects models (evidence: `app/router.py`)
- Query router → Hybrid classifier: dispatches classifier
- Hybrid classifier → Heuristic classifier: scores first (evidence: `app/classifiers/hybrid.py`)
- Hybrid classifier → LLM classifier: asks when uncertain (evidence: `app/classifiers/hybrid.py`)
- LLM classifier → Retries and escalation: labels with model (evidence: `app/classifiers/llm.py`)
- Retries and escalation → Settings and model ladder: uses retry settings (evidence: `app/executor.py`)
- Retries and escalation → Provider protocol and HTTP: calls provider protocol (evidence: `app/executor.py`)
- Retries and escalation → Circuit breaker: checks and updates (evidence: `app/executor.py`)
- Provider registry → Settings and model ladder: reads credentials (evidence: `app/providers/registry.py`)
- Provider registry → OpenAI adapter: registers adapter (evidence: `app/providers/registry.py`)
- Provider registry → Gemini adapter: registers adapter (evidence: `app/providers/registry.py`)
- OpenAI adapter → Provider protocol and HTTP: posts and maps errors (evidence: `app/providers/openai_provider.py`)
- Gemini adapter → Provider protocol and HTTP: posts and maps errors (evidence: `app/providers/gemini_provider.py`)
- OpenAI adapter → OpenAI API: requests completions
- Gemini adapter → Gemini API: requests completions (evidence: `app/providers/gemini_provider.py`)
- App and dependency setup → HTTPX client: creates HTTP client (evidence: `app/main.py`)
- Provider registry → HTTPX client: configures adapters (evidence: `app/providers/registry.py`)
- In-memory metrics → Settings and model ladder: prices baseline (evidence: `app/metrics.py`)
