# codelion/optillm architecture

> Architecture diagram of the GitHub repository codelion/optillm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/codelion/optillm
- Repository: https://github.com/codelion/optillm

## Overview

OptiLLM is an OpenAI-compatible inference proxy that routes chat requests through reasoning optimizations before returning responses. The principal workflow is client request → server-side approach selection → selected reasoning technique → model provider → optimized response. The README documents a broad technique catalog, while the source index and excerpts confirm selected dispatches and internal dependencies; links for unsampled features are limited to documented or indexed relationships.

Main entry points: [`optillm/server.py`](https://github.com/codelion/optillm/blob/main/optillm/server.py) (Approach dispatch), [`optillm/conversation_logger.py`](https://github.com/codelion/optillm/blob/main/optillm/conversation_logger.py) (Conversation logging), [`optillm/mars/mars.py`](https://github.com/codelion/optillm/blob/main/optillm/mars/mars.py) (MARS), [`optillm/moa.py`](https://github.com/codelion/optillm/blob/main/optillm/moa.py) (Mixture of Agents)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and routing"]
  node_server["API server<br/>[server.py]"]
  node_approach_router["Approach dispatch<br/>[server.py]"]
  node_technique_router["Model-based router<br/>[router_plugin.py]"]
  node_conversation_log["Conversation logging"]
end

subgraph group_reasoning["Reasoning techniques"]
  node_reasoning_suite["Reasoning methods"]
  node_cepo["CePO<br/>[cepo.py]"]
  node_mars["MARS<br/>[mars.py]"]
  node_mars_agents["MARS agents<br/>[agent.py]"]
  node_plansearch["PlanSearch<br/>[plansearch.py]"]
  node_moa["Mixture of Agents<br/>[moa.py]"]
  node_bon["Best of N<br/>[bon.py]"]
end

subgraph group_plugins["Plugins and research"]
  node_deep_research["Deep research<br/>[research_engine.py]"]
  node_web_search["Web search"]
  node_longcepo["LongCePO<br/>[main.py]"]
  node_spl["Prompt learning<br/>[main.py]"]
end

subgraph group_providers["Inference and providers"]
  node_inference["Local inference<br/>[inference.py]"]
  node_proxy_client["Multi-provider proxy<br/>[client.py]"]
end

node_client(("OpenAI client"))
node_provider["LLM provider"]

node_client -->|"sends request"| node_server
node_server -->|"routes request"| node_approach_router
node_approach_router -->|"dispatches"| node_bon
node_approach_router -->|"dispatches"| node_moa
node_approach_router -.->|"dispatches"| node_mars
node_approach_router -.->|"dispatches"| node_cepo
node_approach_router -.->|"dispatches methods"| node_reasoning_suite
node_technique_router -.->|"selects approach"| node_reasoning_suite
node_mars -->|"runs agents"| node_mars_agents
node_deep_research -->|"searches sources"| node_web_search
node_longcepo -.->|"requests completions"| node_provider
node_spl -.->|"requests completions"| node_provider
node_proxy_client -.->|"proxies requests"| node_provider
node_server -.->|"uses local inference"| node_inference
node_bon -->|"records calls"| node_conversation_log
node_moa -->|"records calls"| node_conversation_log
node_plansearch -->|"records calls"| node_conversation_log

click node_server "https://github.com/codelion/optillm/blob/main/optillm/server.py"
click node_approach_router "https://github.com/codelion/optillm/blob/main/optillm/server.py"
click node_technique_router "https://github.com/codelion/optillm/blob/main/optillm/plugins/router_plugin.py"
click node_reasoning_suite "https://github.com/codelion/optillm/tree/main/optillm"
click node_cepo "https://github.com/codelion/optillm/blob/main/optillm/cepo/cepo.py"
click node_mars "https://github.com/codelion/optillm/blob/main/optillm/mars/mars.py"
click node_mars_agents "https://github.com/codelion/optillm/blob/main/optillm/mars/agent.py"
click node_plansearch "https://github.com/codelion/optillm/blob/main/optillm/plansearch.py"
click node_moa "https://github.com/codelion/optillm/blob/main/optillm/moa.py"
click node_bon "https://github.com/codelion/optillm/blob/main/optillm/bon.py"
click node_deep_research "https://github.com/codelion/optillm/blob/main/optillm/plugins/deep_research/research_engine.py"
click node_web_search "https://github.com/codelion/optillm/blob/main/optillm/plugins/web_search_plugin.py"
click node_longcepo "https://github.com/codelion/optillm/blob/main/optillm/plugins/longcepo/main.py"
click node_spl "https://github.com/codelion/optillm/blob/main/optillm/plugins/spl/main.py"
click node_conversation_log "https://github.com/codelion/optillm/blob/main/optillm/conversation_logger.py"
click node_inference "https://github.com/codelion/optillm/blob/main/optillm/inference.py"
click node_proxy_client "https://github.com/codelion/optillm/blob/main/optillm/plugins/proxy/client.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_server,node_approach_router,node_technique_router,node_conversation_log,node_client toneBlue
class node_reasoning_suite,node_cepo,node_mars,node_mars_agents,node_plansearch,node_moa,node_bon toneAmber
class node_deep_research,node_web_search,node_longcepo,node_spl toneMint
class node_inference,node_proxy_client toneRose
class node_provider toneIndigo
```

## Components

### API and routing

- [API server](https://github.com/codelion/optillm/blob/main/optillm/server.py): `optillm/server.py`
- [Approach dispatch](https://github.com/codelion/optillm/blob/main/optillm/server.py): `optillm/server.py`
- [Model-based router](https://github.com/codelion/optillm/blob/main/optillm/plugins/router_plugin.py): `optillm/plugins/router_plugin.py`
- [Conversation logging](https://github.com/codelion/optillm/blob/main/optillm/conversation_logger.py): `optillm/conversation_logger.py`

### Reasoning techniques

- [Reasoning methods](https://github.com/codelion/optillm/tree/main/optillm): `optillm`
- [CePO](https://github.com/codelion/optillm/blob/main/optillm/cepo/cepo.py): `optillm/cepo/cepo.py`
- [MARS](https://github.com/codelion/optillm/blob/main/optillm/mars/mars.py): `optillm/mars/mars.py`
- [MARS agents](https://github.com/codelion/optillm/blob/main/optillm/mars/agent.py): `optillm/mars/agent.py`
- [PlanSearch](https://github.com/codelion/optillm/blob/main/optillm/plansearch.py): `optillm/plansearch.py`
- [Mixture of Agents](https://github.com/codelion/optillm/blob/main/optillm/moa.py): `optillm/moa.py`
- [Best of N](https://github.com/codelion/optillm/blob/main/optillm/bon.py): `optillm/bon.py`

### Plugins and research

- [Deep research](https://github.com/codelion/optillm/blob/main/optillm/plugins/deep_research/research_engine.py): `optillm/plugins/deep_research/research_engine.py`
- [Web search](https://github.com/codelion/optillm/blob/main/optillm/plugins/web_search_plugin.py): `optillm/plugins/web_search_plugin.py`
- [LongCePO](https://github.com/codelion/optillm/blob/main/optillm/plugins/longcepo/main.py): `optillm/plugins/longcepo/main.py`
- [Prompt learning](https://github.com/codelion/optillm/blob/main/optillm/plugins/spl/main.py): `optillm/plugins/spl/main.py`

### Inference and providers

- [Local inference](https://github.com/codelion/optillm/blob/main/optillm/inference.py): `optillm/inference.py`
- [Multi-provider proxy](https://github.com/codelion/optillm/blob/main/optillm/plugins/proxy/client.py): `optillm/plugins/proxy/client.py`

### Other components

- **OpenAI client**
- **LLM provider**

## Connections

- OpenAI client → API server: sends request
- API server → Approach dispatch: routes request (evidence: `optillm/server.py`)
- Approach dispatch → Best of N: dispatches (evidence: `optillm/server.py`)
- Approach dispatch → Mixture of Agents: dispatches (evidence: `optillm/server.py`)
- Approach dispatch → MARS: dispatches (evidence: `optillm/server.py`)
- Approach dispatch → CePO: dispatches (evidence: `optillm/server.py`)
- Approach dispatch → Reasoning methods: dispatches methods (evidence: `optillm/plugins/router_plugin.py`)
- Model-based router → Reasoning methods: selects approach (evidence: `optillm/plugins/router_plugin.py`)
- MARS → MARS agents: runs agents (evidence: `optillm/mars/mars.py`)
- Deep research → Web search: searches sources (evidence: `optillm/plugins/deep_research/research_engine.py`)
- LongCePO → LLM provider: requests completions
- Prompt learning → LLM provider: requests completions
- Multi-provider proxy → LLM provider: proxies requests
- API server → Local inference: uses local inference (evidence: `optillm/server.py`)
- Best of N → Conversation logging: records calls (evidence: `optillm/bon.py`)
- Mixture of Agents → Conversation logging: records calls (evidence: `optillm/moa.py`)
- PlanSearch → Conversation logging: records calls (evidence: `optillm/plansearch.py`)
