How codelion/optillm works
OptiLLM is an OpenAI-compatible inference proxy that routes chat requests through reasoning optimizations before returning responses. The principal workflow is client request → server-side approach selection → selected reasoning technique → model provider → optimized response. The README documents a broad technique catalog, while the source index and excerpts confirm selected dispatches and internal dependencies; links for unsampled features are limited to documented or indexed relationships.
Main entry points: server.py Approach dispatch · conversation_logger.py Conversation logging · mars.py MARS · moa.py Mixture of Agents
Components 19
API and routing
- API server
optillm/server.py - Approach dispatch
optillm/server.py - Model-based router
optillm/plugins/router_plugin.py - Conversation logging
optillm/conversation_logger.py
Reasoning techniques
- Reasoning methods
optillm - CePO
optillm/cepo/cepo.py - MARS
optillm/mars/mars.py - MARS agents
optillm/mars/agent.py - PlanSearch
optillm/plansearch.py - Mixture of Agents
optillm/moa.py - Best of N
optillm/bon.py
Plugins and research
- Deep research
optillm/plugins/deep_research/research_engine.py - Web search
optillm/plugins/web_search_plugin.py - LongCePO
optillm/plugins/longcepo/main.py - Prompt learning
optillm/plugins/spl/main.py
Inference and providers
- Local inference
optillm/inference.py - Multi-provider proxy
optillm/plugins/proxy/client.py
Other components
- OpenAI client
- LLM provider
Connections 17
- OpenAI client → API server: sends request
- API server → Approach dispatch: routes request
optillm/server.py - Approach dispatch → Best of N: dispatches
optillm/server.py - Approach dispatch → Mixture of Agents: dispatches
optillm/server.py - Approach dispatch → MARS: dispatches
optillm/server.py - Approach dispatch → CePO: dispatches
optillm/server.py - Approach dispatch → Reasoning methods: dispatches methods
optillm/plugins/router_plugin.py - Model-based router → Reasoning methods: selects approach
optillm/plugins/router_plugin.py - MARS → MARS agents: runs agents
optillm/mars/mars.py - Deep research → Web search: searches sources
optillm/plugins/deep_research/research_engine.py - LongCePO → LLM provider: requests completions
- Prompt learning → LLM provider: requests completions
- Multi-provider proxy → LLM provider: proxies requests
- API server → Local inference: uses local inference
optillm/server.py - Best of N → Conversation logging: records calls
optillm/bon.py - Mixture of Agents → Conversation logging: records calls
optillm/moa.py - PlanSearch → Conversation logging: records calls
optillm/plansearch.py