How karpathy/llm-council works
LLM Council is a local web app for consulting multiple language models, comparing their anonymized peer evaluations, and presenting a chairman’s synthesis. A user submits a question through the chat; the backend runs and streams three distinct stages, while the app displays each result and persists the conversation. Users can also create and revisit conversations. The graph reflects the sampled streaming route and documented product capabilities; unsampled details are represented only where the README supports them.
Main entry points: main.py Conversation endpoints · council.py Council orchestration · ChatInterface.jsx Chat interface · openrouter.py OpenRouter client
Components 20
Web interface
- App state
frontend/src/App.jsx - Conversation sidebar
frontend/src/components/Sidebar.jsx - Chat interface
frontend/src/components/ChatInterface.jsx - First opinions view
frontend/src/components/Stage1.jsx - Peer rankings view
frontend/src/components/Stage2.jsx - Final answer view
frontend/src/components/Stage3.jsx - API client
frontend/src/api.js
Conversation API
- Conversation endpoints
backend/main.py - Conversation operations
backend/main.py
Council workflow
- Council orchestration
backend/council.py - First opinions
backend/council.py - Anonymized peer review
backend/council.py - Aggregate rankings
backend/council.py - Chairman synthesis
backend/council.py - Conversation titling
backend/council.py
Models and storage
- JSON conversation store
backend/storage.py - Council configuration
backend/config.py - OpenRouter client
backend/openrouter.py
Other components
- User
- OpenRouter service
Connections 28
- User → Chat interface: asks question
frontend/src/components/ChatInterface.jsx - App state → Conversation sidebar: renders
frontend/src/App.jsx - App state → Chat interface: renders
frontend/src/App.jsx - Chat interface → App state: submits message
frontend/src/components/ChatInterface.jsx - App state → API client: loads and streams
frontend/src/App.jsx - API client → Conversation endpoints: HTTP requests
frontend/src/api.js - Conversation endpoints → Conversation operations: handles conversations
backend/main.py - Conversation operations → JSON conversation store: reads and writes
backend/main.py - Conversation endpoints → Council orchestration: runs workflow
backend/main.py - Conversation endpoints → First opinions: collects stage one
backend/main.py - Conversation endpoints → Anonymized peer review: collects stage two
backend/main.py - Conversation endpoints → Aggregate rankings: calculates rankings
backend/main.py - Conversation endpoints → Chairman synthesis: requests synthesis
backend/main.py - Conversation endpoints → Conversation titling: generates title
backend/main.py - Council orchestration → First opinions: orchestrates
backend/council.py - Council orchestration → Anonymized peer review: orchestrates
backend/council.py - Council orchestration → Aggregate rankings: calculates
backend/council.py - Council orchestration → Chairman synthesis: orchestrates
backend/council.py - First opinions → OpenRouter client: queries models
backend/council.py - Anonymized peer review → OpenRouter client: queries models
backend/council.py - Chairman synthesis → OpenRouter client: queries chairman
backend/council.py - Conversation titling → OpenRouter client: queries title model
backend/council.py - OpenRouter client → OpenRouter service: sends model requests
backend/openrouter.py - Council orchestration → Council configuration: uses model settings
backend/council.py - OpenRouter client → Council configuration: uses API settings
backend/openrouter.py - Chat interface → First opinions view: renders responses
frontend/src/components/ChatInterface.jsx - Chat interface → Peer rankings view: renders rankings
frontend/src/components/ChatInterface.jsx - Chat interface → Final answer view: renders synthesis
frontend/src/components/ChatInterface.jsx