How udbhavsaxena/agentbench-lite works
AgentBench-Lite evaluates multi-agent systems: a CLI or API submits benchmark tasks, the orchestrator dispatches them to specialist agents, and optional Critic review accompanies metric computation and persisted results. Research retrieves and summarizes tool outputs before LLM synthesis; Planning produces structured plans. The API can also run one-off agent requests, while the dashboard explores stored benchmark history, metrics, traces, and failures. The README documents calculator and document capabilities and seven dashboard views; their implementation details are not sampled, so uncertain call wiring is omitted.
Main entry points: main.py FastAPI · runner.py Benchmark Runner · orchestrator.py Orchestrator · research_agent.py Research Agent
Components 22
Entry Points
- Benchmark CLI
app/cli.py - FastAPI
app/main.py - Benchmark Runner
app/eval/runner.py - Benchmark Tasks
data/benchmark_tasks.jsonl
Agent Runtime
- Orchestrator
app/agents/orchestrator.py - Research Agent
app/agents/research_agent.py - Planning Agent
app/agents/planning_agent.py - Critic Agent
app/agents/critic_agent.py - Agent Base
app/agents/base.py
Tools and Models
- Search Tool
app/tools/search_tool.py - Document Tools
app/tools/document_tool.py - Calculator Tool
app/tools/calculator_tool.py - Mock Tools
app/tools/mock_tools.py - LLM Client
app/llm_client.py
Evaluation and Storage
- Data Models
app/storage/models.py - Task Loader
app/eval/benchmark.py - Metric Engine
app/eval/metrics.py - Critic Parser
app/eval/judge.py - DuckDB Store
app/storage/db.py
Results Views
- Evaluation Dashboard
dashboard/streamlit_app.py
Other components
- User
- OpenAI Service
Connections 26
- User → Benchmark CLI: runs benchmark
- User → FastAPI: submits requests
- Benchmark CLI → Benchmark Runner: starts run
- Benchmark Runner → Task Loader: loads tasks
- Task Loader → Benchmark Tasks: reads JSONL
- Benchmark Runner → Orchestrator: executes tasks
- FastAPI → Orchestrator: runs tasks
- FastAPI → Task Loader: loads tasks
- Orchestrator → Research Agent: dispatches
- Orchestrator → Planning Agent: dispatches
- Orchestrator → Critic Agent: reviews output
- Research Agent → Search Tool: searches
- Research Agent → Document Tools: retrieves and summarizes
- Research Agent → LLM Client: synthesizes with
- Planning Agent → LLM Client: generates plan
- Critic Agent → LLM Client: scores output
- Search Tool → Mock Tools: mock search
- Document Tools → Mock Tools: mock retrieval
- LLM Client → OpenAI Service: calls when configured
- Benchmark Runner → Critic Parser: parses critic score
- Benchmark Runner → Metric Engine: computes metrics
- Benchmark Runner → DuckDB Store: persists run and results
- FastAPI → Critic Parser: parses critic score
- FastAPI → Metric Engine: computes metrics
- FastAPI → DuckDB Store: reads and writes results
- Evaluation Dashboard → DuckDB Store: reads history and results