# udbhavsaxena/agentbench-lite architecture

> Architecture diagram of the GitHub repository udbhavsaxena/agentbench-lite, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/udbhavsaxena/agentbench-lite
- Repository: https://github.com/udbhavsaxena/agentbench-lite

## Overview

AgentBench-Lite evaluates multi-agent systems: a CLI or API submits benchmark tasks, the orchestrator dispatches them to specialist agents, and optional Critic review accompanies metric computation and persisted results. Research retrieves and summarizes tool outputs before LLM synthesis; Planning produces structured plans. The API can also run one-off agent requests, while the dashboard explores stored benchmark history, metrics, traces, and failures. The README documents calculator and document capabilities and seven dashboard views; their implementation details are not sampled, so uncertain call wiring is omitted.

Main entry points: [`app/main.py`](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/main.py) (FastAPI), [`app/eval/runner.py`](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/eval/runner.py) (Benchmark Runner), [`app/agents/orchestrator.py`](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/orchestrator.py) (Orchestrator), [`app/agents/research_agent.py`](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/research_agent.py) (Research Agent)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry Points"]
  node_cli["Benchmark CLI<br/>[cli.py]"]
  node_api["FastAPI<br/>[main.py]"]
  node_runner["Benchmark Runner<br/>[runner.py]"]
  node_tasks["Benchmark Tasks"]
end

subgraph group_agents["Agent Runtime"]
  node_orchestrator["Orchestrator<br/>[orchestrator.py]"]
  node_research["Research Agent<br/>[research_agent.py]"]
  node_planning["Planning Agent<br/>[planning_agent.py]"]
  node_critic["Critic Agent<br/>[critic_agent.py]"]
  node_base["Agent Base<br/>[base.py]"]
end

subgraph group_tools["Tools and Models"]
  node_search["Search Tool<br/>[search_tool.py]"]
  node_documents["Document Tools<br/>[document_tool.py]"]
  node_calculator["Calculator Tool<br/>[calculator_tool.py]"]
  node_mocktools["Mock Tools<br/>[mock_tools.py]"]
  node_llm["LLM Client<br/>[llm_client.py]"]
end

subgraph group_evaluation["Evaluation and Storage"]
  node_models["Data Models<br/>[models.py]"]
  node_benchmark["Task Loader<br/>[benchmark.py]"]
  node_metrics["Metric Engine<br/>[metrics.py]"]
  node_judge["Critic Parser<br/>[judge.py]"]
  node_database[("DuckDB Store<br/>[db.py]")]
end

subgraph group_views["Results Views"]
  node_dashboard["Evaluation Dashboard<br/>[streamlit_app.py]"]
end

node_user(("User"))
node_openai["OpenAI Service"]

node_user -->|"runs benchmark"| node_cli
node_user -->|"submits requests"| node_api
node_cli -->|"starts run"| node_runner
node_runner -->|"loads tasks"| node_benchmark
node_benchmark -->|"reads JSONL"| node_tasks
node_runner -->|"executes tasks"| node_orchestrator
node_api -->|"runs tasks"| node_orchestrator
node_api -->|"loads tasks"| node_benchmark
node_orchestrator -->|"dispatches"| node_research
node_orchestrator -->|"dispatches"| node_planning
node_orchestrator -->|"reviews output"| node_critic
node_research -->|"searches"| node_search
node_research -->|"retrieves and summarizes"| node_documents
node_research -->|"synthesizes with"| node_llm
node_planning -->|"generates plan"| node_llm
node_critic -->|"scores output"| node_llm
node_search -->|"mock search"| node_mocktools
node_documents -->|"mock retrieval"| node_mocktools
node_llm -.->|"calls when configured"| node_openai
node_runner -->|"parses critic score"| node_judge
node_runner -->|"computes metrics"| node_metrics
node_runner -->|"persists run and results"| node_database
node_api -->|"parses critic score"| node_judge
node_api -->|"computes metrics"| node_metrics
node_api -->|"reads and writes results"| node_database
node_dashboard -->|"reads history and results"| node_database

click node_cli "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/cli.py"
click node_api "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/main.py"
click node_runner "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/eval/runner.py"
click node_tasks "https://github.com/udbhavsaxena/agentbench-lite/blob/main/data/benchmark_tasks.jsonl"
click node_orchestrator "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/orchestrator.py"
click node_research "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/research_agent.py"
click node_planning "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/planning_agent.py"
click node_critic "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/critic_agent.py"
click node_base "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/base.py"
click node_search "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/tools/search_tool.py"
click node_documents "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/tools/document_tool.py"
click node_calculator "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/tools/calculator_tool.py"
click node_mocktools "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/tools/mock_tools.py"
click node_llm "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/llm_client.py"
click node_models "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/storage/models.py"
click node_benchmark "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/eval/benchmark.py"
click node_metrics "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/eval/metrics.py"
click node_judge "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/eval/judge.py"
click node_database "https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/storage/db.py"
click node_dashboard "https://github.com/udbhavsaxena/agentbench-lite/blob/main/dashboard/streamlit_app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_api,node_runner,node_tasks,node_user toneBlue
class node_orchestrator,node_research,node_planning,node_critic,node_base toneAmber
class node_search,node_documents,node_calculator,node_mocktools,node_llm toneMint
class node_models,node_benchmark,node_metrics,node_judge,node_database toneRose
class node_dashboard,node_openai toneIndigo
```

## Components

### Entry Points

- [Benchmark CLI](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/cli.py): `app/cli.py`
- [FastAPI](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/main.py): `app/main.py`
- [Benchmark Runner](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/eval/runner.py): `app/eval/runner.py`
- [Benchmark Tasks](https://github.com/udbhavsaxena/agentbench-lite/blob/main/data/benchmark_tasks.jsonl): `data/benchmark_tasks.jsonl`

### Agent Runtime

- [Orchestrator](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/orchestrator.py): `app/agents/orchestrator.py`
- [Research Agent](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/research_agent.py): `app/agents/research_agent.py`
- [Planning Agent](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/planning_agent.py): `app/agents/planning_agent.py`
- [Critic Agent](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/critic_agent.py): `app/agents/critic_agent.py`
- [Agent Base](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/agents/base.py): `app/agents/base.py`

### Tools and Models

- [Search Tool](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/tools/search_tool.py): `app/tools/search_tool.py`
- [Document Tools](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/tools/document_tool.py): `app/tools/document_tool.py`
- [Calculator Tool](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/tools/calculator_tool.py): `app/tools/calculator_tool.py`
- [Mock Tools](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/tools/mock_tools.py): `app/tools/mock_tools.py`
- [LLM Client](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/llm_client.py): `app/llm_client.py`

### Evaluation and Storage

- [Data Models](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/storage/models.py): `app/storage/models.py`
- [Task Loader](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/eval/benchmark.py): `app/eval/benchmark.py`
- [Metric Engine](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/eval/metrics.py): `app/eval/metrics.py`
- [Critic Parser](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/eval/judge.py): `app/eval/judge.py`
- [DuckDB Store](https://github.com/udbhavsaxena/agentbench-lite/blob/main/app/storage/db.py): `app/storage/db.py`

### Results Views

- [Evaluation Dashboard](https://github.com/udbhavsaxena/agentbench-lite/blob/main/dashboard/streamlit_app.py): `dashboard/streamlit_app.py`

### Other components

- **User**
- **OpenAI Service**

## Connections

- User → Benchmark CLI: runs benchmark
- User → FastAPI: submits requests
- Benchmark CLI → Benchmark Runner: starts run
- Benchmark Runner → Task Loader: loads tasks
- Task Loader → Benchmark Tasks: reads JSONL
- Benchmark Runner → Orchestrator: executes tasks
- FastAPI → Orchestrator: runs tasks
- FastAPI → Task Loader: loads tasks
- Orchestrator → Research Agent: dispatches
- Orchestrator → Planning Agent: dispatches
- Orchestrator → Critic Agent: reviews output
- Research Agent → Search Tool: searches
- Research Agent → Document Tools: retrieves and summarizes
- Research Agent → LLM Client: synthesizes with
- Planning Agent → LLM Client: generates plan
- Critic Agent → LLM Client: scores output
- Search Tool → Mock Tools: mock search
- Document Tools → Mock Tools: mock retrieval
- LLM Client → OpenAI Service: calls when configured
- Benchmark Runner → Critic Parser: parses critic score
- Benchmark Runner → Metric Engine: computes metrics
- Benchmark Runner → DuckDB Store: persists run and results
- FastAPI → Critic Parser: parses critic score
- FastAPI → Metric Engine: computes metrics
- FastAPI → DuckDB Store: reads and writes results
- Evaluation Dashboard → DuckDB Store: reads history and results