# bhaskararamkarri/debugarena architecture

> Architecture diagram of the GitHub repository bhaskararamkarri/debugarena, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhaskararamkarri/debugarena
- Repository: https://github.com/bhaskararamkarri/debugarena

## Overview

DebugArena evaluates coding agents on buggy repository tasks: the runner loads a task into the environment, an agent repeatedly proposes JSON edits, commands, or submission, and the environment uses a sandbox and hidden tests to score progress. Completed episodes are judged and logged for review, dashboard analysis, and SFT/DPO dataset export. The core loop and runner ownership are visible in excerpts; dashboard and export connections are documented in the README, while the export implementation itself was not sampled.

Main entry points: [`agentgym/runner.py`](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/runner.py) (Episode runner), [`agentgym/env.py`](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/env.py) (Bug-fix environment), [`runs`](https://github.com/bhaskararamkarri/debugarena/tree/main/runs) (Episode trajectories), [`agentgym/agent.py`](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/agent.py) (Coding agent)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_benchmark["Task Benchmark"]
  node_tasks["Task definitions"]
end

subgraph group_execution["Agent Evaluation"]
  node_runner["Episode runner<br/>[runner.py]"]
  node_env["Bug-fix environment<br/>[env.py]"]
  node_agent["Coding agent<br/>[agent.py]"]
  node_sandbox["Isolated execution<br/>[sandbox.py]"]
  node_reward["Reward scoring<br/>[reward.py]"]
  node_judge["Fix quality judge<br/>[judge.py]"]
  node_tracer["Episode tracing<br/>[tracer.py]"]
end

subgraph group_artifacts["Evaluation Artifacts"]
  node_trajectory["Episode trajectories"]
  node_summaries["Run summaries"]
end

subgraph group_analysis["Review and Training"]
  node_dashboard["Results dashboard<br/>[app.py]"]
  node_datasets["SFT / DPO datasets"]
end

node_user(("Evaluation operator"))
node_model_api["LLM provider"]
node_langsmith["LangSmith"]

node_user -->|"starts evaluation"| node_runner
node_runner -.->|"loads task data"| node_tasks
node_runner -->|"resets and steps"| node_env
node_runner -->|"requests actions"| node_agent
node_agent -.->|"requests completions"| node_model_api
node_env -->|"writes and executes"| node_sandbox
node_env -->|"computes reward"| node_reward
node_env -->|"runs hidden tests"| node_sandbox
node_runner -->|"evaluates final fix"| node_judge
node_runner -->|"writes episode steps"| node_trajectory
node_runner -->|"writes run results"| node_summaries
node_runner -.->|"records trace spans"| node_tracer
node_tracer -.->|"sends spans"| node_langsmith
node_trajectory -->|"displays run data"| node_dashboard
node_trajectory -->|"exports training data"| node_datasets

click node_tasks "https://github.com/bhaskararamkarri/debugarena/tree/main/tasks"
click node_runner "https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/runner.py"
click node_env "https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/env.py"
click node_agent "https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/agent.py"
click node_sandbox "https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/sandbox.py"
click node_reward "https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/reward.py"
click node_judge "https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/judge.py"
click node_tracer "https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/tracer.py"
click node_trajectory "https://github.com/bhaskararamkarri/debugarena/tree/main/runs"
click node_summaries "https://github.com/bhaskararamkarri/debugarena/tree/main/runs"
click node_dashboard "https://github.com/bhaskararamkarri/debugarena/blob/main/dashboard/app.py"
click node_datasets "https://github.com/bhaskararamkarri/debugarena/tree/main/dataset"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tasks toneBlue
class node_runner,node_env,node_agent,node_sandbox,node_reward,node_judge,node_tracer toneAmber
class node_trajectory,node_summaries toneMint
class node_dashboard,node_datasets toneRose
class node_user,node_model_api,node_langsmith toneIndigo
```

## Components

### Task Benchmark

- [Task definitions](https://github.com/bhaskararamkarri/debugarena/tree/main/tasks): `tasks`

### Agent Evaluation

- [Episode runner](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/runner.py): `agentgym/runner.py`
- [Bug-fix environment](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/env.py): `agentgym/env.py`
- [Coding agent](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/agent.py): `agentgym/agent.py`
- [Isolated execution](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/sandbox.py): `agentgym/sandbox.py`
- [Reward scoring](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/reward.py): `agentgym/reward.py`
- [Fix quality judge](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/judge.py): `agentgym/judge.py`
- [Episode tracing](https://github.com/bhaskararamkarri/debugarena/blob/main/agentgym/tracer.py): `agentgym/tracer.py`

### Evaluation Artifacts

- [Episode trajectories](https://github.com/bhaskararamkarri/debugarena/tree/main/runs): `runs`
- [Run summaries](https://github.com/bhaskararamkarri/debugarena/tree/main/runs): `runs`

### Review and Training

- [Results dashboard](https://github.com/bhaskararamkarri/debugarena/blob/main/dashboard/app.py): `dashboard/app.py`
- [SFT / DPO datasets](https://github.com/bhaskararamkarri/debugarena/tree/main/dataset): `dataset`

### Other components

- **Evaluation operator**
- **LLM provider**
- **LangSmith**

## Connections

- Evaluation operator → Episode runner: starts evaluation
- Episode runner → Task definitions: loads task data
- Episode runner → Bug-fix environment: resets and steps (evidence: `agentgym/runner.py`)
- Episode runner → Coding agent: requests actions (evidence: `agentgym/runner.py`)
- Coding agent → LLM provider: requests completions
- Bug-fix environment → Isolated execution: writes and executes (evidence: `agentgym/env.py`)
- Bug-fix environment → Reward scoring: computes reward (evidence: `agentgym/env.py`)
- Bug-fix environment → Isolated execution: runs hidden tests (evidence: `agentgym/env.py`)
- Episode runner → Fix quality judge: evaluates final fix (evidence: `agentgym/runner.py`)
- Episode runner → Episode trajectories: writes episode steps (evidence: `agentgym/runner.py`)
- Episode runner → Run summaries: writes run results (evidence: `agentgym/runner.py`)
- Episode runner → Episode tracing: records trace spans (evidence: `agentgym/runner.py`)
- Episode tracing → LangSmith: sends spans
- Episode trajectories → Results dashboard: displays run data (evidence: `README.md`)
- Episode trajectories → SFT / DPO datasets: exports training data (evidence: `README.md`)
