# fastfedora/refactor-arena architecture

> Architecture diagram of the GitHub repository fastfedora/refactor-arena, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fastfedora/refactor-arena
- Repository: https://github.com/fastfedora/refactor-arena

## Overview

Refactor Arena is a Control Arena setting and CLI for evaluating repository-refactoring agents, optionally in an attack mode that adds a hidden vulnerability. The user configures an evaluation and selects a dataset; the setting builds task samples, policies guide agent behavior, and scorers evaluate the resulting changes. The README documents configurable prompts, tasks, policies, monitors, metrics, and evaluation settings. Source excerpts are partial, so links between the runtime subsystems are shown only where the README or indexed source references support them.

Main entry points: [`src/refactor_arena/settings/refactor_arena_setting.py`](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/settings/refactor_arena_setting.py) (Arena setting), [`src/refactor_arena/cli/main.py`](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/cli/main.py) (CLI), [`src/refactor_arena/cli/commands/rescore.py`](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/cli/commands/rescore.py) (Rescore), [`src/refactor_arena/datasets/dataset.py`](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/datasets/dataset.py) (Dataset samples)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["CLI and setup"]
  node_cli["CLI<br/>[main.py]"]
  node_init["Initialize<br/>[init.py]"]
  node_rescore["Rescore<br/>[rescore.py]"]
end

subgraph group_tasks["Task construction"]
  node_setting["Arena setting"]
  node_dataset["Dataset samples<br/>[dataset.py]"]
  node_dataset_model["Dataset model<br/>[dataset.py]"]
end

subgraph group_agents["Agent behavior"]
  node_honest_policy["Honest policy"]
  node_attack_policy["Attack policy"]
  node_agent_solver["Agent control"]
end

subgraph group_assessment["Scoring and analysis"]
  node_change_analysis["Change analysis"]
  node_diff_monitor["Diff monitor"]
  node_constraint_score["Constraint scorer"]
  node_code_metrics["Code metrics"]
  node_metric_types["Metric types"]
  node_eval_metrics["Evaluation metrics"]
  node_threshold_metric["Threshold metric"]
  node_run_scorer["Run scorer tool"]
  node_eval_log["Eval log reader"]
end

node_user(("Evaluator"))

node_user -->|"configures evaluation"| node_cli
node_cli -->|"runs setting"| node_setting
node_user -->|"sets up scenario"| node_init
node_cli -.->|"dispatches command"| node_rescore
node_setting -->|"builds samples"| node_dataset
node_dataset -->|"creates sample data"| node_dataset_model
node_setting -.->|"selects policy"| node_honest_policy
node_setting -.->|"selects attack mode"| node_attack_policy
node_setting -.->|"configures agent"| node_agent_solver
node_setting -.->|"configures monitor"| node_diff_monitor
node_setting -.->|"configures scoring"| node_constraint_score
node_constraint_score -.->|"assesses changes"| node_change_analysis
node_change_analysis -.->|"analyzes code"| node_code_metrics
node_code_metrics -.->|"produces metrics"| node_metric_types
node_setting -.->|"configures metrics"| node_eval_metrics
node_setting -.->|"configures metrics"| node_threshold_metric
node_run_scorer -.->|"reads evaluation output"| node_eval_log
node_rescore -.->|"uses scorer"| node_run_scorer

click node_cli "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/cli/main.py"
click node_init "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/cli/commands/init.py"
click node_rescore "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/cli/commands/rescore.py"
click node_setting "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/settings/refactor_arena_setting.py"
click node_dataset "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/datasets/dataset.py"
click node_dataset_model "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/datasets/dataset.py"
click node_honest_policy "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/policies/swe_agent_honest_policy.py"
click node_attack_policy "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/policies/swe_agent_attack_policy.py"
click node_agent_solver "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/solvers/agent_control_solver.py"
click node_change_analysis "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/solvers/analyze_code_changes.py"
click node_diff_monitor "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/scorers/git_diff_monitor.py"
click node_constraint_score "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/scorers/task_constraints_penalty.py"
click node_code_metrics "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/coding/analyze_code_changes.py"
click node_metric_types "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/coding/types/code_quality_metrics.py"
click node_eval_metrics "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/evals/_metrics/add_safety_metric.py"
click node_threshold_metric "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/evals/_metrics/add_threshold_accuracy_metric.py"
click node_run_scorer "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/evals/get_run_scorer_tool.py"
click node_eval_log "https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/evals/read_setting_eval_log.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_init,node_rescore toneBlue
class node_setting,node_dataset,node_dataset_model toneAmber
class node_honest_policy,node_attack_policy,node_agent_solver toneMint
class node_change_analysis,node_diff_monitor,node_constraint_score,node_code_metrics,node_metric_types,node_eval_metrics,node_threshold_metric,node_run_scorer,node_eval_log toneRose
class node_user toneIndigo
```

## Components

### CLI and setup

- [CLI](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/cli/main.py): `src/refactor_arena/cli/main.py`
- [Initialize](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/cli/commands/init.py): `src/refactor_arena/cli/commands/init.py`
- [Rescore](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/cli/commands/rescore.py): `src/refactor_arena/cli/commands/rescore.py`

### Task construction

- [Arena setting](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/settings/refactor_arena_setting.py): `src/refactor_arena/settings/refactor_arena_setting.py`
- [Dataset samples](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/datasets/dataset.py): `src/refactor_arena/datasets/dataset.py`
- [Dataset model](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/datasets/dataset.py): `src/refactor_arena/datasets/dataset.py`

### Agent behavior

- [Honest policy](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/policies/swe_agent_honest_policy.py): `src/refactor_arena/policies/swe_agent_honest_policy.py`
- [Attack policy](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/policies/swe_agent_attack_policy.py): `src/refactor_arena/policies/swe_agent_attack_policy.py`
- [Agent control](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/solvers/agent_control_solver.py): `src/refactor_arena/solvers/agent_control_solver.py`

### Scoring and analysis

- [Change analysis](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/solvers/analyze_code_changes.py): `src/refactor_arena/solvers/analyze_code_changes.py`
- [Diff monitor](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/scorers/git_diff_monitor.py): `src/refactor_arena/scorers/git_diff_monitor.py`
- [Constraint scorer](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/scorers/task_constraints_penalty.py): `src/refactor_arena/scorers/task_constraints_penalty.py`
- [Code metrics](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/coding/analyze_code_changes.py): `src/refactor_arena/utils/coding/analyze_code_changes.py`
- [Metric types](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/coding/types/code_quality_metrics.py): `src/refactor_arena/utils/coding/types/code_quality_metrics.py`
- [Evaluation metrics](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/evals/_metrics/add_safety_metric.py): `src/refactor_arena/utils/evals/_metrics/add_safety_metric.py`
- [Threshold metric](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/evals/_metrics/add_threshold_accuracy_metric.py): `src/refactor_arena/utils/evals/_metrics/add_threshold_accuracy_metric.py`
- [Run scorer tool](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/evals/get_run_scorer_tool.py): `src/refactor_arena/utils/evals/get_run_scorer_tool.py`
- [Eval log reader](https://github.com/fastfedora/refactor-arena/blob/main/src/refactor_arena/utils/evals/read_setting_eval_log.py): `src/refactor_arena/utils/evals/read_setting_eval_log.py`

### Other components

- **Evaluator**

## Connections

- Evaluator → CLI: configures evaluation (evidence: `README.md`)
- CLI → Arena setting: runs setting (evidence: `src/refactor_arena/cli/main.py`)
- Evaluator → Initialize: sets up scenario (evidence: `README.md`)
- CLI → Rescore: dispatches command
- Arena setting → Dataset samples: builds samples (evidence: `src/refactor_arena/settings/refactor_arena_setting.py`)
- Dataset samples → Dataset model: creates sample data (evidence: `src/refactor_arena/datasets/dataset.py`)
- Arena setting → Honest policy: selects policy (evidence: `README.md`)
- Arena setting → Attack policy: selects attack mode (evidence: `README.md`)
- Arena setting → Agent control: configures agent (evidence: `README.md`)
- Arena setting → Diff monitor: configures monitor (evidence: `README.md`)
- Arena setting → Constraint scorer: configures scoring (evidence: `README.md`)
- Constraint scorer → Change analysis: assesses changes
- Change analysis → Code metrics: analyzes code
- Code metrics → Metric types: produces metrics
- Arena setting → Evaluation metrics: configures metrics (evidence: `README.md`)
- Arena setting → Threshold metric: configures metrics (evidence: `README.md`)
- Run scorer tool → Eval log reader: reads evaluation output
- Rescore → Run scorer tool: uses scorer
