# gen-verse/past-bench architecture

> Architecture diagram of the GitHub repository gen-verse/past-bench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gen-verse/past-bench
- Repository: https://github.com/gen-verse/past-bench

## Overview

PAST-Bench evaluates whether persistent agents improve across ordered, fresh-session task sequences, using persistence-on/off controls and artifact- and trace-level evidence. The main workflow is to select an agent, model, runtime, and task family; execute episodes; then judge and compare results. The repository also includes runtime adapters, sandboxing, mock services, and Hermes+ improvement mechanisms. Several connections below are documented by the README; where implementation excerpts were unavailable, links are marked unverified rather than asserted as observed calls.

Main entry points: [`src/past_bench/runtime/manager.py`](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/manager.py) (Session manager), [`src/past_bench/runner/agent_orchestrator.py`](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runner/agent_orchestrator.py) (Experiment orchestration), [`src/past_bench/graders/llm_judge.py`](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/graders/llm_judge.py) (LLM judge), [`agents/hermes-plus`](https://github.com/gen-verse/past-bench/tree/main/agents/hermes-plus) (Hermes+ agent)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_evaluation["Benchmark evaluation"]
  node_orchestrator["Experiment orchestration"]
  node_services["Run services<br/>[services.py]"]
  node_judge["LLM judge<br/>[llm_judge.py]"]
  node_results["Run results"]
end

subgraph group_runtime["Agent runtime"]
  node_runtime_manager["Session manager<br/>[manager.py]"]
  node_runtime_registry["Agent registry<br/>[registry.py]"]
  node_runtime_protocol["Session protocol<br/>[protocol.py]"]
  node_runtime_adapters["Runtime adapters"]
  node_runtime_client["Runtime client<br/>[client.py]"]
  node_runtime_server["Runtime server<br/>[server.py]"]
  node_sandbox["Sandbox<br/>[server.py]"]
end

subgraph group_agents["Agent systems"]
  node_hermes["Hermes agent"]
  node_hermes_plus["Hermes+ agent"]
  node_evolve["Improvement mechanisms"]
  node_comparison_agents["Comparison agents"]
  node_agent_zero["Agent Zero<br/>[agent.py]"]
end

subgraph group_support["Evaluation support"]
  node_mock_services["Mock services"]
end

node_user(("Experimenter"))

node_user -.->|"starts experiment"| node_orchestrator
node_orchestrator -.->|"uses services"| node_services
node_orchestrator -.->|"starts sessions"| node_runtime_manager
node_runtime_manager -->|"loads registry"| node_runtime_registry
node_runtime_manager -->|"creates adapter"| node_runtime_adapters
node_runtime_manager -->|"uses protocol"| node_runtime_protocol
node_runtime_client -.->|"sends requests"| node_runtime_server
node_runtime_manager -.->|"runs agent"| node_hermes
node_runtime_manager -.->|"runs agent"| node_hermes_plus
node_runtime_manager -.->|"runs agent"| node_comparison_agents
node_runtime_manager -.->|"runs agent"| node_agent_zero
node_orchestrator -.->|"configures runtime"| node_sandbox
node_orchestrator -.->|"evaluates against"| node_mock_services
node_orchestrator -.->|"requests scoring"| node_judge
node_judge -.->|"scores episodes"| node_results
node_orchestrator -.->|"writes traces"| node_results
node_hermes_plus -.->|"uses mechanisms"| node_evolve

click node_orchestrator "https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runner/agent_orchestrator.py"
click node_services "https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runner/services.py"
click node_judge "https://github.com/gen-verse/past-bench/blob/main/src/past_bench/graders/llm_judge.py"
click node_results "https://github.com/gen-verse/past-bench/tree/main/src/past_bench/runner"
click node_runtime_manager "https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/manager.py"
click node_runtime_registry "https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/registry.py"
click node_runtime_protocol "https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/protocol.py"
click node_runtime_adapters "https://github.com/gen-verse/past-bench/tree/main/src/past_bench/runtime/adapters"
click node_runtime_client "https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/client.py"
click node_runtime_server "https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/server.py"
click node_sandbox "https://github.com/gen-verse/past-bench/blob/main/src/past_bench/sandbox/server.py"
click node_hermes "https://github.com/gen-verse/past-bench/tree/main/agents/hermes-agent"
click node_hermes_plus "https://github.com/gen-verse/past-bench/tree/main/agents/hermes-plus"
click node_evolve "https://github.com/gen-verse/past-bench/blob/main/agents/hermes-plus/agent/evolve_controller.py"
click node_comparison_agents "https://github.com/gen-verse/past-bench/tree/main/agents/nanobot"
click node_agent_zero "https://github.com/gen-verse/past-bench/blob/main/agents/agent-zero/agent.py"
click node_mock_services "https://github.com/gen-verse/past-bench/tree/main/mock_services"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_orchestrator,node_services,node_judge,node_results toneBlue
class node_runtime_manager,node_runtime_registry,node_runtime_protocol,node_runtime_adapters,node_runtime_client,node_runtime_server,node_sandbox toneAmber
class node_hermes,node_hermes_plus,node_evolve,node_comparison_agents,node_agent_zero toneMint
class node_mock_services toneRose
class node_user toneIndigo
```

## Components

### Benchmark evaluation

- [Experiment orchestration](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runner/agent_orchestrator.py): `src/past_bench/runner/agent_orchestrator.py`
- [Run services](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runner/services.py): `src/past_bench/runner/services.py`
- [LLM judge](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/graders/llm_judge.py): `src/past_bench/graders/llm_judge.py`
- [Run results](https://github.com/gen-verse/past-bench/tree/main/src/past_bench/runner): `src/past_bench/runner`

### Agent runtime

- [Session manager](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/manager.py): `src/past_bench/runtime/manager.py`
- [Agent registry](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/registry.py): `src/past_bench/runtime/registry.py`
- [Session protocol](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/protocol.py): `src/past_bench/runtime/protocol.py`
- [Runtime adapters](https://github.com/gen-verse/past-bench/tree/main/src/past_bench/runtime/adapters): `src/past_bench/runtime/adapters`
- [Runtime client](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/client.py): `src/past_bench/runtime/client.py`
- [Runtime server](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/runtime/server.py): `src/past_bench/runtime/server.py`
- [Sandbox](https://github.com/gen-verse/past-bench/blob/main/src/past_bench/sandbox/server.py): `src/past_bench/sandbox/server.py`

### Agent systems

- [Hermes agent](https://github.com/gen-verse/past-bench/tree/main/agents/hermes-agent): `agents/hermes-agent`
- [Hermes+ agent](https://github.com/gen-verse/past-bench/tree/main/agents/hermes-plus): `agents/hermes-plus`
- [Improvement mechanisms](https://github.com/gen-verse/past-bench/blob/main/agents/hermes-plus/agent/evolve_controller.py): `agents/hermes-plus/agent/evolve_controller.py`
- [Comparison agents](https://github.com/gen-verse/past-bench/tree/main/agents/nanobot): `agents/nanobot`
- [Agent Zero](https://github.com/gen-verse/past-bench/blob/main/agents/agent-zero/agent.py): `agents/agent-zero/agent.py`

### Evaluation support

- [Mock services](https://github.com/gen-verse/past-bench/tree/main/mock_services): `mock_services`

### Other components

- **Experimenter**

## Connections

- Experimenter → Experiment orchestration: starts experiment
- Experiment orchestration → Run services: uses services
- Experiment orchestration → Session manager: starts sessions
- Session manager → Agent registry: loads registry (evidence: `src/past_bench/runtime/manager.py`)
- Session manager → Runtime adapters: creates adapter (evidence: `src/past_bench/runtime/manager.py`)
- Session manager → Session protocol: uses protocol (evidence: `src/past_bench/runtime/manager.py`)
- Runtime client → Runtime server: sends requests
- Session manager → Hermes agent: runs agent
- Session manager → Hermes+ agent: runs agent
- Session manager → Comparison agents: runs agent
- Session manager → Agent Zero: runs agent
- Experiment orchestration → Sandbox: configures runtime
- Experiment orchestration → Mock services: evaluates against
- Experiment orchestration → LLM judge: requests scoring
- LLM judge → Run results: scores episodes
- Experiment orchestration → Run results: writes traces
- Hermes+ agent → Improvement mechanisms: uses mechanisms (evidence: `agents/hermes-plus/tools/skill_manager_tool.py`)
