How gen-verse/past-bench works
PAST-Bench evaluates whether persistent agents improve across ordered, fresh-session task sequences, using persistence-on/off controls and artifact- and trace-level evidence. The main workflow is to select an agent, model, runtime, and task family; execute episodes; then judge and compare results. The repository also includes runtime adapters, sandboxing, mock services, and Hermes+ improvement mechanisms. Several connections below are documented by the README; where implementation excerpts were unavailable, links are marked unverified rather than asserted as observed calls.
Main entry points: manager.py Session manager · agent_orchestrator.py Experiment orchestration · llm_judge.py LLM judge · hermes-plus Hermes+ agent
Components 18
Benchmark evaluation
- Experiment orchestration
src/past_bench/runner/agent_orchestrator.py - Run services
src/past_bench/runner/services.py - LLM judge
src/past_bench/graders/llm_judge.py - Run results
src/past_bench/runner
Agent runtime
- Session manager
src/past_bench/runtime/manager.py - Agent registry
src/past_bench/runtime/registry.py - Session protocol
src/past_bench/runtime/protocol.py - Runtime adapters
src/past_bench/runtime/adapters - Runtime client
src/past_bench/runtime/client.py - Runtime server
src/past_bench/runtime/server.py - Sandbox
src/past_bench/sandbox/server.py
Agent systems
- Hermes agent
agents/hermes-agent - Hermes+ agent
agents/hermes-plus - Improvement mechanisms
agents/hermes-plus/agent/evolve_controller.py - Comparison agents
agents/nanobot - Agent Zero
agents/agent-zero/agent.py
Evaluation support
- Mock services
mock_services
Other components
- Experimenter
Connections 17
- Experimenter → Experiment orchestration: starts experiment
- Experiment orchestration → Run services: uses services
- Experiment orchestration → Session manager: starts sessions
- Session manager → Agent registry: loads registry
src/past_bench/runtime/manager.py - Session manager → Runtime adapters: creates adapter
src/past_bench/runtime/manager.py - Session manager → Session protocol: uses protocol
src/past_bench/runtime/manager.py - Runtime client → Runtime server: sends requests
- Session manager → Hermes agent: runs agent
- Session manager → Hermes+ agent: runs agent
- Session manager → Comparison agents: runs agent
- Session manager → Agent Zero: runs agent
- Experiment orchestration → Sandbox: configures runtime
- Experiment orchestration → Mock services: evaluates against
- Experiment orchestration → LLM judge: requests scoring
- LLM judge → Run results: scores episodes
- Experiment orchestration → Run results: writes traces
- Hermes+ agent → Improvement mechanisms: uses mechanisms
agents/hermes-plus/tools/skill_manager_tool.py