How stanleyyoga123/incidentbench works
IncidentBench is a Kubernetes incident-response benchmark: researchers run repeatable scenarios, observe a bundled or external response system, preserve run evidence, then grade and compare outcomes offline. The graph follows the Runner’s experiment flow through applications, traffic, fault injection, response, and archiving, then the Grader’s assessment and reporting. It also preserves the reference agents’ detect–investigate–remediate–verify–learn stages and the MCP evidence tools. Some subsystem links are documented in the README rather than demonstrated in sampled source; those edges cite that documentation.
Main entry points: experiment_runner.py Experiment Phases · pipeline.py Offline Grader · service.py Workflow Orchestrator · engine.py Guarded Remediation
Components 21
Experiment Runner
- Scenario Runner
EvaluationPlatform/Runner/testbed/main.py - Experiment Phases
EvaluationPlatform/Runner/testbed/orchestration/experiment_runner.py - Microservice Apps
EvaluationPlatform/Runner/resources/applications - Traffic Profiles
EvaluationPlatform/Runner/testbed/loadgenerator - Fault Schedules
EvaluationPlatform/Runner/testbed/chaos - Telemetry Capture
EvaluationPlatform/Runner/testbed/evaluator - Run Evidence Archive
EvaluationPlatform/Runner/testbed/artifacts
Response System
- Anomaly Detection
Agents/AnomalyDetector/src - Workflow Orchestrator
EvaluationPlatform/Orchestrator/app/service.py - Root-Cause Analysis
Agents/RCAAgent/app/engine.py - Guarded Remediation
Agents/RemediatorAgent/app/engine.py - Action Verification
Agents/RemediatorAgent/app - Evidence-Linked Learning
Agents/LearningAgent/app/engine.py
Evidence Tools
- MCP Tool Server
Agents/MCPTools/app/server.py - Telemetry and Cluster Tools
Agents/MCPTools/app/tools
Evaluation and Reporting
- Offline Grader
EvaluationPlatform/Grader/grader/pipeline.py - Operational Comparisons
EvaluationPlatform/Grader/grader/research.py - Reports and Timelines
EvaluationPlatform/Grader/grader/reports.py
Other components
- Researcher
- Kubernetes Cluster
- Configured Model Judge
Connections 29
- Researcher → Scenario Runner: runs scenarios
README.md - Scenario Runner → Experiment Phases: starts phases
EvaluationPlatform/Runner/testbed/main.py - Experiment Phases → Microservice Apps: deploys and resets
README.md - Experiment Phases → Traffic Profiles: starts traffic
- Experiment Phases → Fault Schedules: executes faults
README.md - Traffic Profiles → Microservice Apps: drives workloads
README.md - Fault Schedules → Kubernetes Cluster: injects faults
README.md - Microservice Apps → Kubernetes Cluster: run on
README.md - Experiment Phases → Telemetry Capture: captures measurements
README.md - Telemetry Capture → Run Evidence Archive: preserves evidence
README.md - Experiment Phases → Run Evidence Archive: archives run inputs
README.md - Anomaly Detection → Kubernetes Cluster: observes symptoms
README.md - Anomaly Detection → Workflow Orchestrator: submits incident leads
README.md - Workflow Orchestrator → Root-Cause Analysis: queues investigation
EvaluationPlatform/Orchestrator/app/service.py - Root-Cause Analysis → MCP Tool Server: gathers evidence
README.md - MCP Tool Server → Telemetry and Cluster Tools: dispatches tools
Agents/MCPTools/app/server.py - Telemetry and Cluster Tools → Kubernetes Cluster: queries current state
README.md - Workflow Orchestrator → Guarded Remediation: submits remediation
EvaluationPlatform/Orchestrator/app/service.py - Guarded Remediation → MCP Tool Server: uses guarded tools
README.md - Guarded Remediation → Kubernetes Cluster: applies and verifies changes
README.md - Workflow Orchestrator → Evidence-Linked Learning: submits learning
EvaluationPlatform/Orchestrator/app/service.py - Evidence-Linked Learning → Workflow Orchestrator: returns lessons
README.md - Evidence-Linked Learning → Root-Cause Analysis: informs later investigations
README.md - Offline Grader → Run Evidence Archive: reads archived runs
README.md - Offline Grader → Configured Model Judge: requests semantic scoring
README.md - Offline Grader → Operational Comparisons: computes comparisons
EvaluationPlatform/Grader/grader/pipeline.py - Operational Comparisons → Reports and Timelines: feeds assessments
- Offline Grader → Reports and Timelines: produces reports
README.md - Researcher → Offline Grader: inspects outcomes
README.md