How mantisanalytics/shockbench-flow-starter works
ShockBench-Flow starter kit helps users develop policies for a supply-network Gymnasium task, evaluate them locally, and validate or submit them to Codabench. The principal workflow is user → agent → local scoring, with a separate check-and-submit path. The graph includes documented training/search examples and the benchmark engine as external dependencies; their internal implementations are not in this repository. Relationships follow the excerpts, SOURCE INDEX references, or README, so some example links are documented rather than source-verified.
Main entry points: cli.py sbf CLI · agent.py Template agent · scoring.py Episode scoring · examples Training and search
Components 14
Policies and examples
- Template agent
agents/template/agent.py - Random agent
agents/random/agent.py - Heuristic agent
agents/heuristic/agent.py - Training and search
examples
Local scoring and checks
- sbf CLI
src/sbf_starter/cli.py - Agent resolution
src/sbf_starter/agents.py - Episode scoring
src/sbf_starter/scoring.py - Submission checks
src/sbf_starter/check.py - Container runs
src/sbf_starter/container.py
Submission workflow
- Submission packaging
src/sbf_starter/cli.py - Codabench client
src/sbf_starter/codabench.py
Benchmark runtime
- ShockBench engine
Other components
- Agent developer
- Codabench
Connections 17
- Agent developer → sbf CLI: runs commands
README.md - Agent developer → Template agent: edits policy
README.md - Template agent → Training and search: informs experiments
- sbf CLI → Agent resolution: resolves agents
src/sbf_starter/cli.py - sbf CLI → Episode scoring: requests scores
src/sbf_starter/cli.py - Episode scoring → Agent resolution: loads agents
src/sbf_starter/scoring.py - Episode scoring → ShockBench engine: builds episodes
src/sbf_starter/scoring.py - sbf CLI → Submission checks: runs checks
src/sbf_starter/cli.py - sbf CLI → Container runs: requests container run
src/sbf_starter/cli.py - Container runs → ShockBench engine: plays container episodes
src/sbf_starter/container.py - sbf CLI → Submission packaging: packs agent
src/sbf_starter/cli.py - sbf CLI → Codabench client: handles competition commands
src/sbf_starter/cli.py - Codabench client → Codabench: authenticates and submits
src/sbf_starter/codabench.py - Training and search → ShockBench engine: trains and searches
README.md - Template agent → ShockBench engine: provides policy actions
README.md - Random agent → ShockBench engine: provides policy actions
README.md - Heuristic agent → ShockBench engine: provides policy actions
README.md