How ydnyshhh/offsec-gym works
OffSecGym is a research testbed for measuring autonomous security agents on isolated synthetic applications. Its main workflow provisions a range, lets an agent explore through a controlled gateway, validates submitted findings against hidden ground truth, and records evidence and resource use for scoring. Experiments compare scripted, monolithic, and worker-based orchestration, with optional structured memory and model providers. Range B adds an enterprise change-control scenario. The graph follows README-documented flows and relationships visible in the supplied source index; unsampled implementation details and research-script wiring are not inferred.
Main entry points: scripted.py Experiment runner · compose.py Audited HTTP gateway · workers.py Worker orchestration · deterministic.py Finding validator
Components 21
Experiment orchestration
- Command line
src/offsecgym/cli/main.py - Experiment runner
src/offsecgym/experiment/scripted.py - Scripted solver
src/offsecgym/solver/scripted.py - Monolithic solver
src/offsecgym/solver/monolithic.py - Worker orchestration
src/offsecgym/solver/workers.py - Model providers
src/offsecgym/providers/base.py
Ranges and access
- Range runtime
src/offsecgym/runtime/compose.py - Synthetic SaaS range
src/offsecgym/runtime/saas.py - Enterprise range
src/offsecgym/runtime/enterprise.py - Audited HTTP gateway
src/offsecgym/gateway/compose.py
Validation and records
- Finding validator
src/offsecgym/validation/deterministic.py - Hidden ground truth
src/offsecgym/runtime/oracle.py - Scoring and evaluation
src/offsecgym/evaluation.py - Event store
src/offsecgym/storage/event_store.py - Event projections
src/offsecgym/storage/projection.py - Evidence artifacts
src/offsecgym/providers/artifacts.py - Structured worldview
src/offsecgym/worldview/state.py
Contracts and providers
- Domain and event contracts
src/offsecgym/schemas - Runtime interfaces
src/offsecgym/interfaces.py
Research analysis
- Research analyses
src/offsecgym/research
Other components
- Researcher
Connections 29
- Researcher → Command line: runs commands
README.md - Command line → Range runtime: manages ranges
src/offsecgym/cli/main.py - Command line → Experiment runner: starts experiments
src/offsecgym/cli/main.py - Experiment runner → Scripted solver: runs solver
src/offsecgym/experiment/scripted.py - Experiment runner → Monolithic solver: runs solver
README.md - Experiment runner → Worker orchestration: runs orchestration
README.md - Monolithic solver → Model providers: requests model turns
README.md - Worker orchestration → Model providers: requests model turns
README.md - Range runtime → Synthetic SaaS range: builds range
src/offsecgym/runtime/compose.py - Range runtime → Enterprise range: builds range
src/offsecgym/runtime/compose.py - Experiment runner → Audited HTTP gateway: provisions access
README.md - Audited HTTP gateway → Synthetic SaaS range: forwards requests
README.md - Audited HTTP gateway → Enterprise range: forwards requests
README.md - Scripted solver → Audited HTTP gateway: executes actions
README.md - Monolithic solver → Audited HTTP gateway: executes actions
README.md - Worker orchestration → Audited HTTP gateway: executes actions
README.md - Experiment runner → Finding validator: submits findings
README.md - Finding validator → Hidden ground truth: checks ground truth
README.md - Finding validator → Scoring and evaluation: returns results
README.md - Audited HTTP gateway → Event store: records actions
README.md - Experiment runner → Event store: records run
README.md - Finding validator → Event store: records validation
README.md - Event store → Event projections: projects events
- Experiment runner → Evidence artifacts: stores evidence
README.md - Worker orchestration → Structured worldview: retrieves state
src/offsecgym/solver/workers.py - Experiment runner → Domain and event contracts: uses contracts
src/offsecgym/experiment/scripted.py - Worker orchestration → Domain and event contracts: uses task contracts
src/offsecgym/solver/workers.py - Experiment runner → Runtime interfaces: uses boundaries
src/offsecgym/experiment/scripted.py - Researcher → Research analyses: compares results
README.md