How malyadritalentz/sre-incident-env works
1. Repository purpose
- This repo implements a reinforcement-learning / agent-evaluation environment for SRE incident response.
- The core product is an interactive incident simulator with three tasks of increasing difficulty: alert triage, cascading failure diagnosis, and post-mortem writing.
2. Core architecture split
- The code is organized around three boundaries:
- Environment core: episode state, step logic, rewards, and grading.
- Task definitions: scenario content and deterministic scoring rules.
- API/runtime layer: HTTP/WebSocket server plus client/inference entrypoints.
- This separation lets the same environment run locally, in Docker, or as a hosted OpenEnv/HF Space service.
3. Environment state and action model
- models.py defines the typed contract for the simulator: Observation, Action, StepResult, and likely state/session objects.
- Observations expose only what the agent is allowed to know at each step: alerts, logs, metrics, team messages, timeline, feedback, and completion status.
- Actions are task-agnostic at the protocol level (classify, investigate, hypothesize, mitigate, escalate, resolve, write_postmortem) and carry task-specific payloads.
4. Simulation engine
- environment.py is the main runtime for reset/step/state/grade behavior.
- It appears to enforce episode limits, track investigations, reveal hidden information, emit shaped rewards, and determine termination.
- This is the boundary where the agent’s action history is transformed into the next observation and reward signal.
5. Task and grader subsystem
- tasks/scenarios.py contains the scenario-specific content for the three incident types.
- It also holds the deterministic graders that score episode output against ground truth.
- The graders are programmatic, not model-based, which makes evaluation reproducible and suitable for benchmark-style comparison.
6. Information flow by task
- Task 1 exposes all alerts up front and mostly scores classification quality.
- Task 2 uses partial observability: alerts are visible initially, and investigate actions progressively reveal logs and evidence from specific services.
- Task 3 exposes everything at the start and evaluates long-form structured synthesis via a post-mortem artifact.
7. Server/API layer
- app.py and server/app.py provide the runtime interface for agents and humans.
- The README indicates endpoints for /reset, /step, /state, /tasks, /grader, and /baseline, plus a WebSocket channel.
- This layer is a thin transport boundary over the environment core, translating HTTP/WebSocket requests into environment calls and returning serialized observations/results.
8. Client and evaluation entrypoints
- client.py likely contains helper code for interacting with the server programmatically.
- inference.py is the official submission/evaluation runner: it drives the environment, calls an external model endpoint, logs structured step events, and aggregates scores across tasks.
- baseline.py provides a deterministic heuristic policy for local comparison without an LLM.
9. External model dependency
- The inference pipeline can call an OpenAI-compatible / Hugging Face router API via environment variables like HF_TOKEN, API_BASE_URL, and MODEL_NAME.
- That dependency is outside the simulator itself; the repo treats the model as an external decision-maker consuming observations and emitting actions.
10. Deployment/runtime
- Dockerfile, openenv.yaml, and requirements.txt/pyproject.toml define packaging for HF Spaces / Docker SDK deployment and local execution.
- The repo is designed to run as a service on port 7860, with FastAPI/Uvicorn-style startup in local dev.
- uv.lock suggests a reproducible Python dependency lockfile.
11. Validation
- test_env.py contains unit tests for the environment behavior and likely the grading/step transitions.
- This is important architecturally because the environment is deterministic and benchmark-oriented; tests protect the contract between task definitions, state transitions, and scoring.
12. How to diagram it
- Draw the system as: Agent/LLM or baseline policy → API/server layer → environment engine → task/scenario + grader data.
- Show the environment producing observations and rewards back to the agent, with task scenarios controlling what is revealed and how success is scored.
- Keep the model endpoint external, and treat the server as a stateless transport over a per-session environment state.
Main entry points: environment.py Environment · app.py Server App · scenarios.py Scenarios · inference.py Inference
Components 16
API / Runtime
Transport and entrypoints that expose the simulator to agents and deployments.
- App Entry · Top-level application entrypoint for serving or local runtime.
app.py - Server App · FastAPI-style server exposing the simulator over HTTP and WebSocket.
server/app.py - Client · Programmatic client helpers for interacting with the server.
client.py - Inference · Drives episodes against a model endpoint and aggregates scores.
inference.py - Baseline · Deterministic reference policy for local comparison.
baseline.py
Simulation Core
Episode state, action handling, rewards, and grading logic.
- Models · Typed simulator contract for observations, actions, step results, and session state.
models.py - Environment · Reset/step loop, reward shaping, termination, and state transitions.
environment.py - Episode State · Per-session hidden and visible state tracked across steps.
environment.py - Action Flow · Transforms agent actions into investigations, mitigations, and episode progress.
environment.py - Reward / Grade · Computes shaped rewards and final grading signals from task outcomes.
environment.py
Tasks / Scenarios
Scenario content and deterministic scoring for the incident tasks.
- Scenarios · Incident definitions, hidden facts, and task-specific reveal structure.
tasks/scenarios.py - Alert Triage · All alerts are visible up front; scores classification quality.
tasks/scenarios.py - Cascading Diagnosis · Partial observability with investigations that reveal service evidence.
tasks/scenarios.py - Postmortem Writing · Full observability with structured long-form incident analysis.
tasks/scenarios.py - Graders · Programmatic scoring rules that compare outputs against ground truth.
tasks/scenarios.py
External Agent
Model or policy outside the simulator that chooses actions from observations.
- LLM / Policy · External decision-maker that consumes observations and emits actions.
Connections 20
- LLM / Policy → Server App: reset/step · Agent drives the simulator through the server API.
- Baseline → Server App: policy calls · Baseline uses the same runtime boundary for local episodes.
- Inference → LLM / Policy: model query · Inference runner calls an external model endpoint to choose actions.
- Client → Server App: api calls · Client interacts with the simulator over HTTP/WebSocket.
- App Entry → Server App: launches · Application entrypoint starts the server runtime.
- Server App → Environment: session ops · Server forwards reset, step, state, and task requests to the environment.
- Environment → Models: uses contract · Environment is implemented against the typed observation/action protocol.
- Environment → Episode State: stores · Environment maintains per-episode hidden and visible state.
- Environment → Action Flow: applies · Environment applies actions to advance the simulation.
- Environment → Reward / Grade: scores · Environment emits rewards and terminal grading outputs.
- Action Flow → Scenarios: reveals facts · Task content controls what evidence can be uncovered by actions.
- Reward / Grade → Graders: evaluates · Final scoring is driven by deterministic grader logic.
- Scenarios → Alert Triage: defines · Scenario set includes the alert triage task.
- Scenarios → Cascading Diagnosis: defines · Scenario set includes the cascading diagnosis task.
- Scenarios → Postmortem Writing: defines · Scenario set includes the postmortem writing task.
- Alert Triage → Graders: classifies · Task 1 is primarily scored by alert classification quality.
- Cascading Diagnosis → Graders: evidence score · Task 2 is scored by investigation quality and diagnosis.
- Postmortem Writing → Graders: artifact score · Task 3 is scored by postmortem structure and content.
- Inference → Server App: episodes · Runner executes the environment through the service layer.
- Environment → Inference: observations / rewards · Step results flow back to the runner for decision and logging.