# aaditpani-rvu/stepwise architecture

> Architecture diagram of the GitHub repository aaditpani-rvu/stepwise, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aaditpani-rvu/stepwise
- Repository: https://github.com/aaditpani-rvu/stepwise

## Overview

STEPWISE monitors procedures from camera observations and judges detected actions against a compiled procedure specification, producing alerts and session records. The documented primary path is manual-to-spec compilation offline, then camera-to-world-state tracking and deterministic checking online; replay exercises the tracker and checker without hardware. Jenga and LEGO state implementations are represented where source evidence supports them. README also identifies Jigsaw, live reporting, temporal recognition and server/UI as capabilities or planned branches, but their implementation details were not sampled, so their connections are left unverified.

Main entry points: [`stepwise/session.py`](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/session.py) (Session orchestration), [`stepwise/perception/pipeline.py`](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/perception/pipeline.py) (Frame pipeline), [`stepwise/checker/engine.py`](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/checker/engine.py) (Deterministic checker), [`stepwise/live.py`](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/live.py) (Live runner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_spec["Procedure specs"]
  node_compiler["LLM compiler<br/>[compile.py]"]
  node_schema["Spec schema<br/>[schema.py]"]
  node_validation["Spec validation<br/>[validate.py]"]
  node_expressions["Safe expressions<br/>[expr.py]"]
  node_specfile["Procedure spec"]
end

subgraph group_vision["Perception"]
  node_calibration["Calibration<br/>[calibrate.py]"]
  node_hands["Hand tracking<br/>[hands.py]"]
  node_detection["Part detection<br/>[detect.py]"]
  node_pipeline["Frame pipeline<br/>[pipeline.py]"]
end

subgraph group_tracking["World state"]
  node_grid["Stud grid geometry<br/>[grid.py]"]
  node_buildstate["LEGO build state<br/>[build_state.py]"]
  node_jengastate["Jenga lattice<br/>[lattice.py]"]
  node_collapse["Collapse detection<br/>[collapse.py]"]
  node_worldstate["World state interface<br/>[base.py]"]
  node_jigsaw["Jigsaw tracking"]
end

subgraph group_judgment["Judgment and records"]
  node_events["Events and alerts<br/>[events.py]"]
  node_session["Session orchestration<br/>[session.py]"]
  node_checker["Deterministic checker<br/>[engine.py]"]
  node_evaluation["Rule evaluation<br/>[evaluate.py]"]
  node_jengarules["Jenga rules<br/>[rules_jenga.py]"]
  node_matching["Event matching<br/>[match.py]"]
  node_log[("Session log<br/>[sessionlog.py]")]
  node_replay["Scenario replay<br/>[replay.py]"]
  node_live["Live runner<br/>[live.py]"]
  node_temporal["Step recognizer"]
  node_explainer["Session explanation"]
  node_server["Server and web UI"]
end

node_manual["Procedure manual"]
node_camera(("Camera frames"))

node_manual -->|"compiles text"| node_compiler
node_compiler -->|"builds spec"| node_schema
node_compiler -->|"validates spec"| node_validation
node_validation -->|"checks expressions"| node_expressions
node_validation -->|"accepts valid spec"| node_specfile
node_specfile -->|"judged against"| node_checker
node_camera -->|"supplies frames"| node_pipeline
node_pipeline -->|"calibrates frame"| node_calibration
node_pipeline -->|"detects hands"| node_hands
node_pipeline -->|"detects parts"| node_detection
node_pipeline -->|"maps readings"| node_grid
node_live -->|"runs frames through"| node_pipeline
node_replay -->|"feeds scripted scene"| node_session
node_pipeline -.->|"provides readings"| node_buildstate
node_buildstate -->|"implements"| node_worldstate
node_jengastate -->|"implements"| node_worldstate
node_jengastate -->|"produces move events"| node_events
node_collapse -->|"updates collapse state"| node_jengastate
node_session -->|"reads state changes"| node_worldstate
node_session -->|"dispatches events"| node_checker
node_session -->|"collects alerts"| node_events
node_checker -->|"evaluates preconditions"| node_evaluation
node_checker -->|"matches events"| node_matching
node_matching -->|"selects rule policy"| node_jengarules
node_evaluation -->|"evaluates expressions"| node_expressions
node_session -->|"records session"| node_log
node_replay -->|"writes session record"| node_log
node_live -->|"runs session"| node_session
node_session -.->|"selects tracker by spec"| node_jigsaw
node_log -.->|"feeds report"| node_explainer
node_session -.->|"uses step hints"| node_temporal
node_live -.->|"serves live UI"| node_server

click node_compiler "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/compiler/compile.py"
click node_schema "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/compiler/schema.py"
click node_validation "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/compiler/validate.py"
click node_expressions "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/compiler/expr.py"
click node_calibration "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/perception/calibrate.py"
click node_hands "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/perception/hands.py"
click node_detection "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/perception/detect.py"
click node_pipeline "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/perception/pipeline.py"
click node_grid "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/lego/grid.py"
click node_buildstate "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/lego/build_state.py"
click node_jengastate "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/jenga/lattice.py"
click node_collapse "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/jenga/collapse.py"
click node_worldstate "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/base.py"
click node_events "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/events.py"
click node_session "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/session.py"
click node_checker "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/checker/engine.py"
click node_evaluation "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/checker/evaluate.py"
click node_jengarules "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/checker/rules_jenga.py"
click node_matching "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/checker/match.py"
click node_log "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/sessionlog.py"
click node_replay "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/replay.py"
click node_live "https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/live.py"
click node_jigsaw "https://github.com/aaditpani-rvu/stepwise/tree/main/stepwise/state"
click node_temporal "https://github.com/aaditpani-rvu/stepwise/tree/main/stepwise/temporal"
click node_explainer "https://github.com/aaditpani-rvu/stepwise/tree/main/stepwise/explain"
click node_server "https://github.com/aaditpani-rvu/stepwise/tree/main/stepwise/server"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_compiler,node_schema,node_validation,node_expressions,node_specfile toneBlue
class node_calibration,node_hands,node_detection,node_pipeline toneAmber
class node_grid,node_buildstate,node_jengastate,node_collapse,node_worldstate,node_jigsaw toneMint
class node_events,node_session,node_checker,node_evaluation,node_jengarules,node_matching,node_log,node_replay,node_live,node_temporal,node_explainer,node_server toneRose
class node_manual,node_camera toneIndigo
```

## Components

### Procedure specs

- [LLM compiler](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/compiler/compile.py): `stepwise/compiler/compile.py`
- [Spec schema](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/compiler/schema.py): `stepwise/compiler/schema.py`
- [Spec validation](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/compiler/validate.py): `stepwise/compiler/validate.py`
- [Safe expressions](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/compiler/expr.py): `stepwise/compiler/expr.py`
- **Procedure spec**

### Perception

- [Calibration](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/perception/calibrate.py): `stepwise/perception/calibrate.py`
- [Hand tracking](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/perception/hands.py): `stepwise/perception/hands.py`
- [Part detection](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/perception/detect.py): `stepwise/perception/detect.py`
- [Frame pipeline](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/perception/pipeline.py): `stepwise/perception/pipeline.py`

### World state

- [Stud grid geometry](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/lego/grid.py): `stepwise/state/lego/grid.py`
- [LEGO build state](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/lego/build_state.py): `stepwise/state/lego/build_state.py`
- [Jenga lattice](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/jenga/lattice.py): `stepwise/state/jenga/lattice.py`
- [Collapse detection](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/jenga/collapse.py): `stepwise/state/jenga/collapse.py`
- [World state interface](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/state/base.py): `stepwise/state/base.py`
- [Jigsaw tracking](https://github.com/aaditpani-rvu/stepwise/tree/main/stepwise/state): `stepwise/state`

### Judgment and records

- [Events and alerts](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/events.py): `stepwise/events.py`
- [Session orchestration](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/session.py): `stepwise/session.py`
- [Deterministic checker](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/checker/engine.py): `stepwise/checker/engine.py`
- [Rule evaluation](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/checker/evaluate.py): `stepwise/checker/evaluate.py`
- [Jenga rules](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/checker/rules_jenga.py): `stepwise/checker/rules_jenga.py`
- [Event matching](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/checker/match.py): `stepwise/checker/match.py`
- [Session log](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/sessionlog.py): `stepwise/sessionlog.py`
- [Scenario replay](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/replay.py): `stepwise/replay.py`
- [Live runner](https://github.com/aaditpani-rvu/stepwise/blob/main/stepwise/live.py): `stepwise/live.py`
- [Step recognizer](https://github.com/aaditpani-rvu/stepwise/tree/main/stepwise/temporal): `stepwise/temporal`
- [Session explanation](https://github.com/aaditpani-rvu/stepwise/tree/main/stepwise/explain): `stepwise/explain`
- [Server and web UI](https://github.com/aaditpani-rvu/stepwise/tree/main/stepwise/server): `stepwise/server`

### Other components

- **Procedure manual**
- **Camera frames**

## Connections

- Procedure manual → LLM compiler: compiles text (evidence: `README.md`)
- LLM compiler → Spec schema: builds spec (evidence: `stepwise/compiler/compile.py`)
- LLM compiler → Spec validation: validates spec (evidence: `stepwise/compiler/compile.py`)
- Spec validation → Safe expressions: checks expressions (evidence: `stepwise/compiler/validate.py`)
- Spec validation → Procedure spec: accepts valid spec (evidence: `README.md`)
- Procedure spec → Deterministic checker: judged against (evidence: `README.md`)
- Camera frames → Frame pipeline: supplies frames (evidence: `stepwise/perception/pipeline.py`)
- Frame pipeline → Calibration: calibrates frame (evidence: `stepwise/perception/pipeline.py`)
- Frame pipeline → Hand tracking: detects hands (evidence: `stepwise/perception/pipeline.py`)
- Frame pipeline → Part detection: detects parts (evidence: `stepwise/perception/pipeline.py`)
- Frame pipeline → Stud grid geometry: maps readings (evidence: `stepwise/perception/pipeline.py`)
- Live runner → Frame pipeline: runs frames through (evidence: `stepwise/live.py`)
- Scenario replay → Session orchestration: feeds scripted scene (evidence: `stepwise/replay.py`)
- Frame pipeline → LEGO build state: provides readings
- LEGO build state → World state interface: implements (evidence: `stepwise/state/lego/build_state.py`)
- Jenga lattice → World state interface: implements (evidence: `stepwise/state/jenga/lattice.py`)
- Jenga lattice → Events and alerts: produces move events (evidence: `stepwise/state/jenga/lattice.py`)
- Collapse detection → Jenga lattice: updates collapse state (evidence: `stepwise/state/jenga/collapse.py`)
- Session orchestration → World state interface: reads state changes (evidence: `stepwise/session.py`)
- Session orchestration → Deterministic checker: dispatches events (evidence: `stepwise/session.py`)
- Session orchestration → Events and alerts: collects alerts (evidence: `stepwise/session.py`)
- Deterministic checker → Rule evaluation: evaluates preconditions (evidence: `stepwise/checker/engine.py`)
- Deterministic checker → Event matching: matches events (evidence: `stepwise/checker/engine.py`)
- Event matching → Jenga rules: selects rule policy (evidence: `stepwise/checker/match.py`)
- Rule evaluation → Safe expressions: evaluates expressions (evidence: `stepwise/checker/evaluate.py`)
- Session orchestration → Session log: records session (evidence: `README.md`)
- Scenario replay → Session log: writes session record (evidence: `stepwise/replay.py`)
- Live runner → Session orchestration: runs session (evidence: `stepwise/live.py`)
- Session orchestration → Jigsaw tracking: selects tracker by spec (evidence: `stepwise/session.py`)
- Session log → Session explanation: feeds report (evidence: `README.md`)
- Session orchestration → Step recognizer: uses step hints
- Live runner → Server and web UI: serves live UI