How aaditpani-rvu/stepwise works
STEPWISE monitors procedures from camera observations and judges detected actions against a compiled procedure specification, producing alerts and session records. The documented primary path is manual-to-spec compilation offline, then camera-to-world-state tracking and deterministic checking online; replay exercises the tracker and checker without hardware. Jenga and LEGO state implementations are represented where source evidence supports them. README also identifies Jigsaw, live reporting, temporal recognition and server/UI as capabilities or planned branches, but their implementation details were not sampled, so their connections are left unverified.
Main entry points: session.py Session orchestration · pipeline.py Frame pipeline · engine.py Deterministic checker · live.py Live runner
Components 29
Procedure specs
- LLM compiler
stepwise/compiler/compile.py - Spec schema
stepwise/compiler/schema.py - Spec validation
stepwise/compiler/validate.py - Safe expressions
stepwise/compiler/expr.py - Procedure spec
Perception
- Calibration
stepwise/perception/calibrate.py - Hand tracking
stepwise/perception/hands.py - Part detection
stepwise/perception/detect.py - Frame pipeline
stepwise/perception/pipeline.py
World state
- Stud grid geometry
stepwise/state/lego/grid.py - LEGO build state
stepwise/state/lego/build_state.py - Jenga lattice
stepwise/state/jenga/lattice.py - Collapse detection
stepwise/state/jenga/collapse.py - World state interface
stepwise/state/base.py - Jigsaw tracking
stepwise/state
Judgment and records
- Events and alerts
stepwise/events.py - Session orchestration
stepwise/session.py - Deterministic checker
stepwise/checker/engine.py - Rule evaluation
stepwise/checker/evaluate.py - Jenga rules
stepwise/checker/rules_jenga.py - Event matching
stepwise/checker/match.py - Session log
stepwise/sessionlog.py - Scenario replay
stepwise/replay.py - Live runner
stepwise/live.py - Step recognizer
stepwise/temporal - Session explanation
stepwise/explain - Server and web UI
stepwise/server
Other components
- Procedure manual
- Camera frames
Connections 32
- Procedure manual → LLM compiler: compiles text
README.md - LLM compiler → Spec schema: builds spec
stepwise/compiler/compile.py - LLM compiler → Spec validation: validates spec
stepwise/compiler/compile.py - Spec validation → Safe expressions: checks expressions
stepwise/compiler/validate.py - Spec validation → Procedure spec: accepts valid spec
README.md - Procedure spec → Deterministic checker: judged against
README.md - Camera frames → Frame pipeline: supplies frames
stepwise/perception/pipeline.py - Frame pipeline → Calibration: calibrates frame
stepwise/perception/pipeline.py - Frame pipeline → Hand tracking: detects hands
stepwise/perception/pipeline.py - Frame pipeline → Part detection: detects parts
stepwise/perception/pipeline.py - Frame pipeline → Stud grid geometry: maps readings
stepwise/perception/pipeline.py - Live runner → Frame pipeline: runs frames through
stepwise/live.py - Scenario replay → Session orchestration: feeds scripted scene
stepwise/replay.py - Frame pipeline → LEGO build state: provides readings
- LEGO build state → World state interface: implements
stepwise/state/lego/build_state.py - Jenga lattice → World state interface: implements
stepwise/state/jenga/lattice.py - Jenga lattice → Events and alerts: produces move events
stepwise/state/jenga/lattice.py - Collapse detection → Jenga lattice: updates collapse state
stepwise/state/jenga/collapse.py - Session orchestration → World state interface: reads state changes
stepwise/session.py - Session orchestration → Deterministic checker: dispatches events
stepwise/session.py - Session orchestration → Events and alerts: collects alerts
stepwise/session.py - Deterministic checker → Rule evaluation: evaluates preconditions
stepwise/checker/engine.py - Deterministic checker → Event matching: matches events
stepwise/checker/engine.py - Event matching → Jenga rules: selects rule policy
stepwise/checker/match.py - Rule evaluation → Safe expressions: evaluates expressions
stepwise/checker/evaluate.py - Session orchestration → Session log: records session
README.md - Scenario replay → Session log: writes session record
stepwise/replay.py - Live runner → Session orchestration: runs session
stepwise/live.py - Session orchestration → Jigsaw tracking: selects tracker by spec
stepwise/session.py - Session log → Session explanation: feeds report
README.md - Session orchestration → Step recognizer: uses step hints
- Live runner → Server and web UI: serves live UI