How nishika-chauhan/second-look works
Second Look combines a browser console, a belief-and-planning service, and a MuJoCo robot simulator. The principal workflow is to choose a target and scenario, start a retrieval episode, update location belief from observations and events, choose a look/pick/help action, and report the result to the interface. Separate simulation and data-generation capabilities support scenario calibration, recovery demonstrations, and synthetic perception datasets. Edges follow shown imports/calls or README descriptions; unsampled modules are represented only where their role is documented or indexed.
Main entry points: App.jsx App screens · world.py MuJoCo world · server.py Episode API · RobotContext.jsx Robot state
Components 23
Robot interface
- App screens
second-look/second-look/src/App.jsx - Robot state
second-look/second-look/src/RobotContext.jsx - Guidance UI
second-look/second-look/src/context/GuidanceContext.jsx - Playground
second-look/second-look/src/screens/Playground.jsx - Analytics
second-look/second-look/src/screens/Analytics.jsx - Memory view
second-look/second-look/src/screens/Memory.jsx
Memory and planning
- Episode API
ai/server.py - Action planner
ai/planner.py - Region belief
ai/regions_belief.py - Perception
ai/perception.py - Event watcher
ai/event_watcher.py - Nemotron client
ai/nemotron_client.py
Simulation
- Simulator API
sim/simulator_service.py - MuJoCo world
sim/world.py - Scripted expert
sim/expert.py - Scenario engine
sim/scenarios.py - Episode rollout
ai/rollout.py
Synthetic data
- LeRobot writer
sim/dataset_backend.py - LeRobot export
data/export_lerobot_v3.py - Dataset audit
data/validate_lerobot_v3.py - YOLO export
sim/export_yolo.py
Other components
- Operator
- Nebius service
Connections 29
- Operator → Playground: configures task
- App screens → Playground: renders
second-look/second-look/src/App.jsx - App screens → Analytics: renders
second-look/second-look/src/App.jsx - App screens → Memory view: renders
second-look/second-look/src/App.jsx - App screens → Robot state: reads state
second-look/second-look/src/App.jsx - App screens → Guidance UI: provides guidance
second-look/second-look/src/App.jsx - Playground → Robot state: dispatches actions
second-look/second-look/src/RobotContext.jsx - Memory view → Robot state: reads state
second-look/second-look/src/screens/Memory.jsx - Guidance UI → Operator: reports status
second-look/second-look/src/context/GuidanceContext.jsx - Robot state → Episode API: sends commands
ai/server.py - Episode API → Action planner: requests decisions
ai/server.py - Episode API → Event watcher: watches events
ai/server.py - Episode API → Region belief: updates belief
ai/server.py - Action planner → Region belief: uses probabilities
ai/planner.py - Action planner → Nemotron client: requests fallback
ai/planner.py - Nemotron client → Nebius service: calls inference
ai/nemotron_client.py - Episode API → Simulator API: routes actions
ai/server.py - Simulator API → MuJoCo world: controls world
sim/simulator_service.py - Simulator API → Scripted expert: runs policy
sim/simulator_service.py - Scripted expert → MuJoCo world: acts in
sim/expert.py - Scenario engine → MuJoCo world: configures scenarios
sim/scenarios.py - Episode rollout → MuJoCo world: steps episodes
ai/rollout.py - Episode rollout → Scripted expert: runs policy
ai/rollout.py - LeRobot export → MuJoCo world: records episodes
data/export_lerobot_v3.py - LeRobot export → Scripted expert: runs demonstrations
data/export_lerobot_v3.py - LeRobot export → LeRobot writer: writes trajectories
- Dataset audit → LeRobot writer: audits dataset
- YOLO export → MuJoCo world: exports images
README.md - Perception → Region belief: informs belief
ai/perception.py