# nishika-chauhan/second-look architecture

> Architecture diagram of the GitHub repository nishika-chauhan/second-look, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nishika-chauhan/second-look
- Repository: https://github.com/nishika-chauhan/second-look

## Overview

Second Look combines a browser console, a belief-and-planning service, and a MuJoCo robot simulator. The principal workflow is to choose a target and scenario, start a retrieval episode, update location belief from observations and events, choose a look/pick/help action, and report the result to the interface. Separate simulation and data-generation capabilities support scenario calibration, recovery demonstrations, and synthetic perception datasets. Edges follow shown imports/calls or README descriptions; unsampled modules are represented only where their role is documented or indexed.

Main entry points: [`second-look/second-look/src/App.jsx`](https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/App.jsx) (App screens), [`sim/world.py`](https://github.com/nishika-chauhan/second-look/blob/main/sim/world.py) (MuJoCo world), [`ai/server.py`](https://github.com/nishika-chauhan/second-look/blob/main/ai/server.py) (Episode API), [`second-look/second-look/src/RobotContext.jsx`](https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/RobotContext.jsx) (Robot state)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["Robot interface"]
  node_app["App screens<br/>[App.jsx]"]
  node_robot_context["Robot state<br/>[RobotContext.jsx]"]
  node_guidance["Guidance UI"]
  node_playground["Playground<br/>[Playground.jsx]"]
  node_analytics["Analytics<br/>[Analytics.jsx]"]
  node_memory_screen["Memory view<br/>[Memory.jsx]"]
end

subgraph group_reasoning["Memory and planning"]
  node_server["Episode API<br/>[server.py]"]
  node_planner["Action planner<br/>[planner.py]"]
  node_belief["Region belief<br/>[regions_belief.py]"]
  node_perception["Perception<br/>[perception.py]"]
  node_event_watcher["Event watcher<br/>[event_watcher.py]"]
  node_nemotron["Nemotron client<br/>[nemotron_client.py]"]
end

subgraph group_simulation["Simulation"]
  node_sim_service["Simulator API"]
  node_world["MuJoCo world<br/>[world.py]"]
  node_expert["Scripted expert<br/>[expert.py]"]
  node_scenarios["Scenario engine<br/>[scenarios.py]"]
  node_rollout["Episode rollout<br/>[rollout.py]"]
end

subgraph group_data["Synthetic data"]
  node_dataset_writer[("LeRobot writer<br/>[dataset_backend.py]")]
  node_dataset_export["LeRobot export"]
  node_dataset_qa["Dataset audit"]
  node_yolo_export["YOLO export<br/>[export_yolo.py]"]
end

node_user(("Operator"))
node_nebius["Nebius service"]

node_user -->|"configures task"| node_playground
node_app -->|"renders"| node_playground
node_app -->|"renders"| node_analytics
node_app -->|"renders"| node_memory_screen
node_app -->|"reads state"| node_robot_context
node_app -->|"provides guidance"| node_guidance
node_playground -->|"dispatches actions"| node_robot_context
node_memory_screen -->|"reads state"| node_robot_context
node_guidance -->|"reports status"| node_user
node_robot_context -.->|"sends commands"| node_server
node_server -->|"requests decisions"| node_planner
node_server -->|"watches events"| node_event_watcher
node_server -->|"updates belief"| node_belief
node_planner -->|"uses probabilities"| node_belief
node_planner -->|"requests fallback"| node_nemotron
node_nemotron -->|"calls inference"| node_nebius
node_server -->|"routes actions"| node_sim_service
node_sim_service -->|"controls world"| node_world
node_sim_service -->|"runs policy"| node_expert
node_expert -->|"acts in"| node_world
node_scenarios -->|"configures scenarios"| node_world
node_rollout -->|"steps episodes"| node_world
node_rollout -->|"runs policy"| node_expert
node_dataset_export -->|"records episodes"| node_world
node_dataset_export -->|"runs demonstrations"| node_expert
node_dataset_export -.->|"writes trajectories"| node_dataset_writer
node_dataset_qa -.->|"audits dataset"| node_dataset_writer
node_yolo_export -.->|"exports images"| node_world
node_perception -->|"informs belief"| node_belief

click node_app "https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/App.jsx"
click node_robot_context "https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/RobotContext.jsx"
click node_guidance "https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/context/GuidanceContext.jsx"
click node_playground "https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/screens/Playground.jsx"
click node_analytics "https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/screens/Analytics.jsx"
click node_memory_screen "https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/screens/Memory.jsx"
click node_server "https://github.com/nishika-chauhan/second-look/blob/main/ai/server.py"
click node_planner "https://github.com/nishika-chauhan/second-look/blob/main/ai/planner.py"
click node_belief "https://github.com/nishika-chauhan/second-look/blob/main/ai/regions_belief.py"
click node_perception "https://github.com/nishika-chauhan/second-look/blob/main/ai/perception.py"
click node_event_watcher "https://github.com/nishika-chauhan/second-look/blob/main/ai/event_watcher.py"
click node_nemotron "https://github.com/nishika-chauhan/second-look/blob/main/ai/nemotron_client.py"
click node_sim_service "https://github.com/nishika-chauhan/second-look/blob/main/sim/simulator_service.py"
click node_world "https://github.com/nishika-chauhan/second-look/blob/main/sim/world.py"
click node_expert "https://github.com/nishika-chauhan/second-look/blob/main/sim/expert.py"
click node_scenarios "https://github.com/nishika-chauhan/second-look/blob/main/sim/scenarios.py"
click node_rollout "https://github.com/nishika-chauhan/second-look/blob/main/ai/rollout.py"
click node_dataset_writer "https://github.com/nishika-chauhan/second-look/blob/main/sim/dataset_backend.py"
click node_dataset_export "https://github.com/nishika-chauhan/second-look/blob/main/data/export_lerobot_v3.py"
click node_dataset_qa "https://github.com/nishika-chauhan/second-look/blob/main/data/validate_lerobot_v3.py"
click node_yolo_export "https://github.com/nishika-chauhan/second-look/blob/main/sim/export_yolo.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_robot_context,node_guidance,node_playground,node_analytics,node_memory_screen toneBlue
class node_server,node_planner,node_belief,node_perception,node_event_watcher,node_nemotron toneAmber
class node_sim_service,node_world,node_expert,node_scenarios,node_rollout toneMint
class node_dataset_writer,node_dataset_export,node_dataset_qa,node_yolo_export toneRose
class node_user,node_nebius toneIndigo
```

## Components

### Robot interface

- [App screens](https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/App.jsx): `second-look/second-look/src/App.jsx`
- [Robot state](https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/RobotContext.jsx): `second-look/second-look/src/RobotContext.jsx`
- [Guidance UI](https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/context/GuidanceContext.jsx): `second-look/second-look/src/context/GuidanceContext.jsx`
- [Playground](https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/screens/Playground.jsx): `second-look/second-look/src/screens/Playground.jsx`
- [Analytics](https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/screens/Analytics.jsx): `second-look/second-look/src/screens/Analytics.jsx`
- [Memory view](https://github.com/nishika-chauhan/second-look/blob/main/second-look/second-look/src/screens/Memory.jsx): `second-look/second-look/src/screens/Memory.jsx`

### Memory and planning

- [Episode API](https://github.com/nishika-chauhan/second-look/blob/main/ai/server.py): `ai/server.py`
- [Action planner](https://github.com/nishika-chauhan/second-look/blob/main/ai/planner.py): `ai/planner.py`
- [Region belief](https://github.com/nishika-chauhan/second-look/blob/main/ai/regions_belief.py): `ai/regions_belief.py`
- [Perception](https://github.com/nishika-chauhan/second-look/blob/main/ai/perception.py): `ai/perception.py`
- [Event watcher](https://github.com/nishika-chauhan/second-look/blob/main/ai/event_watcher.py): `ai/event_watcher.py`
- [Nemotron client](https://github.com/nishika-chauhan/second-look/blob/main/ai/nemotron_client.py): `ai/nemotron_client.py`

### Simulation

- [Simulator API](https://github.com/nishika-chauhan/second-look/blob/main/sim/simulator_service.py): `sim/simulator_service.py`
- [MuJoCo world](https://github.com/nishika-chauhan/second-look/blob/main/sim/world.py): `sim/world.py`
- [Scripted expert](https://github.com/nishika-chauhan/second-look/blob/main/sim/expert.py): `sim/expert.py`
- [Scenario engine](https://github.com/nishika-chauhan/second-look/blob/main/sim/scenarios.py): `sim/scenarios.py`
- [Episode rollout](https://github.com/nishika-chauhan/second-look/blob/main/ai/rollout.py): `ai/rollout.py`

### Synthetic data

- [LeRobot writer](https://github.com/nishika-chauhan/second-look/blob/main/sim/dataset_backend.py): `sim/dataset_backend.py`
- [LeRobot export](https://github.com/nishika-chauhan/second-look/blob/main/data/export_lerobot_v3.py): `data/export_lerobot_v3.py`
- [Dataset audit](https://github.com/nishika-chauhan/second-look/blob/main/data/validate_lerobot_v3.py): `data/validate_lerobot_v3.py`
- [YOLO export](https://github.com/nishika-chauhan/second-look/blob/main/sim/export_yolo.py): `sim/export_yolo.py`

### Other components

- **Operator**
- **Nebius service**

## Connections

- Operator → Playground: configures task
- App screens → Playground: renders (evidence: `second-look/second-look/src/App.jsx`)
- App screens → Analytics: renders (evidence: `second-look/second-look/src/App.jsx`)
- App screens → Memory view: renders (evidence: `second-look/second-look/src/App.jsx`)
- App screens → Robot state: reads state (evidence: `second-look/second-look/src/App.jsx`)
- App screens → Guidance UI: provides guidance (evidence: `second-look/second-look/src/App.jsx`)
- Playground → Robot state: dispatches actions (evidence: `second-look/second-look/src/RobotContext.jsx`)
- Memory view → Robot state: reads state (evidence: `second-look/second-look/src/screens/Memory.jsx`)
- Guidance UI → Operator: reports status (evidence: `second-look/second-look/src/context/GuidanceContext.jsx`)
- Robot state → Episode API: sends commands (evidence: `ai/server.py`)
- Episode API → Action planner: requests decisions (evidence: `ai/server.py`)
- Episode API → Event watcher: watches events (evidence: `ai/server.py`)
- Episode API → Region belief: updates belief (evidence: `ai/server.py`)
- Action planner → Region belief: uses probabilities (evidence: `ai/planner.py`)
- Action planner → Nemotron client: requests fallback (evidence: `ai/planner.py`)
- Nemotron client → Nebius service: calls inference (evidence: `ai/nemotron_client.py`)
- Episode API → Simulator API: routes actions (evidence: `ai/server.py`)
- Simulator API → MuJoCo world: controls world (evidence: `sim/simulator_service.py`)
- Simulator API → Scripted expert: runs policy (evidence: `sim/simulator_service.py`)
- Scripted expert → MuJoCo world: acts in (evidence: `sim/expert.py`)
- Scenario engine → MuJoCo world: configures scenarios (evidence: `sim/scenarios.py`)
- Episode rollout → MuJoCo world: steps episodes (evidence: `ai/rollout.py`)
- Episode rollout → Scripted expert: runs policy (evidence: `ai/rollout.py`)
- LeRobot export → MuJoCo world: records episodes (evidence: `data/export_lerobot_v3.py`)
- LeRobot export → Scripted expert: runs demonstrations (evidence: `data/export_lerobot_v3.py`)
- LeRobot export → LeRobot writer: writes trajectories
- Dataset audit → LeRobot writer: audits dataset
- YOLO export → MuJoCo world: exports images (evidence: `README.md`)
- Perception → Region belief: informs belief (evidence: `ai/perception.py`)