# crane-1/ros2-isaacsim-lerobot-dataset-collection architecture

> Architecture diagram of the GitHub repository crane-1/ros2-isaacsim-lerobot-dataset-collection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/crane-1/ros2-isaacsim-lerobot-dataset-collection
- Repository: https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection

## Overview

This repository supports a dual-arm object-transfer workflow: Isaac Sim scenes and robot assets provide the simulation context; ROS 2 scripts collect camera, joint-state, and action episodes into LeRobot-style datasets; dataset utilities convert and summarize those datasets; LeRobot fine-tuning and ACT/SmolVLA ROS inference complete the training-and-evaluation loop. Collection and inference details are based on sampled scripts; training is described by the README but its implementation was not sampled.

Main entry points: [`src/data_collection/scripts`](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/src/data_collection/scripts) (Episode Dataset), [`lerobot_related`](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/lerobot_related) (LeRobot Fine-tuning), [`src/data_collection/scripts/data_collection.py`](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/src/data_collection/scripts/data_collection.py) (Single-Arm Collector), [`lerobot_related/eval_act.py`](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/eval_act.py) (ACT ROS Inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_simulation["Simulation Assets"]
  node_scenes["Robot Scenes"]
  node_robot_models["Piper Models"]
end

subgraph group_collection["ROS Data Collection"]
  node_collection_single["Single-Arm Collector<br/>[data_collection.py]"]
  node_collection_dual["Dual-Arm Collector<br/>[dual_collection.py]"]
end

subgraph group_dataset["Dataset Processing"]
  node_episode_store[("Episode Dataset")]
  node_episode_metadata["Episode Metadata"]
  node_episode_stats["Dataset Statistics"]
  node_dataset_convert["Dataset Converter<br/>[convert.py]"]
end

subgraph group_models["Training and Inference"]
  node_lerobot_training["LeRobot Fine-tuning"]
  node_act_eval["ACT ROS Inference<br/>[eval_act.py]"]
  node_act_dual_eval["Dual ACT Inference<br/>[eval_act_dual.py]"]
  node_smol_eval["SmolVLA ROS Inference<br/>[eval_smolvla.py]"]
  node_smol_dual_eval["Dual SmolVLA Inference<br/>[smol_vla_dual.py]"]
end

subgraph group_results["Evaluation Results"]
  node_metrics_act["ACT Evaluation Artifacts"]
  node_metrics_smol["SmolVLA Evaluation Artifacts"]
end

node_operator(("Task Operator"))
node_isaac["Isaac Sim"]
node_robot_control["Robot / Simulator Control"]

node_operator -->|"starts recording"| node_collection_single
node_operator -->|"starts recording"| node_collection_dual
node_isaac -->|"loads assets"| node_scenes
node_isaac -->|"loads assets"| node_robot_models
node_isaac -->|"publishes ROS data"| node_collection_single
node_isaac -->|"publishes ROS data"| node_collection_dual
node_collection_single -->|"publishes joint states"| node_robot_control
node_collection_single -->|"records episodes"| node_episode_store
node_collection_dual -->|"records episodes"| node_episode_store
node_episode_store -->|"reads episodes"| node_episode_metadata
node_episode_store -->|"reads episodes"| node_episode_stats
node_episode_store -->|"converts dataset"| node_dataset_convert
node_episode_store -.->|"trains from data"| node_lerobot_training
node_dataset_convert -.->|"provides dataset"| node_lerobot_training
node_lerobot_training -.->|"provides checkpoint"| node_act_eval
node_lerobot_training -.->|"provides checkpoint"| node_act_dual_eval
node_lerobot_training -.->|"provides checkpoint"| node_smol_eval
node_lerobot_training -.->|"provides checkpoint"| node_smol_dual_eval
node_isaac -->|"publishes observations"| node_act_eval
node_isaac -->|"publishes observations"| node_act_dual_eval
node_isaac -->|"publishes observations"| node_smol_eval
node_isaac -->|"publishes observations"| node_smol_dual_eval
node_act_eval -->|"publishes actions"| node_robot_control
node_act_dual_eval -->|"publishes actions"| node_robot_control
node_smol_eval -->|"publishes actions"| node_robot_control
node_smol_dual_eval -->|"publishes actions"| node_robot_control
node_act_eval -.->|"evaluated in"| node_metrics_act
node_act_dual_eval -.->|"evaluated in"| node_metrics_act
node_smol_eval -.->|"evaluated in"| node_metrics_smol
node_smol_dual_eval -.->|"evaluated in"| node_metrics_smol

click node_scenes "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/Grid"
click node_robot_models "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/USD"
click node_collection_single "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/src/data_collection/scripts/data_collection.py"
click node_collection_dual "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/src/data_collection/scripts/dual_collection.py"
click node_episode_store "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/src/data_collection/scripts"
click node_episode_metadata "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/create_episodes_jsonl.py"
click node_episode_stats "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/create_episodes_stats_jsonl.py"
click node_dataset_convert "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/convert.py"
click node_lerobot_training "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/lerobot_related"
click node_act_eval "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/eval_act.py"
click node_act_dual_eval "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/eval_act_dual.py"
click node_smol_eval "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/eval_smolvla.py"
click node_smol_dual_eval "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/smol_vla_dual.py"
click node_metrics_act "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/pics/act_result"
click node_metrics_smol "https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/pics/smol_vla_result"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scenes,node_robot_models toneBlue
class node_collection_single,node_collection_dual toneAmber
class node_episode_store,node_episode_metadata,node_episode_stats,node_dataset_convert toneMint
class node_lerobot_training,node_act_eval,node_act_dual_eval,node_smol_eval,node_smol_dual_eval,node_operator toneRose
class node_metrics_act,node_metrics_smol,node_isaac,node_robot_control toneIndigo
```

## Components

### Simulation Assets

- [Robot Scenes](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/Grid): `Grid`
- [Piper Models](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/USD): `USD`

### ROS Data Collection

- [Single-Arm Collector](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/src/data_collection/scripts/data_collection.py): `src/data_collection/scripts/data_collection.py`
- [Dual-Arm Collector](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/src/data_collection/scripts/dual_collection.py): `src/data_collection/scripts/dual_collection.py`

### Dataset Processing

- [Episode Dataset](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/src/data_collection/scripts): `src/data_collection/scripts`
- [Episode Metadata](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/create_episodes_jsonl.py): `lerobot_related/create_episodes_jsonl.py`
- [Dataset Statistics](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/create_episodes_stats_jsonl.py): `lerobot_related/create_episodes_stats_jsonl.py`
- [Dataset Converter](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/convert.py): `lerobot_related/convert.py`

### Training and Inference

- [LeRobot Fine-tuning](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/lerobot_related): `lerobot_related`
- [ACT ROS Inference](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/eval_act.py): `lerobot_related/eval_act.py`
- [Dual ACT Inference](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/eval_act_dual.py): `lerobot_related/eval_act_dual.py`
- [SmolVLA ROS Inference](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/eval_smolvla.py): `lerobot_related/eval_smolvla.py`
- [Dual SmolVLA Inference](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/blob/main/lerobot_related/smol_vla_dual.py): `lerobot_related/smol_vla_dual.py`

### Evaluation Results

- [ACT Evaluation Artifacts](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/pics/act_result): `pics/act_result`
- [SmolVLA Evaluation Artifacts](https://github.com/crane-1/ros2-isaacsim-lerobot-dataset-collection/tree/main/pics/smol_vla_result): `pics/smol_vla_result`

### Other components

- **Task Operator**
- **Isaac Sim**
- **Robot / Simulator Control**

## Connections

- Task Operator → Single-Arm Collector: starts recording
- Task Operator → Dual-Arm Collector: starts recording
- Isaac Sim → Robot Scenes: loads assets
- Isaac Sim → Piper Models: loads assets
- Isaac Sim → Single-Arm Collector: publishes ROS data
- Isaac Sim → Dual-Arm Collector: publishes ROS data
- Single-Arm Collector → Robot / Simulator Control: publishes joint states
- Single-Arm Collector → Episode Dataset: records episodes
- Dual-Arm Collector → Episode Dataset: records episodes
- Episode Dataset → Episode Metadata: reads episodes
- Episode Dataset → Dataset Statistics: reads episodes
- Episode Dataset → Dataset Converter: converts dataset
- Episode Dataset → LeRobot Fine-tuning: trains from data
- Dataset Converter → LeRobot Fine-tuning: provides dataset
- LeRobot Fine-tuning → ACT ROS Inference: provides checkpoint
- LeRobot Fine-tuning → Dual ACT Inference: provides checkpoint
- LeRobot Fine-tuning → SmolVLA ROS Inference: provides checkpoint
- LeRobot Fine-tuning → Dual SmolVLA Inference: provides checkpoint
- Isaac Sim → ACT ROS Inference: publishes observations
- Isaac Sim → Dual ACT Inference: publishes observations
- Isaac Sim → SmolVLA ROS Inference: publishes observations
- Isaac Sim → Dual SmolVLA Inference: publishes observations
- ACT ROS Inference → Robot / Simulator Control: publishes actions
- Dual ACT Inference → Robot / Simulator Control: publishes actions
- SmolVLA ROS Inference → Robot / Simulator Control: publishes actions
- Dual SmolVLA Inference → Robot / Simulator Control: publishes actions
- ACT ROS Inference → ACT Evaluation Artifacts: evaluated in
- Dual ACT Inference → ACT Evaluation Artifacts: evaluated in
- SmolVLA ROS Inference → SmolVLA Evaluation Artifacts: evaluated in
- Dual SmolVLA Inference → SmolVLA Evaluation Artifacts: evaluated in