# smtlify/lerobot architecture

> Architecture diagram of the GitHub repository smtlify/lerobot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/smtlify/lerobot
- Repository: https://github.com/smtlify/lerobot

## Overview

LeRobot is a Python robotics library for collecting and managing robot data, training and deploying policies, and evaluating them on simulated or physical robots. Its principal loop takes robot observations, selects policy actions, and sends those actions back to hardware; dataset recording and policy training support that loop. The README documents the product workflow and capability branches. Source excerpts and the SOURCE INDEX clarify selected policy, processor, and dataset internals; other links are omitted where implementation evidence was unavailable.

Main entry points: [`src/lerobot/policies`](https://github.com/smtlify/lerobot/tree/main/src/lerobot/policies) (Policy models), [`src/lerobot/robots/robot.py`](https://github.com/smtlify/lerobot/blob/main/src/lerobot/robots/robot.py) (Robot interface), [`src/lerobot/datasets/lerobot_dataset.py`](https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/lerobot_dataset.py) (Dataset access), [`src/lerobot/scripts/lerobot_train.py`](https://github.com/smtlify/lerobot/blob/main/src/lerobot/scripts/lerobot_train.py) (Policy training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_hardware["Robots and control"]
  node_robot["Robot interface<br/>[robot.py]"]
  node_robot_drivers["Robot drivers"]
  node_teleop["Teleoperation devices"]
  node_cameras["Camera drivers"]
end

subgraph group_data["Datasets and processing"]
  node_dataset["Dataset access<br/>[lerobot_dataset.py]"]
  node_dataset_store[("Dataset storage<br/>[dataset_writer.py]")]
  node_dataset_tools["Dataset operations<br/>[dataset_tools.py]"]
  node_metadata["Dataset metadata"]
  node_processors["Data processors<br/>[pipeline.py]"]
  node_converters["Batch converters<br/>[converters.py]"]
  node_relative_actions["Relative actions"]
  node_features["Dataset features"]
end

subgraph group_models["Policies and rewards"]
  node_policy_factory["Policy factory<br/>[factory.py]"]
  node_policy["Policy models"]
  node_pretrained["Pretrained models<br/>[pretrained.py]"]
  node_pi_gemma["Pi Gemma<br/>[pi_gemma.py]"]
  node_molmo["MolmoAct2"]
  node_wallx["WALL-X<br/>[modeling_wall_x.py]"]
  node_rtc["Real-time control<br/>[modeling_rtc.py]"]
  node_reward["Reward models"]
end

subgraph group_learning["Training and evaluation"]
  node_train["Policy training<br/>[lerobot_train.py]"]
  node_rl["Reinforcement learning<br/>[train_rl.py]"]
  node_environments["Simulation environments"]
  node_evaluation["Evaluation<br/>[lerobot_eval.py]"]
end

subgraph group_tools["Annotation and inference"]
  node_annotation["Video annotation<br/>[executor.py]"]
  node_async_client["Async robot client<br/>[robot_client.py]"]
end

node_operator(("Robot operator"))
node_hub(("Hugging Face Hub"))

node_operator -->|"controls"| node_robot
node_robot -.->|"connects through"| node_robot_drivers
node_teleop -.->|"controls"| node_robot
node_cameras -.->|"provides observations"| node_robot
node_robot -->|"supplies observations"| node_policy
node_policy -->|"returns actions"| node_robot
node_operator -->|"loads dataset"| node_dataset
node_dataset -->|"loads from"| node_hub
node_dataset -.->|"reads dataset"| node_dataset_store
node_dataset_tools -->|"edits dataset"| node_dataset_store
node_dataset -.->|"uses metadata"| node_metadata
node_processors -->|"uses converters"| node_converters
node_processors -->|"loads steps dynamically"| node_relative_actions
node_features -->|"transforms features"| node_processors
node_operator -->|"starts training"| node_train
node_train -.->|"selects policy"| node_policy_factory
node_train -->|"trains on data"| node_dataset
node_policy_factory -->|"creates policy"| node_policy
node_policy -->|"extends base"| node_pretrained
node_policy -->|"uses backbone"| node_pi_gemma
node_policy -.->|"includes model"| node_molmo
node_policy -.->|"includes model"| node_wallx
node_policy -->|"uses control module"| node_rtc
node_operator -->|"starts evaluation"| node_evaluation
node_evaluation -->|"evaluates in"| node_environments
node_evaluation -.->|"runs policy"| node_policy
node_operator -.->|"annotates episodes"| node_annotation
node_robot -.->|"supports remote inference"| node_async_client
node_rl -.->|"uses reward models"| node_reward
node_train -->|"shares models"| node_hub

click node_robot "https://github.com/smtlify/lerobot/blob/main/src/lerobot/robots/robot.py"
click node_robot_drivers "https://github.com/smtlify/lerobot/tree/main/src/lerobot/robots"
click node_teleop "https://github.com/smtlify/lerobot/tree/main/src/lerobot/teleoperators"
click node_cameras "https://github.com/smtlify/lerobot/tree/main/src/lerobot/cameras"
click node_dataset "https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/lerobot_dataset.py"
click node_dataset_store "https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/dataset_writer.py"
click node_dataset_tools "https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/dataset_tools.py"
click node_metadata "https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/dataset_metadata.py"
click node_processors "https://github.com/smtlify/lerobot/blob/main/src/lerobot/processor/pipeline.py"
click node_converters "https://github.com/smtlify/lerobot/blob/main/src/lerobot/processor/converters.py"
click node_relative_actions "https://github.com/smtlify/lerobot/blob/main/src/lerobot/processor/relative_action_processor.py"
click node_features "https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/pipeline_features.py"
click node_policy_factory "https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/factory.py"
click node_policy "https://github.com/smtlify/lerobot/tree/main/src/lerobot/policies"
click node_pretrained "https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/pretrained.py"
click node_pi_gemma "https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/pi_gemma.py"
click node_molmo "https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/molmoact2/modeling_molmoact2.py"
click node_wallx "https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/wall_x/modeling_wall_x.py"
click node_rtc "https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/rtc/modeling_rtc.py"
click node_reward "https://github.com/smtlify/lerobot/tree/main/src/lerobot/rewards"
click node_train "https://github.com/smtlify/lerobot/blob/main/src/lerobot/scripts/lerobot_train.py"
click node_rl "https://github.com/smtlify/lerobot/blob/main/src/lerobot/rl/train_rl.py"
click node_environments "https://github.com/smtlify/lerobot/tree/main/src/lerobot/envs"
click node_evaluation "https://github.com/smtlify/lerobot/blob/main/src/lerobot/scripts/lerobot_eval.py"
click node_annotation "https://github.com/smtlify/lerobot/blob/main/src/lerobot/annotations/steerable_pipeline/executor.py"
click node_async_client "https://github.com/smtlify/lerobot/blob/main/src/lerobot/async_inference/robot_client.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_robot,node_robot_drivers,node_teleop,node_cameras toneBlue
class node_dataset,node_dataset_store,node_dataset_tools,node_metadata,node_processors,node_converters,node_relative_actions,node_features toneAmber
class node_policy_factory,node_policy,node_pretrained,node_pi_gemma,node_molmo,node_wallx,node_rtc,node_reward toneMint
class node_train,node_rl,node_environments,node_evaluation toneRose
class node_annotation,node_async_client,node_operator,node_hub toneIndigo
```

## Components

### Robots and control

- [Robot interface](https://github.com/smtlify/lerobot/blob/main/src/lerobot/robots/robot.py): `src/lerobot/robots/robot.py`
- [Robot drivers](https://github.com/smtlify/lerobot/tree/main/src/lerobot/robots): `src/lerobot/robots`
- [Teleoperation devices](https://github.com/smtlify/lerobot/tree/main/src/lerobot/teleoperators): `src/lerobot/teleoperators`
- [Camera drivers](https://github.com/smtlify/lerobot/tree/main/src/lerobot/cameras): `src/lerobot/cameras`

### Datasets and processing

- [Dataset access](https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/lerobot_dataset.py): `src/lerobot/datasets/lerobot_dataset.py`
- [Dataset storage](https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/dataset_writer.py): `src/lerobot/datasets/dataset_writer.py`
- [Dataset operations](https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/dataset_tools.py): `src/lerobot/datasets/dataset_tools.py`
- [Dataset metadata](https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/dataset_metadata.py): `src/lerobot/datasets/dataset_metadata.py`
- [Data processors](https://github.com/smtlify/lerobot/blob/main/src/lerobot/processor/pipeline.py): `src/lerobot/processor/pipeline.py`
- [Batch converters](https://github.com/smtlify/lerobot/blob/main/src/lerobot/processor/converters.py): `src/lerobot/processor/converters.py`
- [Relative actions](https://github.com/smtlify/lerobot/blob/main/src/lerobot/processor/relative_action_processor.py): `src/lerobot/processor/relative_action_processor.py`
- [Dataset features](https://github.com/smtlify/lerobot/blob/main/src/lerobot/datasets/pipeline_features.py): `src/lerobot/datasets/pipeline_features.py`

### Policies and rewards

- [Policy factory](https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/factory.py): `src/lerobot/policies/factory.py`
- [Policy models](https://github.com/smtlify/lerobot/tree/main/src/lerobot/policies): `src/lerobot/policies`
- [Pretrained models](https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/pretrained.py): `src/lerobot/policies/pretrained.py`
- [Pi Gemma](https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/pi_gemma.py): `src/lerobot/policies/pi_gemma.py`
- [MolmoAct2](https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/molmoact2/modeling_molmoact2.py): `src/lerobot/policies/molmoact2/modeling_molmoact2.py`
- [WALL-X](https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/wall_x/modeling_wall_x.py): `src/lerobot/policies/wall_x/modeling_wall_x.py`
- [Real-time control](https://github.com/smtlify/lerobot/blob/main/src/lerobot/policies/rtc/modeling_rtc.py): `src/lerobot/policies/rtc/modeling_rtc.py`
- [Reward models](https://github.com/smtlify/lerobot/tree/main/src/lerobot/rewards): `src/lerobot/rewards`

### Training and evaluation

- [Policy training](https://github.com/smtlify/lerobot/blob/main/src/lerobot/scripts/lerobot_train.py): `src/lerobot/scripts/lerobot_train.py`
- [Reinforcement learning](https://github.com/smtlify/lerobot/blob/main/src/lerobot/rl/train_rl.py): `src/lerobot/rl/train_rl.py`
- [Simulation environments](https://github.com/smtlify/lerobot/tree/main/src/lerobot/envs): `src/lerobot/envs`
- [Evaluation](https://github.com/smtlify/lerobot/blob/main/src/lerobot/scripts/lerobot_eval.py): `src/lerobot/scripts/lerobot_eval.py`

### Annotation and inference

- [Video annotation](https://github.com/smtlify/lerobot/blob/main/src/lerobot/annotations/steerable_pipeline/executor.py): `src/lerobot/annotations/steerable_pipeline/executor.py`
- [Async robot client](https://github.com/smtlify/lerobot/blob/main/src/lerobot/async_inference/robot_client.py): `src/lerobot/async_inference/robot_client.py`

### Other components

- **Robot operator**
- **Hugging Face Hub**

## Connections

- Robot operator → Robot interface: controls
- Robot interface → Robot drivers: connects through (evidence: `src/lerobot/robots/lekiwi/lekiwi_client.py`)
- Teleoperation devices → Robot interface: controls
- Camera drivers → Robot interface: provides observations
- Robot interface → Policy models: supplies observations
- Policy models → Robot interface: returns actions
- Robot operator → Dataset access: loads dataset
- Dataset access → Hugging Face Hub: loads from
- Dataset access → Dataset storage: reads dataset
- Dataset operations → Dataset storage: edits dataset
- Dataset access → Dataset metadata: uses metadata
- Data processors → Batch converters: uses converters (evidence: `src/lerobot/processor/pipeline.py`)
- Data processors → Relative actions: loads steps dynamically (evidence: `src/lerobot/processor/pipeline.py`)
- Dataset features → Data processors: transforms features (evidence: `src/lerobot/datasets/pipeline_features.py`)
- Robot operator → Policy training: starts training
- Policy training → Policy factory: selects policy
- Policy training → Dataset access: trains on data
- Policy factory → Policy models: creates policy (evidence: `src/lerobot/policies/factory.py`)
- Policy models → Pretrained models: extends base (evidence: `src/lerobot/policies/pi0/modeling_pi0.py`)
- Policy models → Pi Gemma: uses backbone (evidence: `src/lerobot/policies/pi0/modeling_pi0.py`)
- Policy models → MolmoAct2: includes model (evidence: `src/lerobot/policies/factory.py`)
- Policy models → WALL-X: includes model (evidence: `src/lerobot/policies/factory.py`)
- Policy models → Real-time control: uses control module (evidence: `src/lerobot/policies/pi0/modeling_pi0.py`)
- Robot operator → Evaluation: starts evaluation
- Evaluation → Simulation environments: evaluates in
- Evaluation → Policy models: runs policy
- Robot operator → Video annotation: annotates episodes
- Robot interface → Async robot client: supports remote inference
- Reinforcement learning → Reward models: uses reward models
- Policy training → Hugging Face Hub: shares models
