# huggingface/lerobot architecture

> Architecture diagram of the GitHub repository huggingface/lerobot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/huggingface/lerobot
- Repository: https://github.com/huggingface/lerobot

## Overview

LeRobot is a Python robotics library for connecting hardware, collecting and managing datasets, training and running learned policies, and deploying control—including remote inference. The main user workflow is to configure a robot and policy, collect or load robot observations, process them into policy inputs, infer actions, and dispatch them to hardware; datasets also support training and evaluation. The graph combines README-documented capabilities with relationships visible in sampled excerpts and the SOURCE INDEX. Some subsystem internals and wiring were not sampled, so uncertain links are omitted.

Main entry points: [`src/lerobot/datasets/lerobot_dataset.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/datasets/lerobot_dataset.py) (Dataset access), [`src/lerobot/rollout/context.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/rollout/context.py) (Rollout control), [`src/lerobot/robots/robot.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/robots/robot.py) (Robot interfaces), [`src/lerobot/scripts/lerobot_train.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/lerobot_train.py) (Policy training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_hardware["Robotics hardware"]
  node_robot["Robot interfaces<br/>[robot.py]"]
  node_teleop["Teleoperation devices"]
  node_cameras["Camera drivers"]
end

subgraph group_data["Data and annotation"]
  node_dataset[("Dataset access<br/>[lerobot_dataset.py]")]
  node_dataset_meta["Dataset metadata"]
  node_dataset_storage["Dataset storage<br/>[storage.py]"]
  node_annotation["Video annotation"]
end

subgraph group_learning["Policies and learning"]
  node_policy_factory["Policy selection<br/>[factory.py]"]
  node_policy_models["Policy models"]
  node_training["Policy training<br/>[lerobot_train.py]"]
  node_rl["Reinforcement learning<br/>[train_rl.py]"]
  node_environments["Robot environments<br/>[factory.py]"]
  node_rewards["Reward models<br/>[factory.py]"]
end

subgraph group_runtime["Inference and control"]
  node_processor["Processor pipelines<br/>[pipeline.py]"]
  node_inference["Inference engines<br/>[base.py]"]
  node_rollout["Rollout control<br/>[context.py]"]
  node_remote_client["Remote client<br/>[client.py]"]
  node_remote_engine["Remote engine<br/>[engine.py]"]
  node_remote_server["Inference server<br/>[server.py]"]
end

subgraph group_platform["Configuration and services"]
  node_configs["Configuration<br/>[default.py]"]
  node_protocol["Wire protocol<br/>[protocol.py]"]
end

node_user(("Robot operator"))
node_hub[("Hugging Face Hub")]

node_user -->|"configures"| node_configs
node_user -->|"operates"| node_robot
node_teleop -.->|"controls"| node_robot
node_cameras -.->|"provides images"| node_robot
node_robot -->|"records data"| node_dataset
node_dataset -->|"loads metadata"| node_dataset_meta
node_dataset_storage -->|"reads metadata"| node_dataset_meta
node_dataset -->|"loads datasets"| node_hub
node_dataset -->|"supplies data"| node_training
node_dataset -.->|"annotates episodes"| node_annotation
node_user -->|"trains policy"| node_training
node_training -->|"trains"| node_policy_models
node_policy_factory -.->|"selects model"| node_policy_models
node_user -->|"starts rollout"| node_rollout
node_rollout -->|"connects hardware"| node_robot
node_rollout -->|"creates engine"| node_inference
node_rollout -->|"loads policy"| node_policy_factory
node_rollout -->|"configures pipeline"| node_processor
node_inference -.->|"runs policy"| node_policy_models
node_processor -.->|"formats inputs"| node_policy_models
node_inference -.->|"dispatches actions"| node_robot
node_rl -.->|"trains in"| node_environments
node_rl -.->|"uses rewards"| node_rewards
node_remote_client -->|"encodes messages"| node_protocol
node_remote_server -->|"handles messages"| node_protocol
node_remote_server -->|"predicts actions"| node_inference
node_remote_engine -->|"queries client"| node_remote_client
node_rollout -.->|"runs remotely"| node_remote_engine
node_dataset -->|"publishes datasets"| node_hub
node_training -.->|"publishes checkpoints"| node_hub

click node_configs "https://github.com/huggingface/lerobot/blob/main/src/lerobot/configs/default.py"
click node_robot "https://github.com/huggingface/lerobot/blob/main/src/lerobot/robots/robot.py"
click node_teleop "https://github.com/huggingface/lerobot/tree/main/src/lerobot/teleoperators"
click node_cameras "https://github.com/huggingface/lerobot/tree/main/src/lerobot/cameras"
click node_processor "https://github.com/huggingface/lerobot/blob/main/src/lerobot/processor/pipeline.py"
click node_dataset "https://github.com/huggingface/lerobot/blob/main/src/lerobot/datasets/lerobot_dataset.py"
click node_dataset_meta "https://github.com/huggingface/lerobot/blob/main/src/lerobot/datasets/dataset_metadata.py"
click node_dataset_storage "https://github.com/huggingface/lerobot/blob/main/src/lerobot/datasets/storage.py"
click node_annotation "https://github.com/huggingface/lerobot/blob/main/src/lerobot/annotations/steerable_pipeline/modules/plan_subtasks_memory.py"
click node_policy_factory "https://github.com/huggingface/lerobot/blob/main/src/lerobot/policies/factory.py"
click node_policy_models "https://github.com/huggingface/lerobot/tree/main/src/lerobot/policies"
click node_training "https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/lerobot_train.py"
click node_rl "https://github.com/huggingface/lerobot/blob/main/src/lerobot/rl/train_rl.py"
click node_inference "https://github.com/huggingface/lerobot/blob/main/src/lerobot/inference/base.py"
click node_rollout "https://github.com/huggingface/lerobot/blob/main/src/lerobot/rollout/context.py"
click node_remote_client "https://github.com/huggingface/lerobot/blob/main/src/lerobot/remote_inference/client.py"
click node_remote_engine "https://github.com/huggingface/lerobot/blob/main/src/lerobot/remote_inference/engine.py"
click node_remote_server "https://github.com/huggingface/lerobot/blob/main/src/lerobot/remote_inference/server.py"
click node_protocol "https://github.com/huggingface/lerobot/blob/main/src/lerobot/remote_inference/protocol.py"
click node_environments "https://github.com/huggingface/lerobot/blob/main/src/lerobot/envs/factory.py"
click node_rewards "https://github.com/huggingface/lerobot/blob/main/src/lerobot/rewards/factory.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_robot,node_teleop,node_cameras toneBlue
class node_dataset,node_dataset_meta,node_dataset_storage,node_annotation,node_hub toneAmber
class node_policy_factory,node_policy_models,node_training,node_rl,node_environments,node_rewards toneMint
class node_processor,node_inference,node_rollout,node_remote_client,node_remote_engine,node_remote_server toneRose
class node_configs,node_protocol,node_user toneIndigo
```

## Components

### Robotics hardware

- [Robot interfaces](https://github.com/huggingface/lerobot/blob/main/src/lerobot/robots/robot.py): `src/lerobot/robots/robot.py`
- [Teleoperation devices](https://github.com/huggingface/lerobot/tree/main/src/lerobot/teleoperators): `src/lerobot/teleoperators`
- [Camera drivers](https://github.com/huggingface/lerobot/tree/main/src/lerobot/cameras): `src/lerobot/cameras`

### Data and annotation

- [Dataset access](https://github.com/huggingface/lerobot/blob/main/src/lerobot/datasets/lerobot_dataset.py): `src/lerobot/datasets/lerobot_dataset.py`
- [Dataset metadata](https://github.com/huggingface/lerobot/blob/main/src/lerobot/datasets/dataset_metadata.py): `src/lerobot/datasets/dataset_metadata.py`
- [Dataset storage](https://github.com/huggingface/lerobot/blob/main/src/lerobot/datasets/storage.py): `src/lerobot/datasets/storage.py`
- [Video annotation](https://github.com/huggingface/lerobot/blob/main/src/lerobot/annotations/steerable_pipeline/modules/plan_subtasks_memory.py): `src/lerobot/annotations/steerable_pipeline/modules/plan_subtasks_memory.py`

### Policies and learning

- [Policy selection](https://github.com/huggingface/lerobot/blob/main/src/lerobot/policies/factory.py): `src/lerobot/policies/factory.py`
- [Policy models](https://github.com/huggingface/lerobot/tree/main/src/lerobot/policies): `src/lerobot/policies`
- [Policy training](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/lerobot_train.py): `src/lerobot/scripts/lerobot_train.py`
- [Reinforcement learning](https://github.com/huggingface/lerobot/blob/main/src/lerobot/rl/train_rl.py): `src/lerobot/rl/train_rl.py`
- [Robot environments](https://github.com/huggingface/lerobot/blob/main/src/lerobot/envs/factory.py): `src/lerobot/envs/factory.py`
- [Reward models](https://github.com/huggingface/lerobot/blob/main/src/lerobot/rewards/factory.py): `src/lerobot/rewards/factory.py`

### Inference and control

- [Processor pipelines](https://github.com/huggingface/lerobot/blob/main/src/lerobot/processor/pipeline.py): `src/lerobot/processor/pipeline.py`
- [Inference engines](https://github.com/huggingface/lerobot/blob/main/src/lerobot/inference/base.py): `src/lerobot/inference/base.py`
- [Rollout control](https://github.com/huggingface/lerobot/blob/main/src/lerobot/rollout/context.py): `src/lerobot/rollout/context.py`
- [Remote client](https://github.com/huggingface/lerobot/blob/main/src/lerobot/remote_inference/client.py): `src/lerobot/remote_inference/client.py`
- [Remote engine](https://github.com/huggingface/lerobot/blob/main/src/lerobot/remote_inference/engine.py): `src/lerobot/remote_inference/engine.py`
- [Inference server](https://github.com/huggingface/lerobot/blob/main/src/lerobot/remote_inference/server.py): `src/lerobot/remote_inference/server.py`

### Configuration and services

- [Configuration](https://github.com/huggingface/lerobot/blob/main/src/lerobot/configs/default.py): `src/lerobot/configs/default.py`
- [Wire protocol](https://github.com/huggingface/lerobot/blob/main/src/lerobot/remote_inference/protocol.py): `src/lerobot/remote_inference/protocol.py`

### Other components

- **Robot operator**
- **Hugging Face Hub**

## Connections

- Robot operator → Configuration: configures (evidence: `README.md`)
- Robot operator → Robot interfaces: operates (evidence: `README.md`)
- Teleoperation devices → Robot interfaces: controls
- Camera drivers → Robot interfaces: provides images
- Robot interfaces → Dataset access: records data (evidence: `README.md`)
- Dataset access → Dataset metadata: loads metadata
- Dataset storage → Dataset metadata: reads metadata (evidence: `src/lerobot/datasets/storage.py`)
- Dataset access → Hugging Face Hub: loads datasets (evidence: `README.md`)
- Dataset access → Policy training: supplies data (evidence: `README.md`)
- Dataset access → Video annotation: annotates episodes
- Robot operator → Policy training: trains policy (evidence: `README.md`)
- Policy training → Policy models: trains (evidence: `README.md`)
- Policy selection → Policy models: selects model
- Robot operator → Rollout control: starts rollout (evidence: `README.md`)
- Rollout control → Robot interfaces: connects hardware (evidence: `src/lerobot/rollout/context.py`)
- Rollout control → Inference engines: creates engine (evidence: `src/lerobot/rollout/context.py`)
- Rollout control → Policy selection: loads policy (evidence: `src/lerobot/rollout/context.py`)
- Rollout control → Processor pipelines: configures pipeline (evidence: `src/lerobot/rollout/context.py`)
- Inference engines → Policy models: runs policy
- Processor pipelines → Policy models: formats inputs
- Inference engines → Robot interfaces: dispatches actions
- Reinforcement learning → Robot environments: trains in
- Reinforcement learning → Reward models: uses rewards
- Remote client → Wire protocol: encodes messages (evidence: `src/lerobot/remote_inference/client.py`)
- Inference server → Wire protocol: handles messages (evidence: `src/lerobot/remote_inference/server.py`)
- Inference server → Inference engines: predicts actions (evidence: `src/lerobot/remote_inference/server.py`)
- Remote engine → Remote client: queries client (evidence: `src/lerobot/remote_inference/engine.py`)
- Rollout control → Remote engine: runs remotely
- Dataset access → Hugging Face Hub: publishes datasets (evidence: `README.md`)
- Policy training → Hugging Face Hub: publishes checkpoints
