How huggingface/lerobot works
LeRobot is a Python robotics library for connecting hardware, collecting and managing datasets, training and running learned policies, and deploying control—including remote inference. The main user workflow is to configure a robot and policy, collect or load robot observations, process them into policy inputs, infer actions, and dispatch them to hardware; datasets also support training and evaluation. The graph combines README-documented capabilities with relationships visible in sampled excerpts and the SOURCE INDEX. Some subsystem internals and wiring were not sampled, so uncertain links are omitted.
Main entry points: lerobot_dataset.py Dataset access · context.py Rollout control · robot.py Robot interfaces · lerobot_train.py Policy training
Components 23
Robotics hardware
- Robot interfaces
src/lerobot/robots/robot.py - Teleoperation devices
src/lerobot/teleoperators - Camera drivers
src/lerobot/cameras
Data and annotation
- Dataset access
src/lerobot/datasets/lerobot_dataset.py - Dataset metadata
src/lerobot/datasets/dataset_metadata.py - Dataset storage
src/lerobot/datasets/storage.py - Video annotation
src/lerobot/annotations/steerable_pipeline/modules/plan_subtasks_memory.py
Policies and learning
- Policy selection
src/lerobot/policies/factory.py - Policy models
src/lerobot/policies - Policy training
src/lerobot/scripts/lerobot_train.py - Reinforcement learning
src/lerobot/rl/train_rl.py - Robot environments
src/lerobot/envs/factory.py - Reward models
src/lerobot/rewards/factory.py
Inference and control
- Processor pipelines
src/lerobot/processor/pipeline.py - Inference engines
src/lerobot/inference/base.py - Rollout control
src/lerobot/rollout/context.py - Remote client
src/lerobot/remote_inference/client.py - Remote engine
src/lerobot/remote_inference/engine.py - Inference server
src/lerobot/remote_inference/server.py
Configuration and services
- Configuration
src/lerobot/configs/default.py - Wire protocol
src/lerobot/remote_inference/protocol.py
Other components
- Robot operator
- Hugging Face Hub
Connections 30
- Robot operator → Configuration: configures
README.md - Robot operator → Robot interfaces: operates
README.md - Teleoperation devices → Robot interfaces: controls
- Camera drivers → Robot interfaces: provides images
- Robot interfaces → Dataset access: records data
README.md - Dataset access → Dataset metadata: loads metadata
- Dataset storage → Dataset metadata: reads metadata
src/lerobot/datasets/storage.py - Dataset access → Hugging Face Hub: loads datasets
README.md - Dataset access → Policy training: supplies data
README.md - Dataset access → Video annotation: annotates episodes
- Robot operator → Policy training: trains policy
README.md - Policy training → Policy models: trains
README.md - Policy selection → Policy models: selects model
- Robot operator → Rollout control: starts rollout
README.md - Rollout control → Robot interfaces: connects hardware
src/lerobot/rollout/context.py - Rollout control → Inference engines: creates engine
src/lerobot/rollout/context.py - Rollout control → Policy selection: loads policy
src/lerobot/rollout/context.py - Rollout control → Processor pipelines: configures pipeline
src/lerobot/rollout/context.py - Inference engines → Policy models: runs policy
- Processor pipelines → Policy models: formats inputs
- Inference engines → Robot interfaces: dispatches actions
- Reinforcement learning → Robot environments: trains in
- Reinforcement learning → Reward models: uses rewards
- Remote client → Wire protocol: encodes messages
src/lerobot/remote_inference/client.py - Inference server → Wire protocol: handles messages
src/lerobot/remote_inference/server.py - Inference server → Inference engines: predicts actions
src/lerobot/remote_inference/server.py - Remote engine → Remote client: queries client
src/lerobot/remote_inference/engine.py - Rollout control → Remote engine: runs remotely
- Dataset access → Hugging Face Hub: publishes datasets
README.md - Policy training → Hugging Face Hub: publishes checkpoints