How smtlify/lerobot works
LeRobot is a Python robotics library for collecting and managing robot data, training and deploying policies, and evaluating them on simulated or physical robots. Its principal loop takes robot observations, selects policy actions, and sends those actions back to hardware; dataset recording and policy training support that loop. The README documents the product workflow and capability branches. Source excerpts and the SOURCE INDEX clarify selected policy, processor, and dataset internals; other links are omitted where implementation evidence was unavailable.
Main entry points: policies Policy models · robot.py Robot interface · lerobot_dataset.py Dataset access · lerobot_train.py Policy training
Components 28
Robots and control
- Robot interface
src/lerobot/robots/robot.py - Robot drivers
src/lerobot/robots - Teleoperation devices
src/lerobot/teleoperators - Camera drivers
src/lerobot/cameras
Datasets and processing
- Dataset access
src/lerobot/datasets/lerobot_dataset.py - Dataset storage
src/lerobot/datasets/dataset_writer.py - Dataset operations
src/lerobot/datasets/dataset_tools.py - Dataset metadata
src/lerobot/datasets/dataset_metadata.py - Data processors
src/lerobot/processor/pipeline.py - Batch converters
src/lerobot/processor/converters.py - Relative actions
src/lerobot/processor/relative_action_processor.py - Dataset features
src/lerobot/datasets/pipeline_features.py
Policies and rewards
- Policy factory
src/lerobot/policies/factory.py - Policy models
src/lerobot/policies - Pretrained models
src/lerobot/policies/pretrained.py - Pi Gemma
src/lerobot/policies/pi_gemma.py - MolmoAct2
src/lerobot/policies/molmoact2/modeling_molmoact2.py - WALL-X
src/lerobot/policies/wall_x/modeling_wall_x.py - Real-time control
src/lerobot/policies/rtc/modeling_rtc.py - Reward models
src/lerobot/rewards
Training and evaluation
- Policy training
src/lerobot/scripts/lerobot_train.py - Reinforcement learning
src/lerobot/rl/train_rl.py - Simulation environments
src/lerobot/envs - Evaluation
src/lerobot/scripts/lerobot_eval.py
Annotation and inference
- Video annotation
src/lerobot/annotations/steerable_pipeline/executor.py - Async robot client
src/lerobot/async_inference/robot_client.py
Other components
- Robot operator
- Hugging Face Hub
Connections 30
- Robot operator → Robot interface: controls
- Robot interface → Robot drivers: connects through
src/lerobot/robots/lekiwi/lekiwi_client.py - Teleoperation devices → Robot interface: controls
- Camera drivers → Robot interface: provides observations
- Robot interface → Policy models: supplies observations
- Policy models → Robot interface: returns actions
- Robot operator → Dataset access: loads dataset
- Dataset access → Hugging Face Hub: loads from
- Dataset access → Dataset storage: reads dataset
- Dataset operations → Dataset storage: edits dataset
- Dataset access → Dataset metadata: uses metadata
- Data processors → Batch converters: uses converters
src/lerobot/processor/pipeline.py - Data processors → Relative actions: loads steps dynamically
src/lerobot/processor/pipeline.py - Dataset features → Data processors: transforms features
src/lerobot/datasets/pipeline_features.py - Robot operator → Policy training: starts training
- Policy training → Policy factory: selects policy
- Policy training → Dataset access: trains on data
- Policy factory → Policy models: creates policy
src/lerobot/policies/factory.py - Policy models → Pretrained models: extends base
src/lerobot/policies/pi0/modeling_pi0.py - Policy models → Pi Gemma: uses backbone
src/lerobot/policies/pi0/modeling_pi0.py - Policy models → MolmoAct2: includes model
src/lerobot/policies/factory.py - Policy models → WALL-X: includes model
src/lerobot/policies/factory.py - Policy models → Real-time control: uses control module
src/lerobot/policies/pi0/modeling_pi0.py - Robot operator → Evaluation: starts evaluation
- Evaluation → Simulation environments: evaluates in
- Evaluation → Policy models: runs policy
- Robot operator → Video annotation: annotates episodes
- Robot interface → Async robot client: supports remote inference
- Reinforcement learning → Reward models: uses reward models
- Policy training → Hugging Face Hub: shares models