# spirit-ai-team/lerobot architecture

> Architecture diagram of the GitHub repository spirit-ai-team/lerobot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/spirit-ai-team/lerobot
- Repository: https://github.com/spirit-ai-team/lerobot

## Overview

##### Project Type and Purpose
- LeRobot is a robotics framework/library designed for training, evaluating, and deploying policies on both simulation environments and real robots.
- It is a full-stack solution in the robotics domain, integrating datasets, pretrained policies, simulation environments (using Gymnasium), and live control of actual robot devices.
- The README highlights examples of running simulations (e.g., ALOHA, TDMPC, Diffusion Policy) and real-world applications (e.g., SO-100, LeKiwi).

##### File Structure for Architectural Components
- Top-level directories like “lerobot”, “examples”, “benchmarks”, “docker”, “tests” indicate modular organization.
- The “lerobot” directory is the core package:
- Contains subdirectories such as “configs” (for configuration management), “common” (for datasets, environments, policies, robot devices, and utilities), “scripts” (executable command-line tools for training, evaluation, visualization, etc.) and “templates” (for HTML visualizations).
- The “benchmarks” and “examples” directories provide use-case scripts and demonstrations.
- Docker-related directories detail containerization for CPU and GPU variants, indicating deployment aspects.
- The tests directory ensures continuous integration and unit testing of individual modules.

##### Main Components
a. Core Library (lerobot package)
- Configurations: “lerobot/configs” managing default, training, evaluation, and policy-specific options.
- Datasets and Data Handling: “lerobot/common/datasets” that organize human demonstration datasets, including support for multiple dataset formats (e.g., hdf5, safetensors, zarr).
- Simulation Environments: “lerobot/common/envs” which sets up environments for Gymnasium-based simulation.
- Policies: “lerobot/common/policies” with submodules for different policy types (ACT, Diffusion, TDMPC, VQ-BeT, etc.) that encapsulate model configuration and inference logic.
- Robot Devices: “lerobot/common/robot_devices” to interface with real hardware (motors, cameras, etc.).
- Utilities: “lerobot/common/utils” providing support functions for logging, data I/O, randomness, and benchmark utilities.
b. Executable Scripts
- Training Script (lerobot/scripts/train.py)
- Evaluation Script (lerobot/scripts/eval.py)
- Control Script for real robots (lerobot/scripts/control_robot.py)
- Data Visualization and Upload Tools (visualize_dataset.py, push_dataset_to_hub.py)
c. External Dependencies & Services
- Integration with Hugging Face Hub for datasets and pretrained policies.
- WandB for logging and experiment tracking.
- Gymnasium environments and Docker for simulated/real-world execution.
d. Docker & Deployment
- The “docker” directory indicates containerization support for different configurations (e.g., GPU, CPU, GPU dev).
e. Examples and Benchmarks
- Serve as “client” examples showing how to use the LeRobot API to load datasets, train policies, and control robots.

##### Relationships and Data Flow
- Data/Configuration Ingestion: Users start by loading configurations (via “lerobot/configs”) or datasets (using LeRobotDataset from “lerobot/common/datasets”) from local or Hugging Face hub sources.
- Training/Evaluation Flow: The training script (train.py) interacts with policies, datasets, and environment modules while using the configuration modules to set hyperparameters. Evaluation script (eval.py) similarly calls policies and environment modules.
- Real-World Execution: The control scripts take commands from a user and interface with robot devices (hardware interfacing via “lerobot/common/robot_devices”).
- Experiment Tracking: Integration with WandB is used during training/evaluation, represented as an external logging service.
- Docker Deployment and CI: The Docker and GitHub workflows (inside .github/workflows) ensure reproducible builds and tests.

##### Architectural Patterns and Design Principles
- Modular Separation of Concerns: Each module (configs, datasets, policies, environments, devices, etc.) is distinctly separated, making the architecture easy to extend or swap components.
- Pipeline-based Flow: Data is ingested, processed through simulation or real-robot control pathways, and then results (checkpoints, logs, videos) are produced.
- Flexibility and Extensibility: The design supports multiple types of policies and datasets and enables easy addition of new simulations or real device interfaces.
- Clear Dependency Injection: Scripts use configurations to abstract hardware details and policy parameters.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Library Section
    subgraph "Core Library"
        LR["LeRobot Package"]:::core
        CONFIG["Configurations"]:::core
        DATA["Datasets & Data Handling"]:::core
        ENVS["Simulation Environments"]:::core
        POL["Policies"]:::core
        ROBOT["Robot Devices"]:::core
        UTILS["Utilities"]:::core
    end

    %% Executable Scripts Section
    subgraph "Executable Scripts"
        TRAIN["Training Script (train.py)"]:::script
        EVAL["Evaluation Script (eval.py)"]:::script
        CONTROL["Real Robot Control Script (control_robot.py)"]:::script
        VIS1["Data Visualization (visualize_dataset.py)"]:::script
        VIS2["Data Visualization HTML (visualize_dataset_html.py)"]:::script
    end

    %% External Services Section
    subgraph "External Services"
        HF["HuggingFace Hub"]:::external
        WANDB["WandB"]:::external
        GYM["Gymnasium"]:::external
    end

    %% Docker & Deployment Section
    subgraph "Docker Deployment"
        DOCKER_CPU["CPU Container (lerobot-cpu)"]:::docker
        DOCKER_GPU["GPU Container (lerobot-gpu)"]:::docker
        DOCKER_GPU_DEV["GPU Dev Container (lerobot-gpu-dev)"]:::docker
    end

    %% Client Applications Section
    subgraph "Client Applications"
        EXAMPLES["Examples"]:::client
        BENCH["Benchmarks"]:::client
    end

    %% Relationships and Data Flow
    %% Core Library flows
    LR -->|"includes"| CONFIG
    LR -->|"includes"| DATA
    LR -->|"includes"| ENVS
    LR -->|"includes"| POL
    LR -->|"includes"| ROBOT
    LR -->|"includes"| UTILS

    %% Scripts consuming configurations, datasets, environments, and policies
    CONFIG -->|"configures"| TRAIN
    DATA -->|"provides data"| TRAIN
    POL -->|"policy logic"| TRAIN
    ENVS -->|"simulation setup"| TRAIN

    DATA -->|"provides data"| EVAL
    POL -->|"policy inference"| EVAL
    ENVS -->|"simulation setup"| EVAL

    ROBOT -->|"hardware interfacing"| CONTROL
    POL -->|"policy commands"| CONTROL

    DATA -->|"data for visualization"| VIS1
    DATA -->|"data for visualization"| VIS2

    %% External services integration
    HF -->|"sources data/policies"| DATA
    HF -->|"provides pretrained models"| POL

    TRAIN -->|"logs experiments"| WANDB
    EVAL -->|"logs experiments"| WANDB

    GYM -->|"defines API"| ENVS

    %% Docker deployment linking to the core library execution
    DOCKER_CPU ---|"containerizes"| LR
    DOCKER_GPU ---|"containerizes"| LR
    DOCKER_GPU_DEV ---|"containerizes"| LR

    %% Client applications usage of the core library
    EXAMPLES -->|"demonstrates usage"| LR
    BENCH -->|"performance demos"| LR

    %% Styles
    classDef core fill:#AED6F1,stroke:#1f618d,stroke-width:2px;
    classDef script fill:#ABEBC6,stroke:#239B56,stroke-width:2px;
    classDef external fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef docker fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
    classDef client fill:#D7BDE2,stroke:#8E44AD,stroke-width:2px;

    %% Click Events for Core Library Components
    click LR "https://github.com/spirit-ai-team/lerobot/tree/main/lerobot/"
    click CONFIG "https://github.com/spirit-ai-team/lerobot/tree/main/lerobot/configs/"
    click DATA "https://github.com/spirit-ai-team/lerobot/tree/main/lerobot/common/datasets/"
    click ENVS "https://github.com/spirit-ai-team/lerobot/tree/main/lerobot/common/envs/"
    click POL "https://github.com/spirit-ai-team/lerobot/tree/main/lerobot/common/policies/"
    click ROBOT "https://github.com/spirit-ai-team/lerobot/tree/main/lerobot/common/robot_devices/"
    click UTILS "https://github.com/spirit-ai-team/lerobot/tree/main/lerobot/common/utils/"

    %% Click Events for Executable Scripts
    click TRAIN "https://github.com/spirit-ai-team/lerobot/blob/main/lerobot/scripts/train.py"
    click EVAL "https://github.com/spirit-ai-team/lerobot/blob/main/lerobot/scripts/eval.py"
    click CONTROL "https://github.com/spirit-ai-team/lerobot/blob/main/lerobot/scripts/control_robot.py"
    click VIS1 "https://github.com/spirit-ai-team/lerobot/blob/main/lerobot/scripts/visualize_dataset.py"
    click VIS2 "https://github.com/spirit-ai-team/lerobot/blob/main/lerobot/scripts/visualize_dataset_html.py"

    %% Click Events for Docker & Deployment
    click DOCKER_CPU "https://github.com/spirit-ai-team/lerobot/tree/main/docker/lerobot-cpu"
    click DOCKER_GPU "https://github.com/spirit-ai-team/lerobot/tree/main/docker/lerobot-gpu"
    click DOCKER_GPU_DEV "https://github.com/spirit-ai-team/lerobot/tree/main/docker/lerobot-gpu-dev"

    %% Click Events for Client Applications
    click EXAMPLES "https://github.com/spirit-ai-team/lerobot/tree/main/examples/"
    click BENCH "https://github.com/spirit-ai-team/lerobot/tree/main/benchmarks/"
```
