# d-robotics/lerobot architecture

> Architecture diagram of the GitHub repository d-robotics/lerobot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/d-robotics/lerobot
- Repository: https://github.com/d-robotics/lerobot

## Overview

##### Project Type and Purpose
- LeRobot is a modular Python library and CLI toolkit for real‑world robotics research, built on PyTorch and Hugging Face ecosystems.
- It bundles:
– Core API (“lerobot” package) for datasets, environments, policies, and robot‑device drivers
– CLI scripts for training, evaluation, data conversion, and robot control
– Example notebooks, benchmarks, and Docker configurations for CPU/GPU

##### High‑Level Components
A. Core Library (“lerobot” package)
1. configs
- Dataclass configurations for CLI scripts, policies, training, evaluation, environments
2. common
- datasets: LeRobotDataset format, serialization, Hugging Face Datasets integration
- envs: wrappers and factory for Gymnasium‑based simulation (aloha, pusht, xarm)
- policies: plugin implementations (ACT, Diffusion, TDMPC, VQ‑BeT, etc.) + factory loader
- robot_devices: hardware abstractions for cameras and motors (Dynamixel, Feetech, OpenCV)
- utils: logging, I/O, random, benchmark, wandb integration, hub upload
3. scripts
- CLI entrypoints: train.py, eval.py, control_robot.py, push_dataset_to_hub.py, visualize_dataset.py
4. templates
- HTML templates for dataset visualization
B. Examples & Benchmarks
- examples/: step‑by‑step usage tutorials (load data, train, evaluate, real robot control)
- benchmarks/: video and performance measurement scripts
C. Packaging & DevOps
- Docker/: CPU, GPU, dev images
- GitHub Actions workflows for CI (build, test, lint, nightly tests, trufflehog)
- pyproject.toml, Makefile, pre‑commit config for local builds and formatting
D. Tests
- pytest suites covering datasets, envs, policies, robot_devices mock tests, utils

##### External Dependencies & Services
- PyTorch (models, training loops)
- Hugging Face Datasets & Hub (remote dataset/model storage)
- Gymnasium Environments (aloha, xarm, pusht)
- FFmpeg / PyAV for video serialization
- Weights & Biases for experiment logging
- Docker for containerized builds

##### Component Relationships & Data Flow
– Configs → common.datasets to load episodes from HF Hub or local storage.
– common.datasets feeds batched data into common.policies which uses PyTorch models.
– Training/Evaluation loops (in train.py, eval.py) orchestrate data → model → optimizer → metrics → wandb.
– common.envs factory creates simulation environments consumed by policies at runtime.
– control_robot.py uses common.robot_devices to talk to real hardware (motors, cameras), wrapping data into same format as simulation.

##### Key Architectural Patterns
- Factory Pattern: for datasets, environments, policies → allows plug‑and‑play extensions
- Layered CLI Architecture: separation between “presentation” (CLI), “application logic” (scripts + configs), and “domain modules” (common)
- Plugin/Registry: each policy/dataset registers itself in a factory for dynamic loading
- Modular packaging: each concern in its own subpackage for isolation and testability

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Execution Layer
    subgraph "Execution Layer"
        Scripts["lerobot/scripts"]:::cli
        Examples["examples/"]:::cli
        Benchmarks["benchmarks/"]:::cli
    end

    %% Model Layer
    subgraph "Model Layer"
        CoreLib["Core Library"]:::core
        Configs["configs"]:::core
        Datasets["common.datasets"]:::core
        Envs["common.envs"]:::core
        Policies["common.policies"]:::core
        Optim["common.optim"]:::core
        RobotDev["common.robot_devices"]:::core
        Utils["common.utils"]:::core
        Templates["templates"]:::core
    end

    %% Data Layer
    subgraph "Data Layer"
        LocalCache["Local Cache"]:::storage
        HFHub["Hugging Face Hub"]:::storage
    end

    %% Infrastructure
    subgraph "Infrastructure"
        DockerConfig["docker/"]:::infra
        CI_CD[".github/workflows/"]:::infra
        Pyproj["pyproject.toml"]:::infra
        Makefile["Makefile"]:::infra
        Precommit[".pre-commit-config.yaml"]:::infra
        Tests["tests/"]:::infra
    end

    %% External Services
    PyTorch["PyTorch"]:::external
    Gym["Gymnasium"]:::external
    FFmpeg["FFmpeg/PyAV"]:::external
    WnB["Weights & Biases"]:::external
    Hardware["Robot Hardware"]:::external

    %% Relationships
    Scripts -->|"invoke"| Configs
    Scripts -->|"upload dataset"| HFHub
    Scripts -->|"hardware I/O"| RobotDev
    Configs -->|"configure"| Datasets
    Datasets -->|"load"| LocalCache
    Datasets -->|"fetch"| HFHub
    Datasets -->|"provide batches"| Policies
    Policies -->|"train/evaluate"| PyTorch
    Policies -->|"log metrics"| WnB
    Policies -->|"use envs"| Envs
    Envs -->|"simulate"| Gym
    RobotDev -->|"control"| Hardware
    Benchmarks -->|"use scripts"| Scripts
    Examples -->|"tutorials"| Scripts

    %% Click Events
    click CoreLib "https://github.com/d-robotics/lerobot/blob/main/lerobot/__init__.py"
    click Configs "https://github.com/d-robotics/lerobot/tree/main/lerobot/configs/"
    click Datasets "https://github.com/d-robotics/lerobot/tree/main/lerobot/common/datasets/"
    click Envs "https://github.com/d-robotics/lerobot/tree/main/lerobot/common/envs/"
    click Optim "https://github.com/d-robotics/lerobot/tree/main/lerobot/common/optim/"
    click Policies "https://github.com/d-robotics/lerobot/tree/main/lerobot/common/policies/"
    click RobotDev "https://github.com/d-robotics/lerobot/tree/main/lerobot/common/robot_devices/"
    click Utils "https://github.com/d-robotics/lerobot/tree/main/lerobot/common/utils/"
    click Scripts "https://github.com/d-robotics/lerobot/tree/main/lerobot/scripts/"
    click Templates "https://github.com/d-robotics/lerobot/tree/main/lerobot/templates/"
    click Examples "https://github.com/d-robotics/lerobot/tree/main/examples/"
    click Benchmarks "https://github.com/d-robotics/lerobot/tree/main/benchmarks/"
    click DockerConfig "https://github.com/d-robotics/lerobot/tree/main/docker/"
    click CI_CD "https://github.com/d-robotics/lerobot/tree/main/.github/workflows/"
    click Pyproj "https://github.com/d-robotics/lerobot/blob/main/pyproject.toml"
    click Makefile "https://github.com/d-robotics/lerobot/tree/main/Makefile"
    click Precommit "https://github.com/d-robotics/lerobot/blob/main/.pre-commit-config.yaml"
    click Tests "https://github.com/d-robotics/lerobot/tree/main/tests/"

    %% Styles
    classDef core fill:#bbdefb,stroke:#1e88e5
    classDef cli fill:#c8e6c9,stroke:#2e7d32
    classDef storage fill:#fff59d,stroke:#fdd835
    classDef infra fill:#ffe0b2,stroke:#ef6c00
    classDef external fill:#e1bee7,stroke:#8e24aa
```
