# unitreerobotics/unitree_il_lerobot architecture

> Architecture diagram of the GitHub repository unitreerobotics/unitree_il_lerobot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/unitreerobotics/unitree_il_lerobot
- Repository: https://github.com/unitreerobotics/unitree_il_lerobot

## Overview

##### Main Components
- Data Collection & Conversion:
- Raw Data Input: Represent raw JSON datasets obtained from Unitree robots (e.g., from the avp_teleoperate tool).
- Sorting & Renaming: Include the process executed by the script in unitree_lerobot/utils/sort_and_rename_folders.py.
- Data Conversion: Highlight the conversion process (script: convert_unitree_json_to_lerobot.py) that transforms raw Unitree JSON data into LeRobot’s expected format.

- Training Module (LeRobot Integration):
- LeRobot Framework: Represent that the project builds on the LeRobot framework. Show that this subsystem includes training scripts and a dedicated “lerobot” directory (unitree_lerobot/lerobot).
- Training Scripts: Illustrate training entry points for different policies (act, diffusion, pi0) as separate nodes or branches within the training component.
- Visualization: Reference the visualization script (visualize_dataset.py) which is used for dataset inspection.

- Evaluation / Real-World Inference:
- Evaluation Module: Represent the “eval_robot” directory that contains the evaluation pipeline for real robot inference.
- Sub-components:
▪ Image Server: Represent the image_server module that includes an image_client and image_server, facilitating the visual feedback or camera feed.
▪ Robot Control: Capture the robot_control module noting its responsibility to manage commands sent to the robot actuators (e.g., robot_arm.py, robot_hand_unitree.py).

- External Dependencies:
- LeRobot Framework: Clearly note the dependency on the LeRobot ecosystem and its role in training and evaluation.
- unitree_sdk2_python: Represent the additional dependency for DDS communication needed for real robot interaction.
- External Data Sources: Link to external repositories (e.g., the Hugging Face Unitree dataset and the avp_teleoperate repository).

##### Relationships and Interactions
- Data Flow:
- Converted data is either stored locally or pushed to Hugging Face Hub (indicated in the convert_unitree_json_to_lerobot.py command) before being used in training.
- Once training is complete, the trained model checkpoints are fed into the evaluation module to test them on either real hardware or directly on the dataset.

- Control Flow:
- The training pipeline interacts with the LeRobot framework components while model evaluation uses dedicated run scripts to interact with the robot’s control APIs (DDS through unitree_sdk2_python) and image server for feedback.

##### Architectural Patterns & Design Principles
- Separation of Concerns: Clearly distinguish the modules (data conversion, model training, and evaluation on real hardware). Each directory in the repository corresponds to a logical component.
- Modularity: Illustrate that utility functions (in the utils folder) are decoupled from the main training and evaluation workflows.
- Integration Points: Identify integration with external frameworks (LeRobot and unitree_sdk2_python) and external data repositories (Hugging Face).

##### Relevant Technologies and Frameworks
- LeRobot (for training, dataset handling, and visualization).
- Python and Conda (used for environment and dependency management).
- DDS Communication: Through unitree_sdk2_python for robot control and telemetry.
- External APIs: Hugging Face Hub for dataset hosting and model repository.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Collection & Conversion
    subgraph "Data Collection & Conversion"
        rawData["Raw JSON Data (Unitree)"]:::data
        sortRename["Sorting & Renaming (sort_and_rename_folders.py)"]:::data
        convertJson["JSON Conversion (convert_unitree_json_to_lerobot.py)"]:::data
    end

    %% Training Module (LeRobot Framework)
    subgraph "Training Module (LeRobot Framework)"
        trainingFramework["LeRobot Framework"]:::training
        trainingScripts["Training Scripts (act, diffusion, pi0)"]:::training
        visualization["Visualization (visualize_dataset.py)"]:::training
    end

    %% Evaluation / Real-World Inference
    subgraph "Evaluation / Real-World Inference"
        evalModule["Evaluation Module"]:::evaluation
        evalRealWorld["Real-World Inference (eval_g1.py)"]:::evaluation
        evalDataset["Dataset Evaluation (eval_g1_dataset.py)"]:::evaluation
        evalConfig["Evaluation Config (eval_real_config.py)"]:::evaluation
        imageServer["Image Server (image_client & image_server)"]:::evaluation
        robotControl["Robot Control (robot_arm & robot_hand_unitree.py)"]:::evaluation
    end

    %% External Dependencies
    externalLeRobot["LeRobot Ecosystem"]:::external
    externalDDS["unitree_sdk2_python (DDS)"]:::external
    externalHF["Hugging Face Hub"]:::external

    %% Data Flow Connections
    rawData -->|"processedBy"| sortRename
    sortRename -->|"feedsInto"| convertJson
    convertJson -->|"convertedData"| trainingFramework
    trainingFramework -->|"triggers"| trainingScripts
    trainingFramework --> visualization
    trainingScripts -->|"modelCheckpoints"| evalModule

    %% Evaluation Module internal flow
    evalModule --> evalRealWorld
    evalModule --> evalDataset
    evalModule --> evalConfig
    evalModule --> imageServer
    evalModule --> robotControl

    %% External Dependency Integrations
    trainingFramework -->|"integratesWith"| externalLeRobot
    convertJson -->|"uploadsData"| externalHF
    evalModule -->|"usesDDS"| externalDDS

    %% Click Events
    click sortRename "https://github.com/unitreerobotics/unitree_il_lerobot/blob/main/unitree_lerobot/utils/sort_and_rename_folders.py"
    click convertJson "https://github.com/unitreerobotics/unitree_il_lerobot/blob/main/unitree_lerobot/utils/convert_unitree_json_to_lerobot.py"
    click trainingFramework "https://github.com/unitreerobotics/unitree_il_lerobot/tree/main/unitree_lerobot/lerobot"
    click evalModule "https://github.com/unitreerobotics/unitree_il_lerobot/tree/main/unitree_lerobot/eval_robot"
    click evalRealWorld "https://github.com/unitreerobotics/unitree_il_lerobot/blob/main/unitree_lerobot/eval_robot/eval_g1/eval_g1.py"
    click evalDataset "https://github.com/unitreerobotics/unitree_il_lerobot/blob/main/unitree_lerobot/eval_robot/eval_g1/eval_g1_dataset.py"
    click evalConfig "https://github.com/unitreerobotics/unitree_il_lerobot/blob/main/unitree_lerobot/eval_robot/eval_g1/eval_real_config.py"
    click imageServer "https://github.com/unitreerobotics/unitree_il_lerobot/tree/main/unitree_lerobot/eval_robot/eval_g1/image_server"
    click robotControl "https://github.com/unitreerobotics/unitree_il_lerobot/tree/main/unitree_lerobot/eval_robot/eval_g1/robot_control"

    %% Styles
    classDef data fill:#f9f,stroke:#333,stroke-width:2px;
    classDef training fill:#9f9,stroke:#333,stroke-width:2px;
    classDef evaluation fill:#ff9,stroke:#333,stroke-width:2px;
    classDef external fill:#9ff,stroke:#333,stroke-width:2px;
```
