# nvlabs/hover architecture

> Architecture diagram of the GitHub repository nvlabs/hover, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvlabs/hover
- Repository: https://github.com/nvlabs/hover

## Overview

##### Main Components
- Data & Motion Library:
- Motion datasets. Include “AMASS Dataset” and retargeted data (e.g., stable_punch.pkl). Note the role of the third_party/human2humanoid scripts that process the raw motion data.
- Simulation Environment:
- Isaac Lab, Mujoco, and their accompanying wrappers. The “neural_wbc/inference_env”, “neural_wbc/mujoco_wrapper”, and “neural_wbc/isaac_lab_wrapper” modules indicate distinct simulation and evaluation pipelines.
- Core Control and Evaluation:
- The “neural_wbc/core” folder contains functionalities like body state, environment wrapper, evaluator, mask definitions, and math utilities. This core module implements the underlying control/physics logic.
- Policy Modules:
- Teacher Policy (training script, model checkpoint storage, configuration in “scripts/rsl_rl/train_teacher_policy.py”).
- Student Policy (training and policy trainer, with associated configuration and storage).
- Hardware and Real-World Integration:
- Hardware Wrappers (“neural_wbc/hw_wrappers”) that interface with the Unitree H1 robot.
- External Libraries and Third-Party Tools:
- The third_party directory shows inclusion of customized RSL RL algorithms (PPO, actor-critic modules) and additional viewers that integrate with Mujoco.
- Utilities and Testing:
- Incorporate testing modules, build scripts (e.g., install_deps.sh), and configuration overwrite capabilities. These support development and unit testing.

##### Relationships and Data Flow
- Data Flow Sequence:
- The process begins with raw motion capture data (from AMASS), which undergoes retargeting via the provided scripts (retarget_h1.sh and associated YAML configuration).
- This processed data feeds into the simulation environments where policies are trained.
- Training Pipeline:
- In one branch, the Teacher Policy is trained first, storing checkpoints (displayed in neural_wbc/data/data/policy/h1:teacher/).
- Then the Student Policy is trained using the Teacher Policy’s checkpoint (“neural_wbc/student_policy”) representing a policy distillation flow.
- Simulation & Evaluation:
- Training and evaluation flow through Isaac Lab and Mujoco simulation environments.
- Inference modules (e.g., neural_wbc/inference_env) provide means for both sim-to-sim and sim-to-real testing.
- Hardware Integration:
- The hw_wrappers module (within neural_wbc/hw_wrappers) directly interfaces with the real robot, converting simulation commands to hardware commands.

##### Architectural Patterns and Design Principles
- Modularity and Separation of Concerns:
- The project is split into dedicated modules (core, data, policy, wrappers, and inference), each encapsulating a logical unit of functionality.
- Extensibility:
- The architecture supports adding new motion modes (generalist vs. specialist), experimenting with new simulation environments (Mujoco, Isaac Lab), and deploying across different hardware platforms.
- Testing and Deployment:
- Unit tests and integration tests are in place, as seen in the tests directories in each module, ensuring reliable integration of components.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Preprocessing Subgraph
    subgraph "Data & Preprocessing"
        MD["Motion Datasets"]:::data
        R1["retarget_h1.sh"]:::data
        R2["human2humanoid Scripts"]:::data
        RP["Retargeted Motion Data"]:::data
    end

    %% Simulation Environment Subgraph
    subgraph "Simulation Environment"
        IL["Isaac Lab Simulation"]:::simulation
        MJ["Mujoco Simulation"]:::simulation
        IE["Inference Environment"]:::simulation
    end

    %% Core Processing Subgraph
    subgraph "Core Processing"
        CORE["Core Control & Evaluation"]:::core
    end

    %% Policy Modules Subgraph
    subgraph "Policy Modules"
        TP["Teacher Policy"]:::policy
        SP["Student Policy"]:::policy
    end

    %% Hardware Integration Subgraph
    subgraph "Hardware Integration"
        HW["Unitree H1 Hardware Interface"]:::hardware
    end

    %% Third-Party Tools Subgraph
    subgraph "Third-Party Tools"
        RSL["RSL RL Algorithms"]:::thirdparty
        MJV["Mujoco Viewer"]:::thirdparty
    end

    %% Utilities & Testing Subgraph
    subgraph "Utilities & Testing"
        UT["Testing & Utilities"]:::utilities
    end

    %% Connections within Data & Preprocessing
    MD --> R1
    MD --> R2
    R1 --> RP
    R2 --> RP

    %% Data to Simulation Environment
    RP --> IL
    RP --> MJ

    %% Simulation Environment to Core Processing
    IL --> CORE
    MJ --> CORE

    %% Core Processing to Policy Modules and Inference
    CORE --> TP
    TP --> SP
    CORE --> IE

    %% Inference Environment to Hardware Integration
    IE --> HW

    %% Third-Party Tools influencing Simulation and Policy
    RSL -.-> TP
    MJV -.-> MJ

    %% Utilities supporting Core, Policy and Hardware
    UT -.-> CORE
    UT -.-> TP
    UT -.-> HW

    %% Click Events
    click MD "https://github.com/nvlabs/hover/tree/main/neural_wbc/data/data/motion_lib"
    click R1 "https://github.com/nvlabs/hover/blob/main/retarget_h1.sh"
    click R2 "https://github.com/nvlabs/hover/tree/main/third_party/human2humanoid"
    click IL "https://github.com/nvlabs/hover/tree/main/neural_wbc/isaac_lab_wrapper/neural_wbc/isaac_lab_wrapper"
    click MJ "https://github.com/nvlabs/hover/tree/main/neural_wbc/mujoco_wrapper/mujoco_wrapper"
    click IE "https://github.com/nvlabs/hover/tree/main/neural_wbc/inference_env"
    click CORE "https://github.com/nvlabs/hover/tree/main/neural_wbc/core/neural_wbc/core"
    click TP "https://github.com/nvlabs/hover/blob/main/scripts/rsl_rl/train_teacher_policy.py"
    click SP "https://github.com/nvlabs/hover/tree/main/neural_wbc/student_policy/neural_wbc/student_policy"
    click HW "https://github.com/nvlabs/hover/tree/main/neural_wbc/hw_wrappers/hw_wrappers"
    click RSL "https://github.com/nvlabs/hover/tree/main/third_party/rsl_rl"
    click MJV "https://github.com/nvlabs/hover/tree/main/third_party/mujoco_viewer"
    click UT "https://github.com/nvlabs/hover/blob/main/install_deps.sh"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef simulation fill:#87CEEB,stroke:#000,stroke-width:1px;
    classDef core fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef policy fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef hardware fill:#FF6347,stroke:#000,stroke-width:1px;
    classDef thirdparty fill:#DA70D6,stroke:#000,stroke-width:1px;
    classDef utilities fill:#D3D3D3,stroke:#000,stroke-width:1px;
```
