# bytedance/wmp architecture

> Architecture diagram of the GitHub repository bytedance/wmp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bytedance/wmp
- Repository: https://github.com/bytedance/wmp

## Overview

##### Main Components
a. Environment & Simulation:
- The “legged_gym” directory contains all the definitions for environments (“envs”), scripts (train.py, play.py), tasks, robots’ physics and rendering resources, etc.
- The resources folder holds URDF and mesh models for the A1 robot, indicating the visual and physical assets used in simulation.
b. World Model/Perception Module:
- The “dreamer” folder holds the world model components – configurations (configs.yaml), models, networks, and supporting tools – that are central for world model-based perception.
c. Reinforcement Learning Algorithms:
- The “rsl_rl” folder contains all RL modules including algorithms (e.g., PPO variants, AMP PPO/discriminator), environment wrappers, runners (on-policy and WMP runners), memory storage for rollouts, and even dataset utilities.
d. Datasets and Motion Capture Data:
- The “datasets” directory features motion capture files (mocap_motions) which might be used to assist with imitation learning, behavior cloning, or as data guides for training.
e. External Dependencies & Licenses:
- Although not a runtime component, the README outlines the use of external packages like PyTorch and Isaac Gym. Likewise, the licenses (in the licenses and resources directories) show integration with other open-source projects.

##### Relationships and Interactions
- Data Flow: Motion capture datasets are likely used to help train or validate the perception module; the world model (“dreamer”) processes visual inputs and outputs perception cues; simulated data (from legged_gym) feeds into the RL pipeline (“rsl_rl”), where learning and policy updates occur.
- Training Loop: The training script (legged_gym/scripts/train.py) couples the simulation environment with the RL algorithms from “rsl_rl” and, indirectly, incorporates perception outputs from “dreamer”. The play script is used for visualization/validation once training is complete.
- Components communicate through well-defined APIs and shared configuration files (like YAML configurations in “dreamer/configs.yaml” and robot task configurations in legged_gym/envs).

##### Architectural Patterns and Design Principles
- Modular Design: Each module is kept in its own directory (simulation, world model, RL algorithms, datasets) making the project highly modular and extensible.
- Separation of Concerns: The responsibilities are clearly divided (environment simulation, perception, learning algorithms), which also allows independent development and testing—for instance, “legged_gym/tests” validates simulation behavior.
- Integration of Open-Source Components: There is clear reuse of established frameworks (e.g., Isaac Gym, dreamerv3-torch) with additional in-house modules to tie them together.

##### Technologies and Frameworks
- PyTorch is central for developing the world model and RL modules.
- Isaac Gym is used for high-performance simulation of the legged robot.
- YAML for configuration management.
- Additional libraries/tools are used for logging, helper functions, and math utilities (all found under various “utils” subfolders).

##### Final Step – Putting It All Together
- Be very detailed—every major file set (like those in “dreamer”, “legged_gym”, and “rsl_rl”) should be represented as a subsystem or module, with subcomponents (e.g., various algorithms, environment definitions, etc.) optionally shown if needed for clarity.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    subgraph "Data Sources"
        D1(("Datasets (Motion Capture Data)"))
        A1(("Robot Assets"))
    end

    %% Core Modules
    subgraph "Core Modules"
        S1["Legged Simulation Environment"]
        W1["World Model (Dreamer)"]
        R1["Reinforcement Learning Module (rsl_rl)"]
    end

    %% External Dependencies
    L1(("External Dependencies & Licenses"))

    %% Connections from Data Sources to Core Modules
    D1 -->|"motionData"| W1
    D1 -->|"trainingData"| R1
    A1 -->|"assetData"| S1

    %% Connections within Core Modules
    S1 -->|"visualData"| W1
    S1 -->|"simulationOutput"| R1
    W1 -->|"perceptionCues"| R1
    R1 -->|"policyUpdate"| S1

    %% External Dependencies to Core Modules
    L1 -->|"dependencies"| S1
    L1 -->|"dependencies"| W1
    L1 -->|"dependencies"| R1

    %% Click Events
    click S1 "https://github.com/bytedance/wmp/tree/master/legged_gym/envs"
    click W1 "https://github.com/bytedance/wmp/blob/master/dreamer/configs.yaml"
    click R1 "https://github.com/bytedance/wmp/tree/master/rsl_rl/algorithms"
    click D1 "https://github.com/bytedance/wmp/tree/master/datasets/mocap_motions"
    click A1 "https://github.com/bytedance/wmp/tree/master/resources/robots/a1"
    click L1 "https://github.com/bytedance/wmp/tree/master/licenses"

    %% Styles
    classDef sim fill:#add8e6,stroke:#000,stroke-width:2px;
    classDef wm fill:#90ee90,stroke:#000,stroke-width:2px;
    classDef rl fill:#ffa07a,stroke:#000,stroke-width:2px;
    classDef ds fill:#ffff99,stroke:#000,stroke-width:2px;
    classDef assets fill:#dda0dd,stroke:#000,stroke-width:2px;
    classDef lic fill:#d3d3d3,stroke:#000,stroke-width:2px;

    class S1 sim;
    class W1 wm;
    class R1 rl;
    class D1 ds;
    class A1 assets;
    class L1 lic;
```
