# haosulab/maniskill architecture

> Architecture diagram of the GitHub repository haosulab/maniskill, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/haosulab/maniskill
- Repository: https://github.com/haosulab/maniskill

## Overview

##### Main Components
- Simulation & Rendering Engine: Highlight the integration with SAPIEN and Vulkan for rendering. Emphasize the GPU-parallelized simulation aspects (both state-based and heterogeneous environments).
- Environment & Scene Management: Include the “mani_skill/envs” and “mani_skill/utils/scene_builder” modules. This area manages different types of simulation environments, ranging from basic (minimal_template, empty_env) to complex tasks (tabletop, mobile manipulation, dexterity, digital twins).
- Agents, Controllers & Robots: Show components like “mani_skill/agents” which hosts base agents, controllers, and robot models (data in “mani_skill/agents/robots” and “mani_skill/assets/robots”). These represent how behaviors are defined and how various robot embodiments (e.g., panda, humanoid, so100) are structured.
- Task and Data Collection Modules: Represent “mani_skill/envs/tasks” and “mani_skill/utils/scene_builder” for task construction, data collection strategies, and integration of object meshes and assets.
- Learning Baselines & Examples: Include the “examples” directory that demonstrates various learning approaches (e.g., PPO, SAC, TD-MPC2, diffusion policy) as well as demonstration scripts.
- Utility & Build Infrastructure: Group smaller but equally important modules such as “mani_skill/utils”, “scripts”, and “tests” for asset building, logging, configuration management, and testing.
- Documentation & CI/CD: Outline the roles of the “docs” directory and GitHub workflows (in “.github/workflows”) for continuous integration and publishing.

##### Relationships and Data Flow
- For instance, “Agents” interact with “Environments” by sending control commands and receiving sensor observations.
- How learning algorithms (located in examples and baselines) interface with the simulation via APIs provided in modules like “mani_skill/agents” and “mani_skill/envs”.

##### Architectural Patterns and Design Principles
- Modular and Object-Oriented Design: Each major module (agents, envs, utils) serves as a self-contained component with clear interfaces for extension.
- GPU-Parallelization: Indicate that simulation and rendering are optimized for high throughput using GPU computation—this is a central design innovation.
- Separation of Concerns: Separate simulation logic, rendering, task configuration, asset loading, and training examples. This separation is evident across directories (e.g., “mani_skill/agents” vs. “mani_skill/envs” vs. “mani_skill/utils”).
- Extensibility: Note the simple task building API and configurable components (e.g., controllers and scene builders) that allow research users to extend or replace different parts.

##### Relevant Technologies and Dependencies
- SAPIEN and Vulkan are foundational to simulation and rendering.
- The use of PyTorch is implied (installation steps show “pip install torch”), which is critical for the learning (reinforcement and imitation learning) modules.
- Docker support (files in the “docker” directory) indicates containerized deployment/testing.
- CI/CD is managed through GitHub Actions workflows.

## Diagram (Mermaid)

```mermaid
graph TD
    %% External Dependencies
    subgraph "External Dependencies"
        SAPIEN("SAPIEN"):::external
        Vulkan("Vulkan"):::external
        Torch("PyTorch"):::external
        Docker("Docker"):::external
    end

    %% Simulation & Rendering Engine
    subgraph "Simulation & Rendering Engine"
        SAE("SAPIEN Engine"):::simulation
        RM("Rendering Module"):::simulation
        GPU("GPU Parallelization"):::note
    end

    %% Environment & Scene Management
    subgraph "Environment & Scene Management"
        Env("Environments"):::env
        SB("Scene Builder"):::env
    end

    %% Agents & Controllers/Robots
    subgraph "Agents & Robots"
        AC("Agents & Controllers"):::agent
        AR("Assets & Robots"):::agent
    end

    %% Task and Data Collection Modules
    subgraph "Task and Data Collection"
        Tasks("Tasks"):::task
    end

    %% Learning Baselines & Examples
    subgraph "Learning Baselines & Examples"
        LB("Learning Baselines & Examples"):::learning
    end

    %% Utility & Build Infrastructure
    subgraph "Utilities & Build Infrastructure"
        Util("Utils"):::utility
        Scripts("Scripts"):::utility
        Tests("Tests"):::utility
    end

    %% Documentation & CI/CD
    subgraph "Documentation & CI/CD"
        Docs("Docs"):::doc
        CI(".github/workflows"):::doc
    end

    %% Connections: External Dependencies to Simulation and Others
    SAPIEN --> SAE
    Vulkan --> RM
    Torch --> LB
    Docker --> CI

    %% Internal Data Flow
    GPU --> SAE
    GPU --> RM
    SAE --> Env
    RM --> Env
    Env --> SB
    AC --> Env
    Env --> AC
    Env --> Tasks
    SB --> Tasks
    Tasks --> LB
    AC --> LB
    AR --> AC

    %% Supporting Infrastructure (Utilities)
    SAE --> Util
    Env --> Util
    AC --> Util
    Tasks --> Util
    LB --> Util

    %% Click Events
    click SAE "https://github.com/haosulab/maniskill/blob/main/mani_skill/envs/sapien_env.py"
    click RM "https://github.com/haosulab/maniskill/tree/main/mani_skill/render"
    click Env "https://github.com/haosulab/maniskill/tree/main/mani_skill/envs"
    click SB "https://github.com/haosulab/maniskill/tree/main/mani_skill/utils/scene_builder"
    click AC "https://github.com/haosulab/maniskill/tree/main/mani_skill/agents"
    click AR "https://github.com/haosulab/maniskill/tree/main/mani_skill/assets/robots"
    click Tasks "https://github.com/haosulab/maniskill/tree/main/mani_skill/envs/tasks"
    click LB "https://github.com/haosulab/maniskill/tree/main/examples"
    click Util "https://github.com/haosulab/maniskill/tree/main/mani_skill/utils"
    click Scripts "https://github.com/haosulab/maniskill/tree/main/scripts"
    click Tests "https://github.com/haosulab/maniskill/tree/main/tests"
    click Docs "https://github.com/haosulab/maniskill/tree/main/docs"
    click CI "https://github.com/haosulab/maniskill/tree/main/.github/workflows"

    %% Styles
    classDef simulation fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef env fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef agent fill:#FFD700,stroke:#000,stroke-width:2px;
    classDef task fill:#FFA07A,stroke:#000,stroke-width:2px;
    classDef learning fill:#B0E0E6,stroke:#000,stroke-width:2px;
    classDef utility fill:#D3D3D3,stroke:#000,stroke-width:2px;
    classDef doc fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef external fill:#EE82EE,stroke:#000,stroke-width:2px;
    classDef note fill:#f0e68c,stroke:#000,stroke-width:2px;
```
