# nvlabs/mimicgen architecture

> Architecture diagram of the GitHub repository nvlabs/mimicgen, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvlabs/mimicgen
- Repository: https://github.com/nvlabs/mimicgen

## Overview

##### Project type and purpose
- This is an open-source Python toolkit (MimicGen) for generating large-scale robot learning datasets via simulated environments.
- It provides:
- A CLI/API layer (scripts) to launch data-generation runs.
- A core data-generation engine configurable by JSON configs.
- Wrappers around the robosuite simulation environments.
- Assets/models for simulated robots and objects.
- Utility functions and experiment templates.

##### File structure
- Top-level packages under `mimicgen/`:
- configs: configuration definitions for tasks and environments
- datagen: core data-generation pipeline (data_generator, selection_strategy, waypoint, etc.)
- env_interfaces: abstract base and robosuite wrappers
- envs/robosuite: concrete env definitions for each task (coffee, pick_place, etc.)
- exps/templates: JSON templates for experiment configurations
- models: XML/STL assets and object definitions for simulation
- scripts: CLI entry points for generating datasets, configs, videos, etc.
- utils: file, pose, config, robomimic utilities
- Support files: setup.py, requirements, add_license_header.py
- Documentation under docs/ with Sphinx source.

##### README
- Confirms purpose: “data generation system for scalable robot learning.”
- Mentions 48,000+ demonstrations, configurable runs, extant datasets on external storage (Hugging Face).

1. CLI/Scripts Layer
- Entry points: generate_dataset.py, download_datasets.py, etc.
2. Config Module
- Loads JSON/py configs (mimicgen.configs + templates)
3. Data-Generation Engine
- Core pipeline: DataGenerator, SelectionStrategy, Waypoint
4. Environment Interface
- Base interface → RobosuiteInterface
5. Simulation Environments
- Task-specific env classes (coffee, hammer_cleanup, etc.)
6. Models & Assets
- XML, STL, OBJ in mimicgen/models for object meshes and scene definitions
7. Utils
- File I/O, pose conversions, robomimic helpers
8. Data Output
- HDF5 logs, video recordings, JSON metadata
9. External Services
- Dataset hosting (Hugging Face, Google Drive)

- CLI/Scripts → reads Config Module → instantiates Data-Generation Engine
- Engine → calls Environment Interface → spawns Simulation Env and loads Models & Assets
- Simulation Env → executes episodes, returns observations → Engine processes with Waypoint & Strategy
- Engine → writes outputs via Utils to Data Output storage
- Scripts optionally fetch existing datasets from External Services

c. Architectural patterns & principles
- Modularity: clear separation between config, engine, env interfaces, and assets
- Strategy pattern: selection_strategy, waypoint sampling abstracted
- Plugin-style envs: new tasks extend base interface
- Config-driven: JSON templates + Python config definitions
- Reusability: utils and common code support multiple scripts and pipelines

d. Key technologies & libraries
- Python 3.x, robosuite simulator
- HDF5 (h5py) for dataset storage
- Sphinx for docs
- Command-line scripts

##### Detail
- How JSON templates in `exps/templates/robosuite` feed into the config loader
- Outputs (HDF5, video, JSON logs) and optionally the dataset upload process to Hugging Face
- Non-obvious dependencies (e.g., scripts use `config_utils` to assemble configs before passing to the engine)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI/Scripts Layer
    subgraph "CLI/Scripts Layer"
        Scripts["CLI/Scripts"]:::script
    end

    %% Config Module & Templates
    subgraph "Config Module & Templates"
        Configs["Config Module"]:::core
        Templates["Experiment Templates"]:::core
    end

    %% Core Engine
    subgraph "Data-Generation Engine"
        Engine["Data-Generation Engine"]:::core
    end

    %% Environment Interface
    subgraph "Environment Interface"
        Interface["Env Interface"]:::core
    end

    %% Simulation Environments
    subgraph "Simulation Environments"
        Envs["RoboSuite Envs"]:::core
    end

    %% Models & Assets
    subgraph "Models & Assets"
        Models["Models & Assets"]:::asset
    end

    %% Utility Functions
    subgraph "Utilities"
        Utils["Utility Functions"]:::util
    end

    %% Data Output
    subgraph "Data Output"
        Output["HDF5, Video, JSON Logs"]:::storage
    end

    %% External Services
    External["External Services"]:::external

    %% Connections
    Scripts -->|reads configs| Configs
    Templates -->|provide JSON templates| Configs
    Configs -->|instantiates| Engine
    Engine -->|uses| Interface
    Interface -->|spawns| Envs
    Engine -->|loads| Models
    Envs -->|returns observations| Engine
    Engine -->|calls utils| Utils
    Utils -->|writes data| Output
    Scripts -->|fetches existing| External
    Output -->|uploads datasets| External

    %% Click Events
    click Scripts "https://github.com/nvlabs/mimicgen/tree/main/mimicgen/scripts/"
    click Configs "https://github.com/nvlabs/mimicgen/tree/main/mimicgen/configs/"
    click Templates "https://github.com/nvlabs/mimicgen/tree/main/mimicgen/exps/templates/robosuite/"
    click Engine "https://github.com/nvlabs/mimicgen/tree/main/mimicgen/datagen/"
    click Interface "https://github.com/nvlabs/mimicgen/tree/main/mimicgen/env_interfaces/"
    click Envs "https://github.com/nvlabs/mimicgen/tree/main/mimicgen/envs/robosuite/"
    click Models "https://github.com/nvlabs/mimicgen/tree/main/mimicgen/models/robosuite/"
    click Utils "https://github.com/nvlabs/mimicgen/tree/main/mimicgen/utils/"

    %% Styles
    classDef script fill:#E0F7FA,stroke:#006064;
    classDef core fill:#E8F5E9,stroke:#1B5E20;
    classDef asset fill:#FFF3E0,stroke:#E65100,stroke-dasharray:5 5;
    classDef util fill:#FFFDE7,stroke:#F9A825;
    classDef storage fill:#FCE4EC,stroke:#880E4F;
    classDef external fill:#ECEFF1,stroke:#455A64,stroke-dasharray:2 2;
```
