# nicklashansen/puppeteer architecture

> Architecture diagram of the GitHub repository nicklashansen/puppeteer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nicklashansen/puppeteer
- Repository: https://github.com/nicklashansen/puppeteer

## Overview

##### Project type and purpose
- This is a research‐oriented, full-machine-learning “pipeline” project that implements a hierarchical world-model for whole-body humanoid control. It’s not a classical web/full-stack app; rather, it’s a simulation + model-training + evaluation system built on top of MuJoCo/DMControl.

##### File structure
- Top level
– docker/ (Dockerfile & conda environment) – reproducible environment specification
– data/ (download script & raw datasets)
– assets/ (figures & illustrative images)
– nautilus/launch.py – CLI entrypoint (Hydra/Launcher)
– train.py, evaluate.py – high-level scripts for training/eval
– puppeteer/ – core Python library
- puppeteer/ modules
– common/ (utilities: logging, math, buffer, world_model abstractions)
– envs/
· dm_control_wrapper.py – low-level sim wrapper
· tasks/ – arena definitions & task logic
· walkers/ – humanoid URDF/XML & model loader
· wrappers/ – observation/action post-processing
– trainer.py, tdmpc2.py – learning algorithms and optimization loops
– config.yaml – default hyperparameters & I/O spec

##### README for additional insights
- Two‐level hierarchy: low-level “tracking” model and high-level “puppeteer” model
- Supports multiple tasks (stand, walk, run, corridor variants)
- Depends on MuJoCo/DMControl, Python, (PyTorch implied), Hydra (nautilus), wandb for logging
- Outputs: model checkpoints, wandb logs, evaluation episodes

##### Elements
- Blue for simulation & env components
- Green for data pipeline
- Orange for model & trainer
- Purple for I/O & external services

##### Final advice
- Both the “static” structure (modules & packages) and the “dynamic” flow (training loop, eval loop)
- Config inputs (config.yaml, command flags) so viewers know where hyperparams come from

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Infrastructure Layer"
        Dockerfile["Dockerfile"]:::infra
        EnvYaml["environment.yaml"]:::infra
    end

    subgraph "Entry Points"
        Launch["nautilus/launch.py"]:::io
        TrainScript["train.py"]:::io
        EvalScript["evaluate.py"]:::io
    end

    subgraph "Configuration"
        Config["config.yaml"]:::config
    end

    subgraph "Simulation Environment"
        DMControl["DMControl Wrapper"]:::sim
        subgraph "Tasks"
            Arenas["arenas.py"]:::sim
            RunCorr["run_through_corridor.py"]:::sim
            WalkTask["walk.py"]:::sim
            TrackingTask["tracking.py"]:::sim
            TransferTask["transfer.py"]:::sim
        end
        subgraph "Walkers"
            CMUWalker["cmu_humanoid.py"]:::sim
            WalkerXML["humanoid_CMU_V2020.xml"]:::sim
        end
        subgraph "Wrappers"
            HumanoidWrap["humanoid.py"]:::sim
            TensorWrap["tensor.py"]:::sim
            TimeLimit["time_limit.py"]:::sim
        end
    end

    subgraph "Data Pipeline"
        Buffer["buffer.py"]:::data
        Dataset["mocap_dataset.py"]:::data
    end

    subgraph "World Models"
        LowModel["world_model.py"]:::model
        TDModel["tdmpc2.py"]:::model
    end

    subgraph "Trainer & Optimizer"
        Trainer["trainer.py"]:::model
        TrainCore["puppeteer/train.py"]:::model
        Logger["logger.py"]:::io
    end

    subgraph "Utilities"
        Parser["parser.py"]:::io
        Seed["seed.py"]:::io
    end

    subgraph "Outputs & External"
        Checkpoints["Model Checkpoints"]:::io
        Videos["Eval Videos/Metrics"]:::io
        WandB["Weights & Biases"]:::io
        GDrive["Google Drive"]:::io
    end

    %% Relationships
    Launch -->|uses| Config
    TrainScript -->|uses| Config
    EvalScript -->|uses| Config

    Config --> DMControl
    Config --> Arenas
    Config --> RunCorr
    Config --> WalkTask
    Config --> TrackingTask
    Config --> TransferTask
    Config --> CMUWalker
    Config --> WalkerXML
    Config --> HumanoidWrap
    Config --> TensorWrap
    Config --> TimeLimit
    Config --> Buffer
    Config --> Dataset
    Config --> LowModel
    Config --> TDModel
    Config --> Trainer
    Config --> Parser
    Config --> Seed

    DMControl -->|sim observations| Buffer
    Arenas -->|defines| DMControl
    RunCorr -->|defines| DMControl
    WalkTask -->|defines| DMControl
    TrackingTask -->|defines| DMControl
    TransferTask -->|defines| DMControl
    CMUWalker -->|model| DMControl
    WalkerXML -->|URDF/XML| DMControl
    HumanoidWrap -->|post-process| DMControl
    TensorWrap -->|post-process| DMControl
    TimeLimit -->|post-process| DMControl

    Buffer --> Trainer
    Dataset --> Trainer
    LowModel --> Trainer
    TDModel --> Trainer

    Trainer -->|logs| Logger
    Trainer -->|updates| LowModel
    Trainer -->|updates| TDModel
    Trainer -->|saves| Checkpoints
    Logger --> WandB
    Checkpoints --> GDrive

    EvalScript -->|loads| Checkpoints
    EvalScript -->|runs| DMControl
    DMControl --> Videos

    %% Click events
    click Dockerfile "https://github.com/nicklashansen/puppeteer/tree/main/docker/Dockerfile"
    click EnvYaml "https://github.com/nicklashansen/puppeteer/blob/main/docker/environment.yaml"
    click Launch "https://github.com/nicklashansen/puppeteer/blob/main/nautilus/launch.py"
    click TrainScript "https://github.com/nicklashansen/puppeteer/blob/main/train.py"
    click EvalScript "https://github.com/nicklashansen/puppeteer/blob/main/evaluate.py"
    click Config "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/config.yaml"
    click DMControl "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/dm_control_wrapper.py"
    click Arenas "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/tasks/arenas.py"
    click RunCorr "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/tasks/run_through_corridor.py"
    click WalkTask "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/tasks/walk.py"
    click TrackingTask "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/tasks/tracking.py"
    click TransferTask "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/tasks/transfer.py"
    click CMUWalker "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/walkers/cmu_humanoid.py"
    click WalkerXML "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/walkers/assets/humanoid_CMU_V2020.xml"
    click HumanoidWrap "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/wrappers/humanoid.py"
    click TensorWrap "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/wrappers/tensor.py"
    click TimeLimit "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/envs/wrappers/time_limit.py"
    click Buffer "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/common/buffer.py"
    click Dataset "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/common/mocap_dataset.py"
    click LowModel "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/common/world_model.py"
    click TDModel "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/tdmpc2.py"
    click Trainer "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/trainer.py"
    click TrainCore "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/train.py"
    click Logger "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/common/logger.py"
    click Parser "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/common/parser.py"
    click Seed "https://github.com/nicklashansen/puppeteer/blob/main/puppeteer/common/seed.py"

    %% Styles
    classDef infra fill:#dddddd,stroke:#333,stroke-width:1px
    classDef sim fill:#add8e6,stroke:#333,stroke-width:1px
    classDef data fill:#98fb98,stroke:#333,stroke-width:1px
    classDef model fill:#ffa500,stroke:#333,stroke-width:1px
    classDef io fill:#dda0dd,stroke:#333,stroke-width:1px
    classDef config fill:#00ced1,stroke:#333,stroke-width:1px
```
