How nicklashansen/puppeteer works
Project type and purpose
• This is a research‐oriented, full-machine-learning “pipeline” project that implements a hierarchical world-model for whole-body humanoid control. It’s not a classical web/full-stack app; rather, it’s a simulation + model-training + evaluation system built on top of MuJoCo/DMControl.
File structure
• Top level
– docker/ (Dockerfile & conda environment) – reproducible environment specification
– data/ (download script & raw datasets)
– assets/ (figures & illustrative images)
– nautilus/launch.py – CLI entrypoint (Hydra/Launcher)
– train.py, evaluate.py – high-level scripts for training/eval
– puppeteer/ – core Python library
• puppeteer/ modules
– common/ (utilities: logging, math, buffer, world_model abstractions)
– envs/
· dm_control_wrapper.py – low-level sim wrapper
· tasks/ – arena definitions & task logic
· walkers/ – humanoid URDF/XML & model loader
· wrappers/ – observation/action post-processing
– trainer.py, tdmpc2.py – learning algorithms and optimization loops
– config.yaml – default hyperparameters & I/O spec
README for additional insights
• Two‐level hierarchy: low-level “tracking” model and high-level “puppeteer” model
• Supports multiple tasks (stand, walk, run, corridor variants)
• Depends on MuJoCo/DMControl, Python, (PyTorch implied), Hydra (nautilus), wandb for logging
• Outputs: model checkpoints, wandb logs, evaluation episodes
Elements
• Blue for simulation & env components
• Green for data pipeline
• Orange for model & trainer
• Purple for I/O & external services
Final advice
• Both the “static” structure (modules & packages) and the “dynamic” flow (training loop, eval loop)
• Config inputs (config.yaml, command flags) so viewers know where hyperparams come from