How nvlabs/cutamp works
Project type and purpose
• cuTAMP is an open-source Python library and CLI tool for Differentiable GPU-Parallelized Task and Motion Planning in robotics research.
• It integrates with NVIDIA’s cuRobo (for collision checking, robot kinematics, meshes) and PyTorch (for GPU-accelerated optimization).
• The primary use cases are: defining environments (Tetris, stick button, bookshelf), instantiating robot models (Franka, UR5, Robotiq grippers), sampling initial particles, optimizing a cost-based plan skeleton, and (optionally) running a motion planner and visualizing the result.
File structure
Top-level layout:
• cutamp/ – core Python package
– algorithm.py, optimize_plan.py, motion_solver.py, particle_initialization.py, samplers.py, rollout.py
– constraint_checker.py, cost_function.py, cost_reduction.py, costs.py
– config.py, experiment_logger.py
– envs/ – domain definitions + YAML asset files
– robots/ – robot models, URDF/XACRO descriptions, mesh assets, ROS2 launch snippets
– task_planning/ – symbolic structs, search, constraints, costs
– scripts/ – CLI entrypoint run_cutamp.py + script utilities
• docs/ – static assets, examples
• pyproject.toml, README.md, LICENSE
Architectural hints:
– Clear separation between “domain” (envs, tamp_domain/world), “solver” (algorithm, optimize_plan), “physical models” (robots), and “infrastructure” (config, logger, utils, scripts).
– Uses GPU via PyTorch for parallel cost evaluation and gradient-based optimization.
– Integrates external services: cuRobo (C++ extension for collision & kinematics), Rerun (visualization), ROS2 descriptors for robot assets.
Additional insights from README
• The CLI (cutamp-demo → run_cutamp.py) orchestrates end-to-end demos (Tetris, blocks, stick button).
• Emphasizes GPU memory management (num_particles), soft cost tuning, Rerun visualization with port forwarding.
• Cites core dependencies: Python 3.10+, PyTorch 2.0+, CUDA >= Volta, cuRobo.
• Demonstrates three phases: sampling initial plan skeletons, gradient-based optimization of soft/hard costs, optional downstream motion planning.
a. Main components:
– User Interface: CLI entrypoint (cutamp-demo / run_cutamp.py)
– Configuration & Params: config.py, command-line flags, experiment_logger
– Domain Definitions: cutamp/envs (surfaces, obstacles, YAML asset files), cutamp/tamp_domain & tamp_world
– Robot Definitions: cutamp/robots (URDF/XACRO, meshes), cuRobo integration
– Planning Engine:
• Task Planning submodule (plan skeleton generation, search)
• Particle Initialization (samplers, constraint_checker)
• Optimization Core (cost_function, cost_reduction, algorithm, optimize_plan using PyTorch on GPU)
• Motion Solver (optional trajectory refinement)
– Visualization & Logging: experiment_logger + Rerun SDK
– External Dependencies: cuRobo (collision/KD-tree), PyTorch (GPU), Rerun (visualization), ROS2 launch assets
b. Relationships & data flow:
1. CLI → reads config → Instantiates Domain + Robot Models
2. Task Planning → generates symbolic plan skeletons
3. Particle Initialization → samples state particles under hard constraints via constraint_checker
4. Optimization Core → repeatedly evaluates cost_function & cost_reduction on particles (GPU) → updates parameters
5. (Optional) Motion Solver → refines a continuous trajectory via cuRobo motion planning
6. experiment_logger streams data to Rerun visualizer → user inspects timelines
7. Final plan & metrics output to console/files
c. Architectural patterns & principles:
– Modular layering: separation of “symbolic” vs “geometric” planning
– GPU-parallel pipeline: batched tensor operations in PyTorch
– Plug-in extensibility: new envs, robots, cost terms can be added under respective directories
– Clear interface boundaries: constraint_checker & cost_function have well-defined inputs/outputs
d. Key technologies/frameworks:
– Python 3.10+, PyTorch 2.0+ (CUDA)
– cuRobo C++/Python extension (collision, kinematics, meshes)
– Rerun SDK for visualization
– ROS2 URDF/XACRO & launch files for robot asset descriptions
– Conda/pip installation with git-lfs for large assets
Diagramming recommendations
• Blue: core solver modules (algorithm, optimize_plan)
• Green: domain & robot definitions (envs/, robots/)
• Orange: external dependencies (cuRobo, PyTorch, Rerun)
• Grey: configuration & CLI
Detail
• Sub-components where relevant (e.g., “constraint_checker” under Particle Initializer)
• Asset pipelines (YAML → env objects; URDF/XACRO → robot model)
• Optional paths (with/without motion_solver, with/without visualizer)