How applied-ai-lab/amp-ls works
Based on our analysis of the file structure and the README, this project is a PyTorch-based implementation focused on gradient‐based motion planning in latent space. Its core functionality lies in training generative models (like kinematics VAEs and collision predictors) and using these models in a planning framework that leverages scene embeddings. The project handles simulated (via Gazebo) and even real-world dynamic scenes while also providing Docker support.
• Data Layer:
• “core/data” holds all the simulation assets and environment worlds. This is split by simulation types—the Gazebo assets (models, worlds, scripts) and scene collision examples.
• Model Components:
• “core/models” contains script implementations for different variants such as kinematics VAE, collision kinematics VAE, and plain VAE.
• Core Framework & Utilities:
• “core/components” holds common functionality like data loaders, evaluators, logging, and base model classes.
• “core/modules” includes major network components such as layers, loss functions, inference modules, and subnetwork definitions.
• Configuration & Experiment Management:
• “core/configs” is separated by model types (kinematics, collision, vae) and planning configurations. These abstract setup details for experiments.
• Planning & Reinforcement Learning:
• “core/planner” provides the planning agents, environment interfaces (both Gazebo and scene-specific), rollout storage, and planning strategies. This also connects to the RL training part.
• Robot Interaction:
• “core/robots” includes clients and transformation utilities that handle robot simulation (panda clients) and their control logic.
• Training & Execution:
• The top-level entry “core/train.py” along with “core/rl/train” (for planning in latent space) indicate the training and inference executables.
• Auxiliary Tools:
• “core/utils” contains various helper routines for data augmentation, visualization, distributed training, and more.
• Docker & Deployment:
• The “docker” folder shows Dockerfiles and related build scripts to encapsulate the environment.
Relationships and Interactions
• Data flows from the “core/data” layer (both Gazebo environments and scene collision assets) into the “data loaders” found in “core/components”.
• The loaded data is then used by the models (from “core/models”) whose architectures depend on modules (from “core/modules”).
• The configuration files (in “core/configs”) parameterize model training, collision checking, and planning processes.
• The “core/planner” interacts with both the trained VAE models and the collision predictor to compute motion plans in a latent space.
• Interfaces provided in “core/robots” allow simulated or real-world robot data to be integrated into this planning loop.
• Training scripts (“core/train.py” and “core/rl/train”) orchestrate these components, use experiment logging (“core/components/logger.py”), and output results (e.g., through TensorBoard and WandB).
• Docker files externalize the build/deployment environment, ensuring consistent execution.
Architectural Patterns and Design Principles
• Separation of Concerns: Distinct directories for data, configuration, model definition, planning, and utilities ensure modularity.
• Configurable Architecture: The “core/configs” directory shows that the system supports various model choices and planning strategies via configuration files.
• Extensibility: The presence of “base_model.py” and various sub-components indicate a design geared towards extending or swapping parts of the system.
Technologies & Frameworks
• PyTorch is the backbone for model implementation and training.
• Gazebo simulation frameworks are used for generating and testing motion planning in simulated environments.
• Docker is used for environment consistency and deployment.
• External logging and monitoring tools like TensorBoard and WandB are integrated in the pipeline.