How fzi-forschungszentrum-informatik/manipulation_pipeline works
1. This repository is split into two ROS 2 packages with a clear boundary:
- manipulation_pipeline_interfaces: the external API contract.
- manipulation_pipeline: the MoveIt 2–based implementation that serves that API.
2. The public-facing contract is action-centric. The interfaces package defines actions for the main manipulation primitives: MoveToPose, MoveToNamedPose, MoveToConfiguration, ExecutePath, Grasp, Place, and ActuateTool. These are the entry points a client would use to request motion planning or manipulation.
3. The same interfaces package also defines the supporting data model used by those actions: motion parameters, cartesian/linear motion limits, progress reporting, state reporting, tool commands, collision pairs, and object composition metadata. This is the stable boundary between external clients and the planner/executor internals.
4. The implementation package is organized as a layered planning-and-execution pipeline rather than a single monolithic node. The main orchestration classes are manipulation_pipeline, planning_pipeline, planning_interface, planner, and execution_pipeline, which together turn an incoming request into a plan and then into executed robot motion.
5. Planning is split into reusable “planning steps.” The planning_steps directory contains step implementations such as move_to_pose, move_to_named_pose, move_to_configuration, grasp, place, actuate_tool, and execute_path. This suggests the system composes multi-stage manipulation behaviors from smaller steps instead of hardcoding each behavior into a separate node.
6. Execution is its own subsystem. execution_pipeline and the action-specific execution helpers under src/actions handle sending trajectories or commands downstream after planning succeeds. The execute_trajectory and execute_path pieces indicate a clear separation between trajectory generation and controller execution.
7. World and robot state management is another core subsystem. robot_model, robot_model_monitor, and collision_object_manager manage the robot description and collision scene, including runtime changes. The README explicitly calls out support for URDF/SRDF changes during runtime, which is important for tool-changer or reconfiguration scenarios.
8. The package exposes object-scene manipulation through services in the interfaces package: spawn, remove, remove multiple, split, and combine objects. That means the pipeline is not limited to pure motion goals; it also maintains the planning world model as part of the interaction loop.
9. Several internal utility components support this architecture:
- ik_sampler for inverse-kinematics sampling,
- planning_context for carrying request-specific state,
- goal_handle for tracking action lifecycles,
- worker_thread for background/asynchronous processing,
- visualization_interface for publishing or updating visualization state.
10. The data flow is typically: client sends a ROS 2 action or service request → manipulation_pipeline interprets it into a planning context and step sequence → planning uses MoveIt 2/MoveItCpp plus the current robot/world model → execution pipeline sends the resulting motion to controllers → progress and state are reported back through the action result/feedback channels and state messages.
11. The main external dependency is MoveIt 2, and the README makes it clear the repo expects a working _moveit_config package with normal MoveIt concepts and default controller selection. So the architecture is intentionally lightweight around MoveIt rather than replacing it.
12. In diagram terms, the key boundaries are:
- external consumers vs. ROS interface package,
- interface package vs. implementation package,
- planning vs. execution,
- robot/world state management vs. motion generation,
- internal step composition vs. action-level API.
Main entry points: planning_pipeline.h Planning pipeline · execution_pipeline.h Execution pipeline · manipulation_pipeline.h Pipeline entry · planner.h Planner
Components 26
ROS 2 Interfaces
Public action, message, and service contract for external clients.
- Action API · Public manipulation actions for motion and task requests.
manipulation_pipeline_interfaces/action - Shared data · Motion parameters, progress/state reporting, tool commands, and scene metadata.
manipulation_pipeline_interfaces/msg - Scene services · Spawn, remove, split, and combine object services for world updates.
manipulation_pipeline_interfaces/srv
Pipeline Implementation
MoveIt 2–based planning, state management, and execution layers.
- Pipeline entry · Top-level ROS 2 entry point that receives requests and drives planning/execution.
manipulation_pipeline/include/manipulation_pipeline/manipulation_pipeline.h - Planning interface · Adapts incoming ROS requests into internal planning operations.
manipulation_pipeline/include/manipulation_pipeline/planning_interface.h - Planning pipeline · Coordinates the sequence of planning stages for a manipulation request.
manipulation_pipeline/include/manipulation_pipeline/planning_pipeline.h - Planner · Central planning logic built on MoveIt 2 / MoveItCpp concepts.
manipulation_pipeline/include/manipulation_pipeline/planner.h - Planning context · Carries request-specific state through the planning stages.
manipulation_pipeline/include/manipulation_pipeline/planning_context.h - Execution pipeline · Turns successful plans into controller-facing execution.
manipulation_pipeline/include/manipulation_pipeline/execution_pipeline.h - Robot model · Maintains robot description and runtime model changes.
manipulation_pipeline/include/manipulation_pipeline/robot_model.h - Model monitor · Observes and updates robot/world model state at runtime.
manipulation_pipeline/include/manipulation_pipeline/robot_model_monitor.h - Collision scene · Manages collision objects and planning-scene updates.
manipulation_pipeline/include/manipulation_pipeline/collision_object_manager.h - IK sampler · Samples inverse-kinematics solutions for planning steps.
manipulation_pipeline/include/manipulation_pipeline/ik_sampler.h - Visualization · Publishes or updates visualization state for operators and tools.
manipulation_pipeline/include/manipulation_pipeline/visualization_interface.h - Worker thread · Background/asynchronous work support for pipeline tasks.
manipulation_pipeline/include/manipulation_pipeline/worker_thread.h
Planning Steps
Composable manipulation behaviors used by the planner.
- Move pose · Planning step for reaching a Cartesian pose.
manipulation_pipeline/include/manipulation_pipeline/planning_steps/move_to_pose.h - Move named · Planning step for moving to a named configuration.
manipulation_pipeline/include/manipulation_pipeline/planning_steps/move_to_named_pose.h - Move config · Planning step for moving to an explicit joint configuration.
manipulation_pipeline/include/manipulation_pipeline/planning_steps/move_to_configuration.h - Grasp · Planning step for grasp approach and closure.
manipulation_pipeline/include/manipulation_pipeline/planning_steps/grasp.h - Place · Planning step for placement behavior.
manipulation_pipeline/include/manipulation_pipeline/planning_steps/place.h - Actuate tool · Planning step for tool command execution.
manipulation_pipeline/include/manipulation_pipeline/planning_steps/actuate_tool.h - Execute path · Planning step for following a precomputed path.
manipulation_pipeline/include/manipulation_pipeline/planning_steps/execute_path.h
Action Helpers
Execution-side helpers for action-specific downstream behavior.
- Exec trajectory · Sends planned trajectories downstream for execution.
manipulation_pipeline/include/manipulation_pipeline/actions/execute_trajectory.h - Dwell · Represents waiting or pause behavior during action execution.
manipulation_pipeline/include/manipulation_pipeline/actions/dwell.h - Attach change · Updates attached-object state during execution.
manipulation_pipeline/include/manipulation_pipeline/actions/change_attached_object.h
Other components
- External clients · ROS 2 nodes or systems that call the manipulation API.
Connections 28
- External clients → Action API: calls · Clients invoke manipulation actions.
- External clients → Scene services: updates world · Clients manage objects through services.
- Action API → Pipeline entry: requests · Action goals enter the implementation layer.
- Shared data → Pipeline entry: contracts · Shared messages define request, feedback, and state payloads.
- Scene services → Collision scene: scene ops · Object lifecycle services affect the planning scene.
- Pipeline entry → Planning interface: adapts · Top-level orchestration converts API calls into internal requests.
- Planning interface → Planning context: builds · Request data is assembled into planning context.
- Planning interface → Planning pipeline: dispatches · Prepared requests move into the planning flow.
- Planning pipeline → Planner: plans · The pipeline delegates core motion planning to the planner.
- Planning pipeline → Move pose: composes · Pipeline selects planning steps for a task.
- Planning pipeline → Move named: composes · Pipeline selects planning steps for a task.
- Planning pipeline → Move config: composes · Pipeline selects planning steps for a task.
- Planning pipeline → Grasp: composes · Pipeline selects planning steps for a task.
- Planning pipeline → Place: composes · Pipeline selects planning steps for a task.
- Planning pipeline → Actuate tool: composes · Pipeline selects planning steps for a task.
- Planning pipeline → Execute path: composes · Pipeline selects planning steps for a task.
- Planner → IK sampler: samples · IK sampling supports candidate pose generation.
- Planner → Robot model: reads model · Planning uses the current robot description.
- Planner → Collision scene: uses scene · Planning accounts for collision objects and scene state.
- Model monitor → Robot model: syncs · Runtime updates keep the model current.
- Model monitor → Collision scene: syncs · World and scene changes are monitored at runtime.
- Planning pipeline → Execution pipeline: hands off · Successful plans are passed to execution.
- Execution pipeline → Exec trajectory: executes · Execution pipeline sends trajectories to controller-facing helpers.
- Execution pipeline → Dwell: coordinates · Execution pipeline can include dwell/pause behavior.
- Execution pipeline → Attach change: coordinates · Execution updates attached-object state when needed.
- Execution pipeline → Visualization: reports · Execution updates visualization and operator feedback.
- Pipeline entry → Worker thread: runs async · Background work supports non-blocking request handling.
- Pipeline entry → Visualization: publishes state · Top-level pipeline exposes progress and status views.