jon-bon-jono/vip_mmwave_hpe_demo

1. Overall shape

- This is a ROS 2 Jazzy workspace organized as three real runtime layers plus one future ML placeholder:

- vip_hpe_msgs for shared message definitions

- vip_hpe_core for ROS-free preprocessing and constants

- vip_hpe_runtime for ROS nodes, launch/session orchestration, replay, and visualization

- ml/ as a future training framework, intentionally ignored by colcon

2. Primary runtime data flow

- The default demo is a replay pipeline, not live sensor capture.

- Data flows from prerecorded pickle data into ROS topics, then through preprocessing, then to visualization:

- radar point-cloud replay publishes vip_hpe_msgs/RadarFrame on /radar/points

- pose emulation subscribes to /radar/points, preprocesses the point cloud, and publishes vip_hpe_msgs/PersonPose3DArray on /pose/people

- the animated plot node subscribes to both topics and renders the result

3. Session/launch orchestration

- vip_hpe_runtime/launch/session.launch.py starts a full session from YAML under src/vip_hpe_runtime/config/sessions/.

- The session config defines workspace root, topics, source selection, pose mode, preprocessing config, and sinks.

- This is the main composition boundary: the launch layer wires together nodes based on config rather than hardcoding the graph.

4. Runtime nodes

- radar_replay_node replays radar frames from a pickle file.

- pose_gt_inference_emulator_node is the key bridge node: it makes replayed radar look like a future inference path by converting RadarFrame to NumPy, building a preprocessor from YAML, and publishing matching ground-truth pose.

- animated_plot_node is a visualization sink using Matplotlib 3D.

- pose_gt_replay_node and live_model_inference_node exist as additional runtime modes/placeholders in the same node family.

5. Replay subsystem

- vip_hpe_runtime/replay/living_lab.py parses the UNSW Living Lab pickle dataset used by the demo.

- replay_base.py likely provides shared playback behavior for replay-style nodes.

- This subsystem isolates dataset parsing and timed replay from ROS publishing and from pose preprocessing.

6. ROS adapters and message boundary

- vip_hpe_runtime/adapters/ros_conversions.py is the translation layer between ROS message types and internal Python/NumPy representations.

- Custom messages in vip_hpe_msgs/msg/ define the runtime contract:

- RadarFrame, RadarPoint

- Joint3D, PersonPose3D, PersonPose3DArray

- This package is the cross-package interface used by runtime nodes and any future live inference implementation.

7. Shared preprocessing core

- vip_hpe_core is explicitly the non-ROS, non-PyTorch shared domain layer.

- Its main modules are:

- preprocessing/base.py for the preprocessor interface

- preprocessing/config.py for YAML-driven construction

- preprocessing/registry.py for name-to-class lookup

- preprocessing/model_input.py for the model-facing data container

- The dummy preprocessor config in src/vip_hpe_core/config/preprocessing/dummy_pointcloud.yaml is the current example implementation.

8. Dependency boundary

- vip_hpe_core must stay free of ROS, rclpy, vip_hpe_msgs, PyTorch, Ultralytics, ONNX Runtime, and Matplotlib.

- vip_hpe_runtime is allowed to depend on ROS and on vip_hpe_core.

- ml/ is the intended home for future PyTorch training code and should reuse vip_hpe_core rather than duplicating preprocessing logic.

- This separation is the key architectural constraint in the repo.

9. Constants and sensor/domain presets

- vip_hpe_core/constants/ contains sensor- or environment-specific constants such as iwr6843.py, kinect.py, and living_lab.py.

- These files suggest the project is meant to normalize differences between radar hardware, Kinect-derived ground truth, and the Living Lab dataset.

- They act as shared reference data for preprocessing and replay without tying the core to any particular runtime node.

10. Configuration-driven behavior

- Runtime behavior is mostly configured through YAML, not code changes.

- The default session config points to the dummy point-cloud preprocessor YAML, and pose mode replay_gt_synced demonstrates the future inference contract.

- The topic names are part of the config contract and are meant to stay stable as the system evolves from replay/emulation to live inference.

11. Testing and CI

- Tests are split by layer:

- tests/core for ROS-free preprocessing and architecture boundaries

- tests/runtime for replay/runtime parsing and node-adjacent logic

- tests/ml as a placeholder

- The existence of a dependency-boundary test is important architecturally: it enforces the split between shared core logic and ROS runtime.

- GitHub Actions runs pytest for the core package.

12. What a diagram should emphasize

- Show data/ll_replay/07_SW.pickle feeding the replay node.

- Show a ROS topic bus with /radar/points and /pose/people as the central data channels.

- Place vip_hpe_core off to the side as a pure preprocessing library used by runtime and future ML.

- Place vip_hpe_msgs as the shared schema boundary between producers, consumers, and future live inference.