How yizzfz/milipoint works
1. What this repository is
- This is a Python point-cloud learning package for mmWave radar data, centered on the MiliPoint dataset and a small training/evaluation framework called mmrnet.
- The repo is structured around three problems: keypoint estimation, subject identification, and action classification.
2. Top-level architecture
- The code is split into:
- mmrnet/ for reusable library code,
- configs/ for experiment definitions,
- data/ for raw and cached/processed datasets,
- checkpoints/ for trained model artifacts,
- mm as the command-line entrypoint wrapper,
- install_mmrnet.sh for environment/setup convenience.
- This is not a web/service architecture; it is an offline ML research pipeline.
3. Data boundary: raw radar files to cached datasets
- Raw inputs live under data/raw/ as .pkl files.
- Dataset-specific preprocessing writes cached artifacts into data/processed/..., and those cached files are reused unless forced to rebuild.
- The README shows that the processed cache is the main boundary between dataset preparation and model training.
4. Dataset subsystem
- mmrnet/dataset/mmrnet_data.py is the core dataset layer.
- It appears to handle:
- loading raw point-cloud sequences,
- stacking multiple frames together,
- zero-padding strategies,
- train/val/test split creation,
- task-specific labels for keypoints, identity, or action.
- The public entrypoint exposed in the README is get_dataset(...), which returns PyTorch-style data loaders plus dataset metadata.
5. Task-specific dataset variants
- The repo provides separate configs and processed-data paths for:
- mmr_keypoints,
- mmr_iden,
- mmr_action.
- These correspond to different label heads and evaluation targets, but share the same underlying point-cloud source format and much of the same loading logic.
6. Model subsystem
- mmrnet/models/ contains multiple interchangeable point-cloud models:
- mlp.py
- dgcnn.py
- pointnet.py
- point_transformer.py
- pointmlp.py
- The architecture is modular: the dataset/task layer is separated from the model choice, so the same task can be trained with different backbones.
- The README indicates a model_map is available, implying a registry/factory pattern for selecting models by name.
7. Training, evaluation, and visualization
- mmrnet/session/train.py handles training loops.
- mmrnet/session/test.py handles evaluation/inference on a saved checkpoint.
- mmrnet/session/visualize.py supports visualizing keypoint predictions, with the README noting video output for keypoint tasks.
- mmrnet/session/wrapper.py likely centralizes the execution flow shared by train/test/visualize sessions.
8. Command-line interface
- mmrnet/cli.py and the top-level mm script provide the user-facing interface.
- The command pattern is:
- ./mm train <task> <model>
- ./mm test <task> <model>
- CLI arguments can override config-file values, so the runtime configuration boundary is: defaults in TOML + overrides from the command line.
9. Configuration and experiment management
- configs/ contains TOML files per task, such as:
- configs/keypoints/mmr_keypoints_stack_5_point.toml
- configs/iden/mmr_iden_stack_5_point.toml
- configs/action/mmr_action_stack_5_point.toml
- These configs define dataset construction parameters such as split ratios, frame stacking, padding behavior, and cached processed-data paths.
- This makes the repo suitable for reproducible experiments: same dataset code, different task/model/config combinations.
10. Artifacts and persistence
- Trained outputs are expected to be stored in checkpoints/.
- The README uses a --save / --load workflow, so checkpoints are the persistence boundary between training and later evaluation or visualization.
- Processed dataset files are also persisted, which reduces repeated preprocessing cost.
11. Runtime and dependencies
- The project is a PyTorch-based ML codebase.
- It uses torch-cluster, which is a strong signal that some models or point-cloud operations depend on graph/neighborhood building.
- The installation flow (requirements.txt, setup.py, and install_mmrnet.sh) indicates this is intended to be installed as a package and used from scripts or the CLI.
12. End-to-end data flow
- Typical flow is:
- raw .pkl radar data in data/raw/
- preprocessing/caching into data/processed/
- dataset loaders created through mmrnet.dataset
- model selection through mmrnet.models
- training/evaluation through mmrnet.session
- outputs saved to checkpoints/
- For keypoint tasks, there is an additional visualization path that can render prediction results as video.
13. How to think about the architecture diagram
- Draw three major blocks:
- Data preparation/cache
- Task/dataset + model runtime
- Session/CLI orchestration
- Show config files feeding into both dataset construction and session behavior.
- Show checkpoints and processed-data caches as the two persistent storage layers.
- Keep task types as branches off the same shared pipeline rather than separate systems, because the repo is built around reuse across tasks.
Main entry points: __init__.py Model map · wrapper.py Wrapper · configs Configs · checkpoints Checkpoints
Components 18
Interface
User entrypoints and installation helpers.
Configuration
Experiment definitions and task-specific parameters.
- Configs · Task TOMLs for keypoints, identity, and action runs.
configs
Storage
Raw inputs, cached datasets, and saved artifacts.
- Raw data · Original mmWave point-cloud inputs and labels.
data/raw - Processed cache · Cached preprocessed datasets reused across runs.
data/processed - Checkpoints · Saved training outputs used for later test and visualization.
checkpoints
Core runtime
Shared dataset and model code.
- Dataset loader · Loads raw sequences, stacks frames, pads, and builds task splits.
mmrnet/dataset/mmrnet_data.py - Model map · Selects point-cloud backbone implementations by name.
mmrnet/models/__init__.py - MLP · Baseline point-cloud model.
mmrnet/models/mlp.py - DGCNN · Graph-neighborhood point-cloud model.
mmrnet/models/dgcnn.py - PointNet · Classic point-cloud backbone.
mmrnet/models/pointnet.py - PointTrans · Transformer-based point-cloud model.
mmrnet/models/point_transformer.py - PointMLP · MLP-style point-cloud backbone.
mmrnet/models/pointmlp.py
Sessions
Training, evaluation, and visualization flows.
- Wrapper · Shared execution flow for train, test, and visualize runs.
mmrnet/session/wrapper.py - Train · Optimizes a selected model on a task dataset.
mmrnet/session/train.py - Test · Loads a saved checkpoint and runs inference or evaluation.
mmrnet/session/test.py - Visualize · Renders keypoint predictions and video outputs.
mmrnet/session/visualize.py
Connections 21
- mm → CLI: invokes · Top-level script delegates to the Python CLI.
- CLI → Configs: loads · CLI resolves task configuration files and overrides.
- CLI → Wrapper: dispatches · CLI hands execution to the shared session wrapper.
- Configs → Dataset loader: defines · Experiment configs control splits, stacking, padding, and processed paths.
- Configs → Train: drives · Task config parameters shape training and evaluation behavior.
- Raw data → Processed cache: preprocesses · Raw radar files are converted into cached processed datasets.
- Processed cache → Dataset loader: feeds · Dataset loader reuses cached artifacts when available.
- Dataset loader → Model map: supplies · Prepared tensors and labels are passed into model selection and training.
- Model map → MLP: selects · Registry exposes the MLP backbone.
- Model map → DGCNN: selects · Registry exposes the DGCNN backbone.
- Model map → PointNet: selects · Registry exposes the PointNet backbone.
- Model map → PointTrans: selects · Registry exposes the transformer backbone.
- Model map → PointMLP: selects · Registry exposes the PointMLP backbone.
- Wrapper → Train: runs · Training is one of the wrapped execution modes.
- Wrapper → Test: runs · Evaluation is one of the wrapped execution modes.
- Wrapper → Visualize: runs · Visualization is one of the wrapped execution modes.
- Train → Checkpoints: saves · Training persists model artifacts to checkpoints.
- Test → Checkpoints: loads · Evaluation restores a saved checkpoint.
- Visualize → Checkpoints: loads · Visualization uses trained checkpoints for predictions.
- Visualize → Checkpoints: exports · Keypoint visualization can write video outputs alongside model artifacts.
- Install → Configs: prepares · Installation flow supports running the configured experiments.