How nahyuklee/pmtr works
File structure
• Top-level scripts:
• main.py (training entrypoint)
• test.py (evaluation entrypoint)
• setup.py (compiles/installs C++ extensions)
• common/:
• Python utilities (sampling.py, rotation.py, utils.py, node.py)
• common/extensions/: C++/PyBind kernels for grid subsampling, radius neighbor search, nanoflann, cloud operations, plus torch_helper.h and pybind.cpp.
• data/:
• dataset.py and utils.py (data loader, augmentation, file-list parsing)
• data_list/ (train/val splits for “everyday” and “artifact” subsets)
• assets/ (figures, architecture image)
• model/:
• backbone/kpconv/: KPConv implementation (functional.py, modules.py, kernel points, dispositions)
• backbone/kpconvfpn.py (feature pyramid network)
• learnable_sinkhorn.py (coarse matching module)
• local_global_registration.py (pose estimation)
• pmt.py (fine matcher)
• pmtr.py (end-to-end assembler)
• loss.py (loss definitions)
• test.py / main.py: orchestrate Data → Model → Loss → Logging/Checkpoint.
README insights
• Dependencies: Python 3.8, PyTorch 1.10.1, PyTorch Lightning 1.9, plus third-party libs (einops, open3d, wandb, trimesh, torch-batch-svd, chamfer_distance, pytorch3d).
a. Main components:
• Data Layer: raw point clouds, train/val lists, data augmentations.
• Preprocessing Extensions: C++ grid_subsampling & radius_neighbors modules, nanoflann, cloud utilities.
• Model Backbone: KPConv + FPN for feature extraction.
• Coarse Matching: learnable Sinkhorn module.
• Fine Matching/Assembly: PMT/PMTR module → local+global registration.
• Loss & Evaluation: loss.py, test.py.
• Orchestration: main.py (PyTorch Lightning Trainer), test.py.
• External Services: WandB logging, checkpoint storage, GPU resources.
b. Relationships and data flow:
1. Raw point clouds + labels → dataset.py (Data Layer).
2. Batches passed to C++ extensions for subsampling/neighborhood graphs.
3. Processed points → KPConv backbone → multi-scale features.
4. Features → Sinkhorn coarse matcher → initial correspondences.
5. Correspondences + features → PMT/PMTR fine matcher → refined correspondences.
6. Correspondences → local_global_registration → predicted transformation.
7. Predictions → loss.py → backprop → checkpoint & WandB logs.
8. test.py uses the same pipeline for inference & metric computation.
c. Architectural patterns:
• Modular Pipeline: strict separation of Data, Preprocessing, Model, Matching, Registration, Loss, and Orchestration.
• Cross‐language Modules: Python core + compiled C++ kernels.
• Feature Pyramid (FPN) for multi-scale representation.
• Differentiable matching layers (Sinkhorn, transformer-based PMT).
• PyTorch Lightning for training loop abstraction.
d. Key technologies/frameworks:
• Python 3.8, PyTorch 1.10.1, PyTorch Lightning 1.9
• C++ + PyBind for performance-critical ops
• External libs: einops, open3d, wandb, trimesh, torch-batch-svd, chamfer_distance, pytorch3d