How jiang-hb/diffusionreg works
1. What this repo is
This is a PyTorch implementation of an SE(3) diffusion-based point cloud registration system for 6D object pose estimation. The core idea is to treat pose estimation as iterative denoising of a rigid transform between a source point cloud and an object model point cloud.
2. Top-level execution boundary
The runtime entry points are train.py and test.py. These scripts are the orchestration layer: they load configuration, build the dataset, instantiate the registration/diffusion components, run optimization or evaluation, and write checkpoints/results into results/.
3. Data subsystem
Dataset handling is split between datasets/get_dataset.py and datasets/tudl_db.py. The repo includes a concrete TUD-L dataset layout under datasets/tudl/, with:
- raw/processed object models in .ply
- metadata files such as models_info.json
- cached preprocessing artifacts in *.pth
This indicates the code relies on preprocessed dataset descriptors rather than reading only raw BOP-style files at runtime.
4. Dataset boundary and external dependency
The data pipeline is tied to BOP-format object pose data, specifically TUD-L in the checked-in tree. The README mentions LINEMOD and Occluded-LINEMOD support conceptually, but only TUD-L assets are present here. So the repo’s architecture is dataset-agnostic in code structure, but the included data boundary is TUD-L-centric.
5. Registration backbone
The main registration network lives under modules/DCP/dcp.py, with supporting feature extraction utilities in modules/DCP/rpmnet_emb/feature_net.py and pointnet_util.py. This suggests the diffusion framework is built around a deep point-cloud registration backbone, using a DCP-style architecture and RPMNet-derived embedding components as the surrogate alignment model.
6. Diffusion and pose refinement logic
The diffusion process itself is not isolated in a separate model directory, but is supported by utilities such as utils/diffusion_scheduler.py, utils/losses.py, and utils/criterion.py. Architecturally, these files form the training-time control layer for:
- adding SE(3) noise/perturbation
- scheduling the reverse denoising steps
- computing the registration-specific objective
7. SE(3)/Lie algebra math layer
The utils/se_math/ package is a distinct mathematical boundary for rigid-body transforms on the manifold:
- se3.py, so3.py for transformations and rotations
- transforms.py, mesh.py, invmat.py, sinc.py for supporting operations
This layer is important because the method operates in SE(3) rather than Euclidean parameter space, so pose perturbation and recovery are manifold-aware.
8. Training infrastructure
utils/options.py, utils/get_lr_scheduler.py, utils/commons.py, and utils/attr_dict.py provide experiment configuration, learning-rate scheduling, and shared helpers. Together they form the experiment-management layer used by train.py to assemble the full system.
9. Evaluation and outputs
The results/ directory shows the output boundary of the system:
- trained checkpoints such as model_epoch19.pth
- per-object evaluation artifacts under eval_results/
This implies evaluation is checkpoint-driven and can emit object-specific pose outputs or metrics for each test object instance.
10. End-to-end data flow
The main flow is:
- dataset loaders read preprocessed TUD-L assets
- a point-cloud registration backbone produces pose refinements
- the diffusion scheduler perturbs and then denoises SE(3) transforms
- SE(3) math utilities enforce rigid-transform correctness
- losses/criteria optimize the reverse process
- checkpoints and evaluation outputs are written to results/
11. Architectural separation
The repo is cleanly separated into:
- data access: datasets/
- model/backbone: modules/DCP/
- math/manifold ops: utils/se_math/
- training/evaluation control: train.py, test.py, and utility modules
This makes the diffusion framework modular: the backbone can be treated as a surrogate registration network while the SE(3) diffusion machinery sits around it as a higher-level refinement process.
Main entry points: train.py train.py · se3.py SE(3) · test.py test.py · dcp.py DCP
Components 18
Execution
Entry points and orchestration
Data
Dataset loading and checked-in assets
- Get dataset · Selects and constructs the dataset implementation
datasets/get_dataset.py - TUD-L DB · Dataset-specific access layer for TUD-L pose data
datasets/tudl_db.py - TUD-L assets · Checked-in object models, metadata, and cached preprocessing artifacts
datasets/tudl/models/models_info.json
Model
Registration backbone and embeddings
- DCP · Main deep point-cloud registration network used for pose refinement
modules/DCP/dcp.py - Feature net · Point-cloud feature extraction used by the registration backbone
modules/DCP/rpmnet_emb/feature_net.py - PointNet utils · Shared point-cloud sampling and neighborhood utilities
modules/DCP/rpmnet_emb/pointnet_util.py
SE(3) math
Rigid transform and manifold utilities
- SE(3) · Rigid-transform operations on the SE(3) manifold
utils/se_math/se3.py - SO(3) · Rotation algebra and pose-relevant rotational operations
utils/se_math/so3.py - Transforms · Supporting transform conversions and rigid-body utilities
utils/se_math/transforms.py
Training
Optimization, scheduling, and losses
- Scheduler · Controls SE(3) noise levels and reverse denoising steps
utils/diffusion_scheduler.py - Criterion · Registration and diffusion-specific training criterion
utils/criterion.py - Losses · Objective terms for pose refinement and alignment
utils/losses.py - Options · Experiment and runtime configuration parsing
utils/options.py - LR schedule · Learning-rate scheduling helpers
utils/get_lr_scheduler.py - Commons · General-purpose training and evaluation utilities
utils/commons.py
Outputs
Checkpoints and evaluation artifacts
- Results · Saved checkpoints and per-object evaluation outputs
results/DiffusionReg-DiffusionDCP-tudl-diffusion_200_0.00010_0.05_0.05_0.03-nvids3_cosine/model_epoch19.pth
Connections 21
- train.py → Options: loads config · Training starts by parsing experiment settings
- train.py → Get dataset: builds data · Training constructs the dataset pipeline
- train.py → DCP: instantiates · Training creates the registration backbone
- train.py → Scheduler: sets schedule · Training uses the diffusion scheduler for perturbation and denoising
- train.py → Criterion: optimizes · Training computes the main training criterion
- train.py → LR schedule: updates lr · Training applies learning-rate scheduling
- train.py → Results: writes checkpoints · Training persists model checkpoints and run artifacts
- test.py → Options: loads config · Evaluation reuses the experiment configuration
- test.py → Get dataset: builds data · Evaluation loads the same dataset boundary
- test.py → Results: writes evals · Evaluation emits per-object output artifacts
- Get dataset → TUD-L DB: uses · Dataset factory delegates to the TUD-L backend
- TUD-L DB → TUD-L assets: reads · The backend reads checked-in object models and metadata
- DCP → Feature net: uses · The registration model depends on learned point features
- DCP → PointNet utils: uses · The backbone uses point-cloud utility operations
- train.py → SE(3): constrains · Training works in rigid-transform space rather than plain Euclidean parameters
- SE(3) → SO(3): depends on · SE(3) utilities build on rotation algebra
- SE(3) → Transforms: supports · Rigid-body operations rely on transform helpers
- Scheduler → SE(3): perturbs · The scheduler injects and removes pose noise in SE(3)
- Criterion → Losses: composes · The criterion aggregates training losses
- Criterion → SE(3): evaluates · Losses are computed over rigid transforms and pose residuals
- Commons → Results: supports output · Shared utilities help manage experiment artifacts