# nahyuklee/pmtr architecture

> Architecture diagram of the GitHub repository nahyuklee/pmtr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nahyuklee/pmtr
- Repository: https://github.com/nahyuklee/pmtr

## Overview

##### File structure
- Top-level scripts:
- main.py (training entrypoint)
- test.py (evaluation entrypoint)
- setup.py (compiles/installs C++ extensions)
- common/:
- Python utilities (sampling.py, rotation.py, utils.py, node.py)
- common/extensions/: C++/PyBind kernels for grid subsampling, radius neighbor search, nanoflann, cloud operations, plus torch_helper.h and pybind.cpp.
- data/:
- dataset.py and utils.py (data loader, augmentation, file-list parsing)
- data_list/ (train/val splits for “everyday” and “artifact” subsets)
- assets/ (figures, architecture image)
- model/:
- backbone/kpconv/: KPConv implementation (functional.py, modules.py, kernel points, dispositions)
- backbone/kpconvfpn.py (feature pyramid network)
- learnable_sinkhorn.py (coarse matching module)
- local_global_registration.py (pose estimation)
- pmt.py (fine matcher)
- pmtr.py (end-to-end assembler)
- loss.py (loss definitions)
- test.py / main.py: orchestrate Data → Model → Loss → Logging/Checkpoint.

##### README insights
- Dependencies: Python 3.8, PyTorch 1.10.1, PyTorch Lightning 1.9, plus third-party libs (einops, open3d, wandb, trimesh, torch-batch-svd, chamfer_distance, pytorch3d).

a. Main components:
- Data Layer: raw point clouds, train/val lists, data augmentations.
- Preprocessing Extensions: C++ grid_subsampling & radius_neighbors modules, nanoflann, cloud utilities.
- Model Backbone: KPConv + FPN for feature extraction.
- Coarse Matching: learnable Sinkhorn module.
- Fine Matching/Assembly: PMT/PMTR module → local+global registration.
- Loss & Evaluation: loss.py, test.py.
- Orchestration: main.py (PyTorch Lightning Trainer), test.py.
- External Services: WandB logging, checkpoint storage, GPU resources.
b. Relationships and data flow:
1. Raw point clouds + labels → dataset.py (Data Layer).
2. Batches passed to C++ extensions for subsampling/neighborhood graphs.
3. Processed points → KPConv backbone → multi-scale features.
4. Features → Sinkhorn coarse matcher → initial correspondences.
5. Correspondences + features → PMT/PMTR fine matcher → refined correspondences.
6. Correspondences → local_global_registration → predicted transformation.
7. Predictions → loss.py → backprop → checkpoint & WandB logs.
8. test.py uses the same pipeline for inference & metric computation.
c. Architectural patterns:
- Modular Pipeline: strict separation of Data, Preprocessing, Model, Matching, Registration, Loss, and Orchestration.
- Cross‐language Modules: Python core + compiled C++ kernels.
- Feature Pyramid (FPN) for multi-scale representation.
- Differentiable matching layers (Sinkhorn, transformer-based PMT).
- PyTorch Lightning for training loop abstraction.
d. Key technologies/frameworks:
- Python 3.8, PyTorch 1.10.1, PyTorch Lightning 1.9
- C++ + PyBind for performance-critical ops
- External libs: einops, open3d, wandb, trimesh, torch-batch-svd, chamfer_distance, pytorch3d

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        RawData["Raw Point Clouds\n(data/assets)"]:::data
        Dataset["dataset.py"]:::data
        UtilsData["utils.py"]:::data
        subgraph "Data Lists"
            direction TB
            ArtifactTrain["artifact_train.txt"]:::data
            ArtifactVal["artifact_val.txt"]:::data
            EverydayTrain["everyday_train.txt"]:::data
            EverydayVal["everyday_val.txt"]:::data
        end
    end

    %% Preprocessing Extensions
    subgraph "Preprocessing Extensions (C++/PyBind)" 
        direction TB
        Pybind["pybind.cpp"]:::preprocess
        TorchHelper["torch_helper.h"]:::preprocess
        subgraph "Grid Subsampling"
            direction TB
            GS1["grid_subsampling.cpp"]:::preprocess
            GS2["grid_subsampling.h"]:::preprocess
            GS3["grid_subsampling_cpu.cpp"]:::preprocess
            GS4["grid_subsampling_cpu.h"]:::preprocess
        end
        subgraph "Radius Neighbors"
            direction TB
            RN1["radius_neighbors.cpp"]:::preprocess
            RN2["radius_neighbors.h"]:::preprocess
            RN3["radius_neighbors_cpu.cpp"]:::preprocess
            RN4["radius_neighbors_cpu.h"]:::preprocess
        end
        Cloud["cloud.cpp"]:::preprocess
        CloudH["cloud.h"]:::preprocess
        Nanoflann["nanoflann.hpp"]:::preprocess
    end

    %% Model Layer
    subgraph "Model Layer"
        direction TB
        subgraph "Backbone (KPConv + FPN)" 
            direction TB
            KPFunctional["functional.py"]:::model
            KPKernel["kernel_points.py"]:::model
            KPConvMod["kpconv.py"]:::model
            KPModules["modules.py"]:::model
            KPPly["k_015_center_3D.ply"]:::model
            FPN["kpconvfpn.py"]:::model
        end
        Coarse["learnable_sinkhorn.py"]:::model
        FinePMT["pmt.py"]:::model
        FinePMTR["pmtr.py"]:::model
        Registration["local_global_registration.py"]:::model
        Loss["loss.py"]:::model
    end

    %% Orchestration & IO
    subgraph "Orchestration & I/O"
        direction TB
        Main["main.py\n(PyTorch Lightning Trainer)"]:::orchestr
        TestScript["test.py\n(Evaluation)"]:::orchestr
        Setup["setup.py"]:::orchestr
        Readme["README.md"]:::orchestr
        Checkpoints["Checkpoint Storage"]:::ext
        WandB["WandB Logging"]:::ext
        GPU["GPU Resources"]:::ext
    end

    %% Data Flow
    RawData -->|reads| Dataset
    ArtifactTrain -->|list| Dataset
    ArtifactVal -->|list| Dataset
    EverydayTrain -->|list| Dataset
    EverydayVal -->|list| Dataset
    Dataset -->|uses utils| UtilsData

    Dataset -->|batch point clouds| GS1
    Dataset -->|batch point clouds| GS2
    Dataset -->|batch point clouds| GS3
    Dataset -->|batch point clouds| GS4
    Dataset -->|batch point clouds| RN1
    Dataset -->|batch point clouds| RN2
    Dataset -->|batch point clouds| RN3
    Dataset -->|batch point clouds| RN4
    Dataset -->|cloud ops| Cloud
    Dataset -->|cloud ops| CloudH
    Dataset -->|neighbor search| Nanoflann
    Dataset -->|PyBind init| Pybind
    Dataset -->|helper| TorchHelper

    GS1 -->|subsampled pts| KPFunctional
    GS2 -->|subsampled pts| KPFunctional
    GS3 -->|subsampled pts| KPFunctional
    GS4 -->|subsampled pts| KPFunctional
    RN1 -->|neighbor graphs| KPFunctional
    RN2 -->|neighbor graphs| KPFunctional
    RN3 -->|neighbor graphs| KPFunctional
    RN4 -->|neighbor graphs| KPFunctional
    Cloud -->|cloud utils| KPFunctional
    Nanoflann -->|search utils| KPFunctional

    KPFunctional -->|features| FPN
    KPKernel -->|kernel pts| KPConvMod
    KPConvMod -->|features| KPModules
    KPModules -->|multi-scale feats| FPN
    FPN -->|feature tensors| Coarse

    Coarse -->|corr. matrix| FinePMT
    FinePMT -->|refined corr.| FinePMTR
    FinePMTR -->|corr. + feats| Registration
    Registration -->|transformation| Loss

    Loss -->|loss value| Main
    Main -->|checkpoint & logs| Checkpoints
    Main -->|logs| WandB
    Main -->|runs on| GPU

    TestScript -->|inference pipeline| GS1
    TestScript -->|uses model| FPN
    TestScript -->|computes metrics| Loss

    Setup -->|build/extensions| Pybind
    Readme -->|docs| Main

    %% Click Events
    click ArtifactTrain "https://github.com/nahyuklee/pmtr/blob/main/data/data_list/artifact_train.txt"
    click ArtifactVal "https://github.com/nahyuklee/pmtr/blob/main/data/data_list/artifact_val.txt"
    click EverydayTrain "https://github.com/nahyuklee/pmtr/blob/main/data/data_list/everyday_train.txt"
    click EverydayVal "https://github.com/nahyuklee/pmtr/blob/main/data/data_list/everyday_val.txt"
    click Dataset "https://github.com/nahyuklee/pmtr/blob/main/data/dataset.py"
    click UtilsData "https://github.com/nahyuklee/pmtr/blob/main/data/utils.py"
    click Setup "https://github.com/nahyuklee/pmtr/blob/main/setup.py"
    click Pybind "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/pybind.cpp"
    click TorchHelper "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/common/torch_helper.h"
    click GS1 "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/cpu/grid_subsampling/grid_subsampling.cpp"
    click GS2 "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/cpu/grid_subsampling/grid_subsampling.h"
    click GS3 "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/cpu/grid_subsampling/grid_subsampling_cpu.cpp"
    click GS4 "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/cpu/grid_subsampling/grid_subsampling_cpu.h"
    click RN1 "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/cpu/radius_neighbors/radius_neighbors.cpp"
    click RN2 "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/cpu/radius_neighbors/radius_neighbors.h"
    click RN3 "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/cpu/radius_neighbors/radius_neighbors_cpu.cpp"
    click RN4 "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/cpu/radius_neighbors/radius_neighbors_cpu.h"
    click Cloud "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/extra/cloud/cloud.cpp"
    click CloudH "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/extra/cloud/cloud.h"
    click Nanoflann "https://github.com/nahyuklee/pmtr/blob/main/common/extensions/extra/nanoflann/nanoflann.hpp"
    click KPFunctional "https://github.com/nahyuklee/pmtr/blob/main/model/backbone/kpconv/functional.py"
    click KPKernel "https://github.com/nahyuklee/pmtr/blob/main/model/backbone/kpconv/kernel_points.py"
    click KPConvMod "https://github.com/nahyuklee/pmtr/blob/main/model/backbone/kpconv/kpconv.py"
    click KPModules "https://github.com/nahyuklee/pmtr/blob/main/model/backbone/kpconv/modules.py"
    click KPPly "https://github.com/nahyuklee/pmtr/blob/main/model/backbone/kpconv/dispositions/k_015_center_3D.ply"
    click FPN "https://github.com/nahyuklee/pmtr/blob/main/model/backbone/kpconvfpn.py"
    click Coarse "https://github.com/nahyuklee/pmtr/blob/main/model/learnable_sinkhorn.py"
    click FinePMT "https://github.com/nahyuklee/pmtr/blob/main/model/pmt.py"
    click FinePMTR "https://github.com/nahyuklee/pmtr/blob/main/model/pmtr.py"
    click Registration "https://github.com/nahyuklee/pmtr/blob/main/model/local_global_registration.py"
    click Loss "https://github.com/nahyuklee/pmtr/blob/main/model/loss.py"
    click TestScript "https://github.com/nahyuklee/pmtr/blob/main/test.py"
    click Main "https://github.com/nahyuklee/pmtr/blob/main/main.py"
    click Readme "https://github.com/nahyuklee/pmtr/blob/main/README.md"

    %% Styles
    classDef data fill:#D0E8FF,stroke:#0366D6,color:#0366D6
    classDef preprocess fill:#D8FFE0,stroke:#2E8B57,color:#2E8B57
    classDef model fill:#FFD8A8,stroke:#CD6600,color:#CD6600
    classDef orchestr fill:#E8D0FF,stroke:#6A0DAD,color:#6A0DAD
    classDef ext fill:#F0F0F0,stroke:#888888,color:#444444
```
