# kuanchihhuang/ptt architecture

> Architecture diagram of the GitHub repository kuanchihhuang/ptt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kuanchihhuang/ptt
- Repository: https://github.com/kuanchihhuang/ptt

## Overview

##### Project type and purpose
- This is a research‐oriented, open‐source deep-learning pipeline for temporal 3D object detection on LiDAR data. It extends the OpenPCDet framework with a Point-Trajectory Transformer (PTT) model targeting Waymo and other autonomous-driving datasets.

##### File structure
- Top-level directories:
- pcdet/      ← core library (config, datasets, models, ops, utils)
- tools/      ← entry‐point scripts (train.py, test.py, demo.py), configs, eval & visual utils
- docker/     ← containerization
- docs/       ← tutorials, changelog
- pcdet/datasets: loaders & augmentors for Waymo, KITTI, NuScenes, custom, etc.
- pcdet/models: backbones (2D/3D), VFE modules, dense heads, detector templates, ROI heads (including PTT head).
- pcdet/ops: custom CUDA/C++ kernels (IoU NMS, PointNet2, ROI‐aware pooling).
- tools/cfgs: YML files describing model hyperparameters and dataset paths.
- tools/train.py, test.py: orchestrate data loading, model instantiation, distributed training/inference.

##### README for additional insights
- Confirms PTT builds on OpenPCDet; requires pretraining CenterPoint RPN; then trains PTT using ROI proposals from earlier model.
- Workflow:
a) Preprocess raw Waymo data with OpenPCDet
b) Run CenterPoint (4 frames) → outputs ROI pickle files
c) Train PTT (32 frames) using those pickles
d) Evaluate/test with test.py

##### Final advice
- Every stage: data, model, ops, training scripts, environment.
- Do not over-abstract; each major folder is its own block.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Sources
    subgraph "Data Sources"
        direction TB
        RawPointClouds["Raw Point Clouds"]:::external
        ROIProposals["Precomputed ROI Proposals (.pkl)"]:::external
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        WaymoLoader["WaymoDataset Loader"]:::python
        DataAugmentor["Data Augmentor"]:::python
        DataProcessor["Data Processor"]:::python
    end

    %% Model Core
    subgraph "Model Core"
        direction TB

        subgraph "Feature Encoding"
            direction TB
            VFE["Voxel Feature Encoder (PillarVFE)"]:::python
            SparseBackbone["SparseConv3D Backbone"]:::python
            PointPillar["PointPillar BEV Backbone"]:::python
        end

        subgraph "Proposal Heads"
            direction TB
            CenterHead["CenterPoint Head (RPN)"]:::python
            AnchorHead["Anchor Head"]:::python
            VoxelNext["VoxelNeXt Head"]:::python
        end

        subgraph "ROI Heads"
            direction TB
            ROIHeadTemplate["ROI Head Template"]:::python
            PTTHead["PTT Transformer Head"]:::python
        end

        subgraph "Detectors"
            direction TB
            CenterPointDet["CenterPoint Detector"]:::python
            MPPNet["PTT-based Detector"]:::python
        end
    end

    %% Custom Ops
    subgraph "Custom CUDA Ops"
        direction TB
        IoUNMS["IoU3D NMS"]:::cuda
        PointNet2["PointNet2 Sampling"]:::cuda
        ROIPool["ROI-Aware Pooling"]:::cuda
    end

    %% Training & Inference
    subgraph "Training & Inference"
        direction TB
        TrainScript["train.py"]:::python
        TestScript["test.py"]:::python
        EvalUtils["eval_utils.py"]:::python
    end

    %% Configuration & CLI
    subgraph "Configuration & CLI"
        direction TB
        InitPath["_init_path.py"]:::config
        ConfigPy["config.py"]:::config
        YAMLConfig["ptt_32frames.yaml"]:::config
    end

    %% Deployment
    subgraph "Docker Container" style docker
        direction TB
        Dockerfile["Dockerfile"]:::docker
    end

    %% Connections
    RawPointClouds --> WaymoLoader
    ROIProposals --> WaymoLoader
    WaymoLoader --> DataAugmentor
    DataAugmentor --> DataProcessor
    DataProcessor --> VFE
    VFE --> SparseBackbone
    VFE --> PointPillar
    SparseBackbone --> CenterHead
    PointPillar --> CenterHead
    CenterHead --> ROIHeadTemplate
    AnchorHead --> ROIHeadTemplate
    VoxelNext --> ROIHeadTemplate
    ROIHeadTemplate --> PTTHead
    PTTHead --> MPPNet
    CenterPointDet --> MPPNet
    CenterHead -->|"uses"| IoUNMS
    ROIHeadTemplate -->|"samples"| PointNet2
    ROIHeadTemplate -->|"pools"| ROIPool

    TrainScript --> WaymoLoader
    TrainScript --> MPPNet
    TrainScript --> EvalUtils
    TestScript --> WaymoLoader
    TestScript --> MPPNet
    TestScript --> EvalUtils
    TrainScript --> InitPath
    TrainScript --> ConfigPy
    TrainScript --> YAMLConfig

    %% Click Events
    click WaymoLoader "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/datasets/waymo/waymo_dataset.py"
    click DataAugmentor "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/datasets/augmentor/data_augmentor.py"
    click DataProcessor "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/datasets/processor/data_processor.py"
    click SparseBackbone "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/backbones_3d/spconv_backbone.py"
    click PointPillar "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/backbones_2d/base_bev_backbone.py"
    click PointPillar "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/backbones_2d/map_to_bev/pointpillar_scatter.py"
    click VFE "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/backbones_3d/vfe/pillar_vfe.py"
    click CenterHead "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/dense_heads/center_head.py"
    click AnchorHead "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/dense_heads/anchor_head_multi.py"
    click VoxelNext "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/dense_heads/voxelnext_head.py"
    click PTTHead "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/roi_heads/ptt_head.py"
    click MPPNet "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/detectors/mppnet.py"
    click MPPNet "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/detectors/mppnet_e2e.py"
    click CenterPointDet "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/detectors/centerpoint.py"
    click ROIHeadTemplate "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/models/roi_heads/roi_head_template.py"
    click IoUNMS "https://github.com/kuanchihhuang/ptt/tree/main/pcdet/ops/iou3d_nms"
    click PointNet2 "https://github.com/kuanchihhuang/ptt/tree/main/pcdet/ops/pointnet2/pointnet2_batch"
    click PointNet2 "https://github.com/kuanchihhuang/ptt/tree/main/pcdet/ops/pointnet2/pointnet2_stack"
    click ROIPool "https://github.com/kuanchihhuang/ptt/tree/main/pcdet/ops/roiaware_pool3d"
    click ROIPool "https://github.com/kuanchihhuang/ptt/tree/main/pcdet/ops/roipoint_pool3d"
    click TrainScript "https://github.com/kuanchihhuang/ptt/blob/main/tools/train.py"
    click TestScript "https://github.com/kuanchihhuang/ptt/blob/main/tools/test.py"
    click EvalUtils "https://github.com/kuanchihhuang/ptt/blob/main/tools/eval_utils/eval_utils.py"
    click InitPath "https://github.com/kuanchihhuang/ptt/blob/main/tools/_init_path.py"
    click ConfigPy "https://github.com/kuanchihhuang/ptt/blob/main/pcdet/config.py"
    click YAMLConfig "https://github.com/kuanchihhuang/ptt/blob/main/tools/cfgs/waymo_models/ptt_32frames.yaml"
    click Dockerfile "https://github.com/kuanchihhuang/ptt/tree/main/docker/Dockerfile"

    %% Styles
    classDef python fill:#D0E8FF,stroke:#0366D6,stroke-width:1px
    classDef cuda fill:#FFD3B4,stroke:#D9631A,stroke-width:1px
    classDef config fill:#E6F4EA,stroke:#22863A,stroke-width:1px
    classDef docker fill:#F0F0F0,stroke:#666666,stroke-width:2px,stroke-dasharray: 5 5
    classDef external fill:#F5F5F5,stroke:#999999,stroke-width:1px
```
