# pointcept/pointtransformerv3 architecture

> Architecture diagram of the GitHub repository pointcept/pointtransformerv3, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pointcept/pointtransformerv3
- Repository: https://github.com/pointcept/pointtransformerv3

## Overview

##### Project type and purpose
- This is not a full-stack web app or microservice, but a standalone, plug-and-play deep learning model library: “Point Transformer V3” (PTv3).
- Its role is to provide a PyTorch-based Point Transformer encoder (model.py) and plug-in serialization strategies (serialization/default.py, hilbert.py, z_order.py) that can be cloned or copied into a larger “Pointcept” framework (or any custom project) to produce 3D point-cloud features for downstream tasks (semantic segmentation, representation learning, etc.).

##### File structure
- Top-level:
- .gitmodules, LICENSE, README.md – repo metadata and documentation.
- assets/ – teaser images.
- model.py – the entire PTv3 architecture (input→patch embedding→transformer layers→output feature).
- serialization/ – a Python package with interchangeable serialization strategies:
– __init__.py (factory or registry)
– default.py (naïve ordering)
– hilbert.py (Hilbert-curve ordering)
– z_order.py (Morton/Z-order curve ordering)
- The design is modular: model.py depends on a “serialization” interface to convert raw point clouds into a 1D sequence of patches before feeding into the transformer.

##### README for additional insights
- It explains two usage scenarios:
a) Pointcept-driven: clone full Pointcept repo, run training scripts (bash) that load model via import.
b) Custom-framework-driven: copy model.py and serialization/ into your own project, align input dict, and call the encoder to get features.
- Dependencies (FlashAttention, PyTorch ≥1.12, CUDA≥11.6, torch-geometric, spconv, etc.) must be satisfied but are external to this repo.

##### Main components and their interactions
a) Data Input Layer
- Raw point cloud batch (coordinates, features, optional labels) provided as a Python dict.
b) Serialization Module (preprocessor)
- Factory in serialization/__init__.py selects one of default, Hilbert, or Z-order implementations to sort or group points into patches.
- Outputs a tensor of patch embeddings or indices.
c) PTv3 Model Core (model.py)
- Patch embedding (linear + positional encoding → token sequence).
- Transformer encoder layers (multi-head attention, feed-forward, residuals).
- Optional FlashAttention, RPE (controlled by parameters).
- Decoder head or output feature extractor returns per-point or per-patch features.
d) Configuration & Control
- Hyperparameters (patch_size, enable_flash, enable_rpe, upcast flags) drawn from a config object or dict.
- Logging, tensorboard, and checkpointing are managed by the outer framework (Pointcept).
e) External Framework / User Code
- Either Pointcept’s training scripts or a custom training loop imports model.py, calls model(batch), and handles optimizer, loss, evaluation.

##### Architectural patterns & design principles
- Modular preprocessor/plugin pattern for serialization strategies.
- Single-process dataflow pipeline: Input → Serialization → Model → Output.
- Clear separation of concerns: data ordering vs. model architecture vs. training orchestration.
- Config-driven behavior toggles (FlashAttention, RPE).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Project Metadata / Documentation
    subgraph "Project Metadata / Documentation"
        MD1["README.md"]:::external
        MD2["LICENSE"]:::external
        MD3[".gitmodules"]:::external
    end
    click MD1 "https://github.com/pointcept/pointtransformerv3/blob/main/README.md"
    click MD2 "https://github.com/pointcept/pointtransformerv3/tree/main/LICENSE"
    click MD3 "https://github.com/pointcept/pointtransformerv3/blob/main/.gitmodules"

    %% Assets
    subgraph Assets
        IMG["Assets"]:::external
    end
    click IMG "https://github.com/pointcept/pointtransformerv3/tree/main/assets/"

    %% Main Pipeline
    DataInput["Data Input\n(raw point cloud)"]:::dataFlow

    subgraph "Serialization Module"
        direction TB
        Interface["«interface» Serialization Factory"]:::dataFlow
        Default["Default Strategy"]:::dataFlow
        Hilbert["Hilbert Strategy"]:::dataFlow
        Zorder["Z-order Strategy"]:::dataFlow
    end
    click Interface "https://github.com/pointcept/pointtransformerv3/blob/main/serialization/__init__.py"
    click Default "https://github.com/pointcept/pointtransformerv3/blob/main/serialization/default.py"
    click Hilbert "https://github.com/pointcept/pointtransformerv3/blob/main/serialization/hilbert.py"
    click Zorder "https://github.com/pointcept/pointtransformerv3/blob/main/serialization/z_order.py"

    subgraph "PTv3 Model Core"
        direction TB
        ModelCore["PTv3 Model Core"]:::dataFlow
        PatchEmb["Patch Embedding"]:::dataFlow
        Transformer["Transformer Layers"]:::dataFlow
        FeatureHead["Feature Head"]:::dataFlow
    end
    click ModelCore "https://github.com/pointcept/pointtransformerv3/blob/main/model.py"

    OutputFeatures["Output Features\n(encoded features)"]:::dataFlow

    %% Configuration
    Config["Config & Hyperparameters"]:::config

    %% User Framework
    subgraph "User Framework"
        direction TB
        Loader["Data Loader"]:::framework
        Invoke["Pipeline Invocation"]:::framework
        Optimizer["Optimizer & Loss"]:::framework
        Logging["Logging & Checkpointing"]:::framework
        Pointcept["Pointcept Training Scripts"]:::framework
    end
    click Pointcept "https://github.com/pointcept/pointtransformerv3/tree/main/Pointcept/"

    %% External Dependencies
    subgraph "External Dependencies"
        PyTorch["PyTorch & CUDA"]:::external
        FA["FlashAttention"]:::external
        TG["torch-geometric, spconv"]:::external
    end

    %% Data Flow
    Loader --> DataInput
    DataInput --> Interface
    Interface --> Default
    Interface --> Hilbert
    Interface --> Zorder
    Default --> ModelCore
    Hilbert --> ModelCore
    Zorder --> ModelCore
    ModelCore --> PatchEmb
    PatchEmb --> Transformer
    Transformer --> FeatureHead
    FeatureHead --> OutputFeatures
    OutputFeatures --> Optimizer

    Invoke --> DataInput
    Invoke --> Optimizer
    Invoke --> Logging
    Optimizer --> Logging

    %% Config Injection
    Config -.-> Interface
    Config -.-> ModelCore

    %% External Dependencies links
    PyTorch -.-> ModelCore
    FA -.-> ModelCore
    TG -.-> ModelCore

    %% Class Definitions
    classDef dataFlow fill:#D0E7FF,stroke:#4285F4,color:#000;
    classDef config fill:#E6F4EA,stroke:#34A853,color:#000;
    classDef external fill:#FFE5D9,stroke:#FB8C00,color:#000;
    classDef framework fill:#F5F5F5,stroke:#9E9E9E,color:#000;
```
