# casia-iva-lab/dpt architecture

> Architecture diagram of the GitHub repository casia-iva-lab/dpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/casia-iva-lab/dpt
- Repository: https://github.com/casia-iva-lab/dpt

## Overview

##### Project Type and Purpose
- This is an open‐source research codebase implementing “Deformable Patch‐based Transformer (DPT)” for visual recognition.
- It provides two main pipelines:
a. Image Classification
b. Object Detection
- A third component is a custom operator library (`ops`) that implements the multi‐scale deformable attention kernel in C++/CUDA.

##### Top‐Level Components (to appear as big blocks)
- Classification Module (`classification/`)
- Detection Module (`detection/`)
- Custom Ops Library (`ops/`)
- External Frameworks & Runtimes
- PyTorch & CUDA
- Python interpreter, third‐party Python libs

##### Drill into Each Component
A. Classification Module
– Data Loader & Dataset Definitions (`classification/mcloader/`, `classification/datasets.py`)
– Model Definitions (`classification/models/dpt/`, `classification/models/pvt.py`)
– Training & Evaluation Engine (`classification/engine.py`, `classification/main.py`, `classification/losses.py`, `classification/samplers.py`)
– Entry Scripts (`classification/dist_train.sh`, `classification/dist_resume.sh`)
– Requirements (`classification/requirements.txt`)
B. Detection Module
– Configuration System (`detection/configs/…`): base and task‐specific configs
– Model Wrappers & Extensions (`detection/dpt_models/`, `detection/pvt.py`)
– Training Script (`detection/train.py`), Testing (`detection/test.py`)
– Distributed Launch Scripts (`detection/dist_train.sh`, `detection/dist_test.sh`)
C. Ops Library
– Python Interface (`ops/functions/`, `ops/modules/`, `ops/test.py`)
– Build System (`ops/setup.py`, `ops/make.sh`)
– Native Kernels (C++/CUDA) under `ops/src/`
- CPU: `*.cpp / *.h`
- CUDA: `*.cu / *.cuh`

##### Core “DPT” Logic
- The heart of the architecture is the DePatch Transformer block (in `models/dpt/box_coder.py`, `depatch_embed.py`, `dpt.py`)
- Both the classification and detection pipelines import and use these modules
- They also invoke the custom multi‐scale deformable attention operator from `ops/`

##### Architectural Patterns
- Modular, plug‐and‐play design: the DePatch block is packaged so it can be used in different vision tasks
- Config‐driven design in detection: can swap backbones or training schedules via config files
- Custom native operator compiled on install, loaded at runtime via Python bindings

##### Final Checklist
[ ] File path references or module names as labels

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Top-level components
    subgraph "Classification Pipeline"
        direction TB
        CL_Data["Data Loader & Datasets"]:::classification
        CL_Model["Model Definitions (DPT & PVT)"]:::classification
        CL_Engine["Training & Evaluation Engine"]:::classification
        CL_Scripts["Entry Scripts"]:::classification
        CL_Req["Requirements"]:::classification
    end

    subgraph "Detection Pipeline"
        direction TB
        DET_Config["Configuration System"]:::detection
        DET_Model["Model Wrappers & Extensions"]:::detection
        DET_Scripts["Training & Testing Scripts"]:::detection
        DET_Dist["Distributed Launch Scripts"]:::detection
    end

    subgraph "Ops Library (C++/CUDA)"
        direction TB
        OPS_Py["Python Interface"]:::ops
        OPS_Build["Build System"]:::ops
        OPS_Native["Native Kernels (C++/CUDA)"]:::ops
    end

    subgraph "External Frameworks & Runtimes"
        direction TB
        EXT_PT["PyTorch & CUDA"]:::external
        EXT_PY["Python Interpreter & Libs"]:::external
    end

    subgraph "Core DPT Logic" 
        direction TB
        DPT_Core["DePatch Transformer Block"]:::core
    end

    %% Connections within Classification
    CL_Data -->|"feeds"| CL_Model
    CL_Model -->|"training"| CL_Engine
    CL_Engine --> CL_Scripts
    CL_Engine -->|"requires"| CL_Req

    %% Connections within Detection
    DET_Config -->|"config load"| DET_Model
    DET_Model -->|"train/test"| DET_Scripts
    DET_Scripts --> DET_Dist

    %% Cross-component connections
    CL_Model -->|"uses core logic"| DPT_Core
    DET_Model -->|"uses core logic"| DPT_Core
    DPT_Core -->|"calls operator"| OPS_Py
    CL_Model -->|"operator call"| OPS_Py
    DET_Model -->|"operator call"| OPS_Py
    CL_Engine -->|"executes on"| EXT_PT
    DET_Scripts -->|"executes on"| EXT_PT
    CL_Engine -->|"scripts run on"| EXT_PY
    DET_Dist -->|"launches via"| EXT_PY

    %% Ops Library internal
    OPS_Build --> OPS_Native
    OPS_Py --> OPS_Native

    %% Click Events
    click CL_Data "https://github.com/casia-iva-lab/dpt/tree/main/classification/mcloader/"
    click CL_Data "https://github.com/casia-iva-lab/dpt/blob/main/classification/datasets.py"
    click CL_Model "https://github.com/casia-iva-lab/dpt/tree/main/classification/models/dpt/"
    click CL_Engine "https://github.com/casia-iva-lab/dpt/blob/main/classification/engine.py"
    click CL_Scripts "https://github.com/casia-iva-lab/dpt/blob/main/classification/dist_train.sh"
    click CL_Scripts "https://github.com/casia-iva-lab/dpt/blob/main/classification/dist_resume.sh"
    click CL_Req "https://github.com/casia-iva-lab/dpt/blob/main/classification/requirements.txt"
    click DET_Config "https://github.com/casia-iva-lab/dpt/tree/main/detection/configs/"
    click DET_Model "https://github.com/casia-iva-lab/dpt/tree/main/detection/dpt_models/"
    click DET_Model "https://github.com/casia-iva-lab/dpt/blob/main/detection/pvt.py"
    click DET_Scripts "https://github.com/casia-iva-lab/dpt/blob/main/detection/train.py"
    click DET_Scripts "https://github.com/casia-iva-lab/dpt/blob/main/detection/test.py"
    click DET_Dist "https://github.com/casia-iva-lab/dpt/blob/main/detection/dist_train.sh"
    click DET_Dist "https://github.com/casia-iva-lab/dpt/blob/main/detection/dist_test.sh"
    click OPS_Py "https://github.com/casia-iva-lab/dpt/tree/main/ops/functions/"
    click OPS_Py "https://github.com/casia-iva-lab/dpt/tree/main/ops/modules/"
    click OPS_Py "https://github.com/casia-iva-lab/dpt/blob/main/ops/test.py"
    click OPS_Build "https://github.com/casia-iva-lab/dpt/blob/main/ops/setup.py"
    click OPS_Build "https://github.com/casia-iva-lab/dpt/blob/main/ops/make.sh"
    click OPS_Native "https://github.com/casia-iva-lab/dpt/tree/main/ops/src/"

    %% Styles
    classDef classification fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef detection fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef core fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef ops fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px;
```
