How casia-iva-lab/dpt works
Project Type and Purpose
• This is an open‐source research codebase implementing “Deformable Patch‐based Transformer (DPT)” for visual recognition.
• It provides two main pipelines:
a. Image Classification
b. Object Detection
• A third component is a custom operator library (ops) that implements the multi‐scale deformable attention kernel in C++/CUDA.
Top‐Level Components (to appear as big blocks)
• Classification Module (classification/)
• Detection Module (detection/)
• Custom Ops Library (ops/)
• External Frameworks & Runtimes
• PyTorch & CUDA
• Python interpreter, third‐party Python libs
Drill into Each Component
A. Classification Module
– Data Loader & Dataset Definitions (classification/mcloader/, classification/datasets.py)
– Model Definitions (classification/models/dpt/, classification/models/pvt.py)
– Training & Evaluation Engine (classification/engine.py, classification/main.py, classification/losses.py, classification/samplers.py)
– Entry Scripts (classification/dist_train.sh, classification/dist_resume.sh)
– Requirements (classification/requirements.txt)
B. Detection Module
– Configuration System (detection/configs/…): base and task‐specific configs
– Model Wrappers & Extensions (detection/dpt_models/, detection/pvt.py)
– Training Script (detection/train.py), Testing (detection/test.py)
– Distributed Launch Scripts (detection/dist_train.sh, detection/dist_test.sh)
C. Ops Library
– Python Interface (ops/functions/, ops/modules/, ops/test.py)
– Build System (ops/setup.py, ops/make.sh)
– Native Kernels (C++/CUDA) under ops/src/
• CPU: *.cpp / *.h
• CUDA: *.cu / *.cuh
Core “DPT” Logic
• The heart of the architecture is the DePatch Transformer block (in models/dpt/box_coder.py, depatch_embed.py, dpt.py)
• Both the classification and detection pipelines import and use these modules
• They also invoke the custom multi‐scale deformable attention operator from ops/
Architectural Patterns
• Modular, plug‐and‐play design: the DePatch block is packaged so it can be used in different vision tasks
• Config‐driven design in detection: can swap backbones or training schedules via config files
• Custom native operator compiled on install, loaded at runtime via Python bindings
Final Checklist
[ ] File path references or module names as labels