How pnnnnnnn/uni-prevpredmap works
Project Type and Purpose
• Uni-PrevPredMap is a research-grade, open-source framework for online vectorized HD map construction.
• It builds on two foundations: the mmdetection3d core library (for 3D object detection pipelines) and MapTRv2 (for BEV-based map prediction).
• The key innovation is a unified, prior-informed modeling pipeline that fuses previous model predictions and simulated outdated HD maps in three modes (non-prior, temporal-prior, temporal-map-fusion).
High-Level File Structure and Architectural Choices
• Top-level directories:
• assets, docs, figs – static assets and documentation
• mmdetection3d – core library (datasets, models, ops, apis, training/test scripts, CI, docs)
• projects – research projects (MapTR, MapTRv2, Uni-PrevPredMap configs and plugin code)
• tools – utility scripts (data conversion, training wrappers, benchmarking, visualization)
• Patterns:
• Config-driven design: almost every experiment or model is defined by a .py config under mmdetection3d/configs and projects/configs.
• Plugin architecture: new detectors, dataset loaders, loss functions, and ops are registered via mmdet hooks and the projects/mmdet3d_plugin folder.
• Modular data pipeline: raw data → data_converter scripts → mmdet3d Dataset classes → pipeline transforms → model input tensors.
Key Components
a. Data Sources & Conversion
• Raw HD map datasets (nuScenes, ArgoVerse2, KITTI, Waymo, etc.)
• tools/data_converter & mmdetection3d/tools/create_data.py
• Output: tile-indexed 3D vectorized prior databases
b. Data Loading & Preprocessing
• mmdetection3d/datasets + pipelines (loading, augment, formatting)
• projects/mmdet3d_plugin datasets for map data (e.g., av2_map_dataset.py)
c. Core Library (mmdetection3d)
• Model registry (builder.py, backbones, heads, detectors, losses, middle_encoders, voxel_encoders)
• Ops (custom CUDA kernels: spconv, voxelization, ball_query, etc.)
• Training/Evaluation APIs (mmdet3d/apis/train.py, test.py, inference.py)
d. Uni-PrevPredMap Plugin
• projects/mmdet3d_plugin (bevformer, maptr, uniprevpredmap modules)
• Tile-indexed 3D vectorized global map processor
• Temporal fusion modules (custom hooks, fusion_layers, modules under plugin)
e. Configuration Manager
• experiments defined in projects/configs/uniprevpredmap/*.py
• Base configs inherited from mmdetection3d/configs/_base_
f. Training & Evaluation Workflow
• CLI wrappers: tools/train.py, tools/dist_train.sh, mmdet3d/tools/train.py
• Multi-GPU launcher, Slurm scripts (tools/slurm_train.sh)
• Benchmarking & flops analysis (tools/analysis_tools)
g. Inference & Visualization
• Demo scripts: mmdetection3d/demo/*.py, tools/misc/visualize_results.py
• Visualization of vectorized HD maps (Open3D, matplotlib)
h. External Dependencies
• PyTorch, MMCV, MMDetection3D, spconv, CUDA, Open3D, numpy, OpenCV
Relationships and Data Flow
• Raw data → data_converter → Dataset class → pipeline transforms → model input
• Model input → backbone → middle encoder → neck → dense heads + decode heads → predicted map tokens
• Predictions + previous-frame prior + outdated HD map → fusion modules → refined predictions
• Loss computation → backprop → weight update
• Trained model → inference API → vectorized HD map output → visualization
Architectural Patterns & Principles
• Config-first, plugin-based modularity: new research modules register seamlessly with the mmdetection3d core.
• Separation of concerns: data conversion, data loading, model definition, training loops, and visualization are decoupled.
• Layered design:
• Data layer (conversion, loading)
• Core compute layer (ops, backbones, detectors)
• Research layer (plugin modules, prior-informed fusion)
• Application layer (training scripts, inference demos)
Final Advice
• Break the system into as many logical components as needed to reveal data, control, and module dependencies.
• Be explicit about the interplay between mmdetection3d core and the Uni-PrevPredMap research plugin.