How lu-jiang/mask2map works
Project Type & Purpose
• This is a research-oriented deep learning pipeline for autonomous driving. Its purpose is to generate HD maps by predicting instance masks and then driving map point predictions.
Overview of the Architecture Components
• Data Input and Preprocessing:
• Data ingestion layer where raw sensor data (e.g., LiDAR, camera inputs) and dataset-specific formats (such as nuScenes, Argoverse2, etc.) are loaded.
• Data preparation pipeline (hinted by the docs such as “Prepare Dataset” and various data_converter scripts) which handles augmentation, transformation, and conversion.
• Core Networks:
• MMPNet (Mask-Driven Map Prediction Network): This network uses the masks and queries from IMPNet along with additional local contextual information. Within MMPNet, indicate its two submodules:
PQG: The module responsible for generating positional queries from BEV information.
GFE: The module that extracts point-level geometric features.
• Inter-Network Denoising Training:
• This component as a feedback loop or an additional module that bridges IMPNet and MMPNet. It highlights the training strategy that denoises the outputs of both networks to ensure consistent predictions against the ground truth.
• Training, Evaluation, and Inference Pipeline:
• Components for loss calculation, training scripts (found in tools and tests folders), evaluation metrics (mAP, etc.), and model conversion/demos.
Integration with mmdetection3d Ecosystem
• Since the project builds extensively on mmdetection3d, show a dependency layer (e.g., “mmdet3d Core Library”) which houses the fundamental building blocks such as annotation handling, voxel generation, backbones, and loss modules.
• That Mask2Map’s own networks and configurations leverage these base components for backbone networks, data pipelines, and training routines.
Relationships and Interaction Flows
• Input data → Preprocessing/Data Converter module → IMPNet.
• IMPNet outputs BEV segmentation masks and mask-aware queries → fed into MMPNet.
• Within MMPNet, show the two branches leading to PQG and GFE, which then converge to output the final vectorized HD map predictions.
• Training loop with the inter-network denoising strategy that “closes the loop” between the predictions and the loss backpropagation.
• Evaluation module where generated maps are compared against ground truth and performance is logged.