How yilmazkadir/mask4former works
Project type and purpose
• Mask4Former is a specialized deep‐learning research codebase implementing a transformer-based model for 4D panoptic segmentation on LiDAR data (SemanticKITTI).
• It’s not a web app or microservice suite but a full ML pipeline: data preprocessing → training → inference/evaluation.
File structure
• Top-level directories:
• conf/ → Hydra YAML configs for every component (data, model, optimizer, scheduler, callbacks, logging, trainer, etc.)
• datasets/ → preprocessing logic & dataset definitions (SemanticKITTI)
• data/ → preprocessed/serialized data storage
• models/ → backbone (ResNet/UNet), transformer heads, criterion, matcher, metrics
• trainer/ → PyTorch Lightning trainer wrapper
• scripts/ → shell scripts for train, test, val
• main_panoptic.py → entrypoint for training & inference
• Notice use of PyTorch Lightning (lightning.Trainer), Hydra (conf/), and custom modules under utils/.
Key README insights
• Relies on Hydra for configuration, PyTorch Lightning for training orchestration.
• External dependencies: PyTorch, MinkowskiEngine, PyTorch3D.
• Two main workflows:
• Data preprocessing (datasets.preprocessing.semantic_kitti_preprocessing)
• Training/inference via main_panoptic.py with Hydra overrides.
Components
a. Main components
• Raw Data Storage (SemanticKITTI)
• Data Preprocessor (datasets.preprocessing)
• Preprocessed Data (data/)
• Data Pipeline & Augmentation (datasets/lidar.py + conf/data/* + volumentations)
• Model Definition (models/… ; backbone + transformer heads + criterion + matcher)
• Config Manager (Hydra reading conf/*)
• Training Orchestrator (PyTorch Lightning / trainer/trainer.py)
• Checkpoint & Logging (conf/logging + wandb or file sink)
• Inference & Post-processing (main_panoptic.py with optional DBSCAN)
• External Libraries (PyTorch, MinkowskiEngine, PyTorch3D)
b. Relationships & interactions
• Config Manager feeds settings to Preprocessor, DataLoader, Model, Trainer
• Model outputs pass to Criterion & Matcher → Loss → Lightning Trainer → Optimizer & Scheduler
• Trainer writes checkpoints & metrics to disk/W&B
• Inference path: Checkpoint → Model → Predictions → (Optional) DBSCAN → Evaluation Metrics
c. Architectural patterns & design principles
• Modular design: separate preprocessing, modeling, training, config
• Config-driven via Hydra for reproducibility and easy hyperparameter sweeps
• Use of PyTorch Lightning to abstract the training loop and callbacks
• Plug-and-play components: different backbones, matchers, augmentations, metrics
d. Key technologies/frameworks
• Hydra (config management)
• PyTorch Lightning (training orchestration)
• PyTorch + MinkowskiEngine + PyTorch3D (3D sparse convs, point cloud ops)
• YAML configs under conf/ for every functional area