How saltoricristiano/cosmix-uda works
Project Type and Purpose
• This is a research‐oriented, full‐stack Python application for Unsupervised and Semi-Supervised Domain Adaptation (UDA/SSDA) in 3D LiDAR semantic segmentation.
• It implements the CoSMix approach: a two-branch network that mixes source (synthetic) and target (real) point cloud samples during training to reduce domain shift.
Top-Level Components
• Entry-point Scripts
• train_source.py – warm-up pre-training on source data
• adapt_cosmix_uda.py – UDA adaptation stage
• adapt_cosmix_ssda.py – SSDA adaptation stage
• finetune_ssda.py – one-shot target fine-tuning
• eval.py – evaluation/inference
• Configuration Layer
• configs/ – YAML files for all experiments (source, UDA, SSDA, finetuning)
• Data Handling
• utils/datasets/ – dataset classes for SynLiDAR, SemanticKITTI, SemanticPOSS
• utils/sampling/ – voxelizer, SSDA samplers
• utils/collation/ – custom collate functions
• Core Pipelines
• utils/pipelines/masked_simm_pipeline.py – UDA mixing pipeline
• utils/pipelines/masked_ssda_pipeline.py – SSDA mixing pipeline
• utils/pipelines/trainer_lighting.py – PyTorch Lightning trainer wrapper
• Model Definitions
• utils/models/ – MinkUNet, ResNet backbones, integration with NVIDIA’s MinkowskiEngine
• Loss Computation
• utils/losses/losses.py – segmentation and mix losses
• External Services & Assets
• WandB for logging
• Docker images (crissalto/online-adaptation-mink)
Interactions & Data Flow
• Each script parses command‐line args → reads one or more YAML config files → instantiates a config object.
• Config object selects which pipeline to use.
• Pipeline constructs two DataLoader streams (source & target), applying voxelization, sampling, and mixing transforms.
• Mixed batches feed into the two-branch network (teacher & student) built from utils/models.
• Losses computed via utils/losses → backpropagation through PyTorch Lightning engine.
• Checkpoints, metrics, and visualizations are logged to WandB and saved to disk.
• eval.py reloads a saved checkpoint, runs forward passes on source/target data, optionally saves predictions.
Architectural Patterns
• Modular layering: Config → Pipeline → Dataset → Model → Loss → Trainer → Logger
• Two-branch, teacher-student domain adaptation
• Sample-mixing at point-cloud level for domain invariance
• Use of PyTorch Lightning for training orchestration
Key Technologies & Frameworks
• Python 3.8, PyTorch 1.8, PyTorch Lightning 1.4.1
• NVIDIA MinkowskiEngine for sparse 3D CNNs
• Open3D for point-cloud handling
• WandB for experiment tracking
• tqdm for progress bars; Docker for environment reproducibility