How thangvubk/softgroup works
Main Components
• Data Preparation & Input:
– The “dataset” directory contains dataset-specific scripts, shell scripts (prepare_data.sh), and configuration YAML files (e.g., semantic-kitti.yaml) to format point cloud data.
– This is where your raw/input data is preprocessed and split into training, validation, and testing subsets.
• Configuration Management:
– The “configs” directory holds many YAML configuration files for different datasets and training setups (e.g., softgroup_scannet.yaml, softgroup_s3dis_foldX.yaml).
– These files define hyperparameters, network settings, and training details.
• Core Library / Modules:
– “softgroup/data”: Data loader and custom data parsing logic (e.g., kitti.py, s3dis.py) that turns raw datasets into inputs suitable for training.
– “softgroup/model”: Contains the definition of the model (e.g., blocks.py, softgroup.py) implementing the SoftGroup architecture, including backbone and grouping modules.
– “softgroup/evaluation”: Contains scripts for various evaluation metrics (point-wise, instance evaluation, panoptic evaluation).
– “softgroup/util”: Provides utilities such as distributed training helpers, logging (logger.py), data type conversions (fp16.py), and more.
• Custom Operations (Ops):
– Under “softgroup/ops”, you have custom CUDA/C++ kernels (in folders like bfs_cluster, octree_ball_query, roipool, sec_mean, voxelize) – these are performance-critical low-level operations used by the network for grouping, clustering, and voxel processing.
– This component is essential for the efficient computation of operations that allow for the “soft” grouping strategy.
• Training, Testing, and Inference Pipeline:
– The “tools” directory contains scripts (train.py, test.py, dist_train.sh, dist_test.sh) that orchestrate the training and testing processes.
– These scripts use the configuration files and the core library components to kick off training/inference, manage distributed training, and coordinate evaluation steps.
• Documentation & Deployment:
– The “docs” directory contains documents explaining configuration details, custom dataset integration, visualizations, and logs.
– Build scripts and environment files (requirements.txt, setup.py, setup.cfg) provide deployment setups.
Interactions and Data Flow
• The configuration files (from “configs”) are read by the training/inference scripts to set up model parameters and training protocols.
• The model (softgroup/model), which uses custom operations from “softgroup/ops”, processes the input data.
• During training, utilities from “softgroup/util” manage logging, optimizers, and distributed computation.
• Evaluation modules (softgroup/evaluation) analyze output predictions against ground truth, while visualization scripts (in “tools” and referenced in “docs/visualization.md”) showcase results.
– Data flow: From dataset preparation to data loaders.
– Control flow: From configuration to the training and model instantiation.
– Dependency: The model relies on custom CUDA/C++ operations defined in “ops”.
Architectural Patterns and Technologies Involved
• Modular Design: The project is split into modules (data, model, ops, evaluation, utilities) to keep concerns separate.
• Custom High-Performance Operations: Use of CUDA-based kernels for critical functions (e.g. voxelization, clustering).
• Config-Driven Training: Training setups and hyperparameters are entirely externalized in YAML configuration files.
• Distributed and Mixed-Precision Training: The scripts and utilities point to support for distributed training which speeds up computation (using distributed tools in “tools/dist_train.sh”).
• Integration with external frameworks: The project uses PyTorch with spconv (sparse convolution) and other related libraries.
Additional Details to Capture
• Presence of both “backend-type” computations (model computations, custom ops) and “orchestration” scripts (tools/ for managing distributed and mixed precision training).
• External dependencies such as PyTorch and spconv clearly so it’s evident they play a critical role.
• Incorporate a reference to the design improvements (refactored code to support both SoftGroup and SoftGroup++) and training/inference pipeline differences for the distinct supported datasets.
Be sure to include:
This approach should make the architecture clear to both current and future team members and aid in further development or optimization of the pipeline.