How xhanxu/mamba3d works
Project type and purpose
• This is an open-source, command-line driven machine-learning framework for 3D point-cloud classification and analysis.
• Its core contribution is “Mamba3D,” a state-space-model-based architecture that trains and evaluates point-cloud classifiers on datasets such as ModelNet40, ScanObjectNN, and ShapeNet.
File structure
• Top-level components:
• cfgs/ ← configuration files (YAML) for pretraining, fine-tuning, few-shot experiments, dataset specs
• datasets/ ← PyTorch Dataset classes, data transforms, I/O and few-shot data generator
• extensions/ ← custom CUDA/C++ extensions (Chamfer distance, Earth Mover’s Distance)
• models/ ← core model code: Mamba3D wrapper, the bimamba_ssm package (submodules: models, modules, ops, utils), helper modules (rope, z_order)
• tools/ ← experiment drivers (builder, runner scripts for pretrain/finetune/fewshot/SVM/TSNE)
• script/ ← shell scripts invoking Python entrypoints with specific configs
• utils/ ← generic utilities (config loader, registry, logging, checkpointing, distributed-training helpers)
• main.py ← primary CLI entrypoint that parses a cfg, builds components, and kicks off training/evaluation
README for additional insights
• Describes dependencies (PyTorch, CUDA, mamba-ssm, causal-conv1d) and external installs (PointNet++, GPU kNN).
• Points to DATASET.md for data details and demonstrates how to run training, voting, fine-tuning, few-shot experiments.
• Emphasizes pretrained weights and performance targets.
a. Main components
1. Data Sources
• Raw datasets on disk (ShapeNet, ModelNet, ScanObjectNN)
2. Data Module
• datasets/ package: Dataset classes, transforms, few-shot data generator
3. Configuration
• YAML config files in cfgs/ + utils/config.py
4. Model Module
• models/Mamba3D.py (high-level builder)
• models/bimamba_ssm/ sub-package (state space modules, mixers, ops)
5. Custom Extensions
• Chamfer Distance and EMD CUDA kernels under extensions/
6. Training Engine
• main.py + tools/runner_*.py orchestrate training loops, loss computation, optimization
7. Utilities
• Logging (utils/logger.py), checkpointing (utils/checkpoint.py), distributed training (utils/dist_utils.py), registry pattern etc.
8. CLI & Scripts
• Shell wrappers in script/ invoke the Python entrypoint with specific config files
9. External Libraries
• PyTorch, mamba-ssm, causal-conv1d, PointNet++ ops, GPU kNN
b. Relationships and interactions
• Data Sources → Data Module → DataLoader
• Config Loader reads YAML → passes hyperparameters to Data Module, Model Module, Training Engine
• Training Engine builds model via Model Module → forward pass + custom extensions → loss
• Optimizer updates model → outputs checkpoints & logs via Utilities
• CLI/Scripts invoke Training Engine with specific configs (pretrain, finetune, few-shot, vote)
• Tools/runner scripts wrap common experiment patterns
c. Architectural patterns and design principles
• Configuration-driven design: all hyperparameters, dataset specs, model settings in YAML files
• Modular code organization: separation of concerns (datasets, models, training logic, utilities, extensions)
• Registry pattern: dynamic creation of datasets, models, losses based on config
• Plugin-style extensions: custom CUDA ops loaded at runtime
d. Relevant technologies, frameworks, and libraries
• PyTorch for deep learning
• CUDA/C++ for speed-critical kernels (Chamfer, EMD)
• mamba-ssm & causal-conv1d for state-space modeling
• YAML for experiment configuration
• Shell scripting + Python CLI for reproducible runs
Elements
• Blue for code modules (datasets, models, utils)
• Green for data components (raw datasets, DataLoader)
• Orange for configuration and CLI/scripts
• Red for external libraries and extensions
• Annotations for key file paths or classes (e.g., datasets/ModelNetDataset.py, models/Mamba3D.py, extensions/chamfer_dist)