How atlasanalyticslab/2dmamba works
Main Components
• Data Input & Preprocessing:
• Dataset management resides in the 2DMambaMIL/dataset directory containing CSV files (for classification and survival tasks) and dataset utility scripts.
• Data processing scripts under 2DVMamba/classification/data (and similar segmentation data directories) serve to build image datasets and data loaders.
• Model Architectures & Framework Core:
• The 2DMambaMIL directory includes the core model implementations in mamba_ssm (state space model components), models, modules, ops, and utilities.
• The additional models (ABMIL, CLAM, DSMIL, etc.) provided in the models/ directory indicate various MIL (multiple instance learning) architectures.
• Extended Modules for Different Tasks:
• The 2DVMamba subproject holds configurations, model definitions, and training/testing scripts for both classification and segmentation.
• Custom CUDA Kernels:
• The cuda_kernel folder (with its CMakeLists.txt, build.sh, and CUDA source files) shows the custom implementation of “pscan” and “selective_scan” operations.
• The v2dmamba_scan module serves as the Python interface to the compiled CUDA kernels.
• Utilities & Deployment:
• Various utility modules (in utils, mamba_ssm/utils, etc.) handle common functionalities like core utilities, file handling, and survival analysis.
• Deployment scripts and configuration files for docker/singularity and PyTorch-to-TorchScript converters are integrated.
Component Relationships and Interactions
• Data flows from the dataset directories into preprocessing pipelines (handled by scripts in dataset and data subdirectories).
• Preprocessed data is fed into the model components (from both 2DMambaMIL and 2DVMamba) where the state space model architectures are instantiated.
• The models make use of custom ops from mamba_ssm/ops, which in turn may call into the custom CUDA kernels (via v2dmamba_scan).
• Results and training progress are monitored and potentially visualized using provided tools (e.g., draw_heatmap.py, visualization scripts in the segmentation/vis folder).
• Configuration files (YAML and Python scripts in the segmentation and classification configs) drive the training experiments and model behaviors.
Architectural Patterns and Design Principles
• Modularity: The separation of concerns is clearly seen in the distinct directories (core model code, dataset handling, utilities, custom CUDA operations, and configuration).
• Extensibility: The structure supports different tasks (classification and segmentation) with separate configuration sets, making it easier to extend or modify parts of the system.
• Acceleration: The inclusion of custom CUDA kernels and their integration through Python (via v2dmamba_scan) points to a design optimized for high performance processing.
• Reuse and Adaptation: The project builds upon previous frameworks like Mamba, VMamba, and mamba.py, suggesting an inheritance of design patterns and interoperability between modules.
Incorporate Relevant Technologies, Frameworks, and Libraries
• PyTorch is the central deep learning framework.
• CUDA (with custom kernels, built using CMake and integrated via Python modules) is key for acceleration.
• Utilities like torchvision, albumentations, and timm are used for data augmentation and model design.
• Config management via YAML and dynamic config scripts in the segmentation and classification folders.
• Deployment utilities (docker, singularity) ensure ease of reproducing the experimental environment.
o Blue for data-handling components.
o Green for core model logic.
o Orange for custom operations and CUDA kernel integration.
o Gray for configuration and utility scripts.
• Interactions (for example, how the model configuration files in 2DVMamba/classification/configs link into the training script, and how custom kernels are integrated into model ops).