How 23aibox/scmamba works
1. Purpose and scope
- This repository is the official codebase for scMamba, a foundation model for single-cell multi-omics integration.
- The architecture is centered on inference and model internals; training support exists in the package, but the README emphasizes pretrained checkpoint usage.
2. Top-level execution paths
- inference.py is the main entrypoint for single-modality or paired multi-omics inference using a pretrained checkpoint.
- inference_rna_adt.py is a second inference path specialized for RNA + ADT style inputs.
- These scripts sit outside the package and act as orchestration layers that load config, data, model weights, and output results.
3. Core package boundary
- The scmamba2/ package contains the reusable application logic:
- dataset/ for data loading and sample construction
- preprocess.py for feature preparation
- tokenizer/ for genomic patch/token conversion
- models/ for the neural network
- loss.py and trainer.py for optimization
- utils/ for evaluation, plotting, metrics, processing, and trajectory analysis
- This cleanly separates pipeline orchestration from model/data primitives.
4. Input data and preprocessing flow
- The project is designed around single-cell multi-omics datasets stored in formats like .h5mu as shown in the README.
- preprocess.py and utils/process.py likely handle filtering, normalization, feature selection parameters, and dataset-specific preparation.
- The README’s n_top_genes / n_top_peaks arguments indicate the input pipeline can constrain feature space even though the model is described as avoiding prior feature selection in the method narrative.
5. Tokenization subsystem
- scmamba2/tokenizer/tokenizer.py is a key architectural component.
- scMamba represents genomic regions as tokens and cells as sequences/sentences, so this layer converts sparse omics matrices into model-ready token sequences.
- This tokenization boundary is important because it is where biological structure becomes the sequence abstraction consumed by the model.
6. Dataset and batching layer
- scmamba2/dataset/dataset.py encapsulates how samples are read, indexed, and batched for training/inference.
- This layer bridges raw or preprocessed omics data with the tokenizer and model input shapes.
- It is also where modality-specific handling likely happens, especially for mixed inputs like RNA+ADT.
7. Model architecture
- scmamba2/models/ contains the neural core:
- scmamba.py is the main model definition
- block.py and module.py define reusable building blocks
- config_scmamba.py and related JSON/YAML configs define model hyperparameters
- The README and dependencies indicate the model relies on Mamba / state space duality rather than a standard transformer stack.
- This is the central compute boundary of the system: tokenized omics sequences enter here, latent representations and predictions come out.
8. Multi-omics alignment and objective functions
- loss.py likely implements the training objectives, including the contrastive learning and cosine similarity regularization described in the README.
- This subsystem is the main mechanism for aligning embeddings across omics layers and is distinct from the reconstruction or prediction backbone.
- In the architecture diagram, this should be shown as a parallel supervision path connected to model outputs.
9. Training and optimization orchestration
- trainer.py coordinates forward passes, loss computation, backpropagation, validation, and checkpointing.
- The presence of config_files/accelerate_config.yaml and the README references to accelerate and DeepSpeed indicate the training path can be distributed across multiple GPUs.
- So the training boundary is not just a loop; it is designed to plug into a distributed execution framework.
10. Evaluation and downstream analysis
- scmamba2/utils/evaluation.py, metrics.py, trajectory.py, and plot.py support post-training analysis and benchmarking.
- These utilities indicate the model is evaluated not only on prediction quality but also on biological preservation, cross-omics alignment, clustering, cell annotation, and trajectory inference.
- This is an important downstream subsystem because it turns embeddings into biological-readout results.
11. Configuration and reproducibility
- config_files/scmamba2_config.json, scmamba2_config_rna_adt.json, and scmamba2_scale_config.json separate model/runtime presets from code.
- The configs make the same codebase adaptable across dataset types and scale regimes.
- This should be treated as a thin configuration layer controlling model size, input modality, and runtime behavior.
12. External dependencies and runtime constraints
- The repo depends on PyTorch 2.3.1, CUDA 12.1, cudnn, causal_conv1d, mamba, and optionally flash-attention.
- These are materially important because they define the model’s execution model and explain why the code is GPU-centric and performance-sensitive.
- Distributed training uses accelerate and deepspeed when enabled.
13. Output artifacts
- Inference writes results to a user-specified results directory and consumes pretrained checkpoints from external storage.
- The README points to Dropbox-hosted checkpoints and datasets, so the operational flow is: external data/checkpoint source → local preprocessing/tokenization → model inference → results/plots/metrics.
- For an architecture diagram, this is the clearest end-to-end data flow.
14. Suggested diagram boundaries
- Show four main boxes: Data prep, Tokenization & dataset, scMamba model + losses, and Evaluation/output.
- Put inference.py and inference_rna_adt.py as entrypoints above those boxes, and the config files as a side input controlling all stages.
- Keep accelerate/deepspeed as an optional execution layer around training rather than part of the model itself.
Main entry points: evaluation.py Evaluation · scmamba.py scMamba · scmamba2_config.json Configs · inference.py Inference
Components 17
Orchestration
Entry scripts and runtime configuration for inference and training
- Inference · Main pretrained inference path for single- or multi-omics inputs
inference.py - RNA+ADT Inference · Specialized inference path for RNA and ADT inputs
inference_rna_adt.py - Configs · Model and execution presets for standard, RNA+ADT, and scale settings
config_files/scmamba2_config.json
Data pipeline
Preprocessing, tokenization, and dataset assembly
- Preprocess · Feature filtering, normalization, and dataset preparation
scmamba2/preprocess.py - Process utils · Shared preprocessing and dataset processing utilities
scmamba2/utils/process.py - Tokenizer · Converts omics features into token sequences for the model
scmamba2/tokenizer/tokenizer.py - Dataset · Reads, indexes, and batches samples for training and inference
scmamba2/dataset/dataset.py
Core model
Neural backbone, configuration, and optimization
- scMamba · Main Mamba-based sequence model for omics representations
scmamba2/models/scmamba.py - Blocks · Reusable architectural blocks used inside the main model
scmamba2/models/block.py - Modules · Additional reusable neural modules and sublayers
scmamba2/models/module.py - Model config · Hyperparameter and shape definitions for scMamba variants
scmamba2/models/config_scmamba.py - Losses · Contrastive and similarity-based objectives for alignment
scmamba2/loss.py - Trainer · Coordinates forward passes, optimization, validation, and checkpointing
scmamba2/trainer.py
Analysis
Evaluation, metrics, plotting, and trajectory analysis
- Evaluation · Benchmarks embeddings and predictions after inference or training
scmamba2/utils/evaluation.py - Metrics · Quantitative evaluation and comparison utilities
scmamba2/utils/metrics.py - Plots · Visualization helpers for results and embedding summaries
scmamba2/utils/plot.py - Trajectory · Trajectory and biological progression analysis utilities
scmamba2/utils/trajectory.py
Connections 20
- Configs → Inference: controls · Selects inference mode, model size, and runtime settings
- Configs → RNA+ADT Inference: controls · Selects RNA+ADT-specific parameters and checkpoints
- Configs → Trainer: controls · Configures training, scaling, and distributed execution
- Inference → Preprocess: loads data · Starts the preprocessing path for inference inputs
- RNA+ADT Inference → Preprocess: loads data · Starts the preprocessing path for paired RNA+ADT inputs
- Preprocess → Process utils: uses · Delegates shared filtering and dataset preparation logic
- Process utils → Tokenizer: prepares · Shapes features for token conversion
- Tokenizer → Dataset: feeds · Creates model-ready sequences and sample structures
- Dataset → scMamba: inputs · Supplies tokenized batches to the model
- Model config → scMamba: defines · Provides architecture and size parameters
- Blocks → scMamba: builds · Implements internal model building blocks
- Modules → scMamba: builds · Provides reusable neural submodules
- scMamba → Losses: optimizes · Produces representations used by the training objectives
- Losses → Trainer: drives · Computes supervision signals for optimization
- Trainer → Evaluation: verifies · Runs validation and downstream assessment
- Evaluation → Metrics: scores · Computes benchmark and similarity metrics
- Evaluation → Plots: renders · Generates figures for results inspection
- Evaluation → Trajectory: analyzes · Supports trajectory and progression analysis
- Inference → Evaluation: outputs · Writes results for post-run analysis
- RNA+ADT Inference → Evaluation: outputs · Writes paired-modality results for analysis