# 23aibox/scmamba architecture

> Architecture diagram of the GitHub repository 23aibox/scmamba, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/23aibox/scmamba
- Repository: https://github.com/23aibox/scmamba

## Overview

1. **Purpose and scope**
   - This repository is the official codebase for **scMamba**, a foundation model for **single-cell multi-omics integration**.
   - The architecture is centered on **inference and model internals**; training support exists in the package, but the README emphasizes pretrained checkpoint usage.

2. **Top-level execution paths**
   - `inference.py` is the main entrypoint for **single-modality or paired multi-omics inference** using a pretrained checkpoint.
   - `inference_rna_adt.py` is a second inference path specialized for **RNA + ADT** style inputs.
   - These scripts sit outside the package and act as orchestration layers that load config, data, model weights, and output results.

3. **Core package boundary**
   - The `scmamba2/` package contains the reusable application logic:
     - `dataset/` for data loading and sample construction
     - `preprocess.py` for feature preparation
     - `tokenizer/` for genomic patch/token conversion
     - `models/` for the neural network
     - `loss.py` and `trainer.py` for optimization
     - `utils/` for evaluation, plotting, metrics, processing, and trajectory analysis
   - This cleanly separates **pipeline orchestration** from **model/data primitives**.

4. **Input data and preprocessing flow**
   - The project is designed around single-cell multi-omics datasets stored in formats like `.h5mu` as shown in the README.
   - `preprocess.py` and `utils/process.py` likely handle filtering, normalization, feature selection parameters, and dataset-specific preparation.
   - The README’s `n_top_genes` / `n_top_peaks` arguments indicate the input pipeline can constrain feature space even though the model is described as avoiding prior feature selection in the method narrative.

5. **Tokenization subsystem**
   - `scmamba2/tokenizer/tokenizer.py` is a key architectural component.
   - scMamba represents **genomic regions as tokens** and **cells as sequences/sentences**, so this layer converts sparse omics matrices into model-ready token sequences.
   - This tokenization boundary is important because it is where biological structure becomes the sequence abstraction consumed by the model.

6. **Dataset and batching layer**
   - `scmamba2/dataset/dataset.py` encapsulates how samples are read, indexed, and batched for training/inference.
   - This layer bridges raw or preprocessed omics data with the tokenizer and model input shapes.
   - It is also where modality-specific handling likely happens, especially for mixed inputs like RNA+ADT.

7. **Model architecture**
   - `scmamba2/models/` contains the neural core:
     - `scmamba.py` is the main model definition
     - `block.py` and `module.py` define reusable building blocks
     - `config_scmamba.py` and related JSON/YAML configs define model hyperparameters
   - The README and dependencies indicate the model relies on **Mamba / state space duality** rather than a standard transformer stack.
   - This is the central compute boundary of the system: tokenized omics sequences enter here, latent representations and predictions come out.

8. **Multi-omics alignment and objective functions**
   - `loss.py` likely implements the training objectives, including the **contrastive learning** and **cosine similarity regularization** described in the README.
   - This subsystem is the main mechanism for aligning embeddings across omics layers and is distinct from the reconstruction or prediction backbone.
   - In the architecture diagram, this should be shown as a parallel supervision path connected to model outputs.

9. **Training and optimization orchestration**
   - `trainer.py` coordinates forward passes, loss computation, backpropagation, validation, and checkpointing.
   - The presence of `config_files/accelerate_config.yaml` and the README references to `accelerate` and `DeepSpeed` indicate the training path can be distributed across multiple GPUs.
   - So the training boundary is not just a loop; it is designed to plug into a distributed execution framework.

10. **Evaluation and downstream analysis**
   - `scmamba2/utils/evaluation.py`, `metrics.py`, `trajectory.py`, and `plot.py` support post-training analysis and benchmarking.
   - These utilities indicate the model is evaluated not only on prediction quality but also on **biological preservation**, **cross-omics alignment**, **clustering**, **cell annotation**, and **trajectory inference**.
   - This is an important downstream subsystem because it turns embeddings into biological-readout results.

11. **Configuration and reproducibility**
   - `config_files/scmamba2_config.json`, `scmamba2_config_rna_adt.json`, and `scmamba2_scale_config.json` separate model/runtime presets from code.
   - The configs make the same codebase adaptable across dataset types and scale regimes.
   - This should be treated as a thin configuration layer controlling model size, input modality, and runtime behavior.

12. **External dependencies and runtime constraints**
   - The repo depends on **PyTorch 2.3.1**, **CUDA 12.1**, **cudnn**, `causal_conv1d`, `mamba`, and optionally **flash-attention**.
   - These are materially important because they define the model’s execution model and explain why the code is GPU-centric and performance-sensitive.
   - Distributed training uses `accelerate` and `deepspeed` when enabled.

13. **Output artifacts**
   - Inference writes results to a user-specified results directory and consumes pretrained checkpoints from external storage.
   - The README points to Dropbox-hosted checkpoints and datasets, so the operational flow is: external data/checkpoint source → local preprocessing/tokenization → model inference → results/plots/metrics.
   - For an architecture diagram, this is the clearest end-to-end data flow.

14. **Suggested diagram boundaries**
   - Show four main boxes: **Data prep**, **Tokenization & dataset**, **scMamba model + losses**, and **Evaluation/output**.
   - Put `inference.py` and `inference_rna_adt.py` as entrypoints above those boxes, and the config files as a side input controlling all stages.
   - Keep `accelerate/deepspeed` as an optional execution layer around training rather than part of the model itself.

Main entry points: [`scmamba2/utils/evaluation.py`](https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/evaluation.py) (Evaluation), [`scmamba2/models/scmamba.py`](https://github.com/23aibox/scmamba/blob/main/scmamba2/models/scmamba.py) (scMamba), [`config_files/scmamba2_config.json`](https://github.com/23aibox/scmamba/blob/main/config_files/scmamba2_config.json) (Configs), [`inference.py`](https://github.com/23aibox/scmamba/blob/main/inference.py) (Inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Orchestration"]
  node_inference(("Inference<br/>entrypoint<br/>[inference.py]"))
  node_inference_rna_adt(("RNA+ADT Inference<br/>entrypoint"))
  node_configs["Configs<br/>runtime presets"]
end

subgraph group_data["Data pipeline"]
  node_preprocess["Preprocess<br/>data prep<br/>[preprocess.py]"]
  node_process_utils["Process utils<br/>prep helpers<br/>[process.py]"]
  node_tokenizer["Tokenizer<br/>tokenization<br/>[tokenizer.py]"]
  node_dataset[("Dataset<br/>sample builder<br/>[dataset.py]")]
end

subgraph group_model["Core model"]
  node_scmamba["scMamba<br/>model core<br/>[scmamba.py]"]
  node_block["Blocks<br/>[block.py]"]
  node_module["Modules<br/>[module.py]"]
  node_model_config["Model config<br/>architecture config<br/>[config_scmamba.py]"]
  node_loss["Losses<br/>training objective<br/>[loss.py]"]
  node_trainer["Trainer<br/>training loop<br/>[trainer.py]"]
end

subgraph group_analysis["Analysis"]
  node_evaluation["Evaluation<br/>assessment<br/>[evaluation.py]"]
  node_metrics["Metrics<br/>scoring<br/>[metrics.py]"]
  node_plot["Plots<br/>visualization<br/>[plot.py]"]
  node_trajectory["Trajectory<br/>analysis<br/>[trajectory.py]"]
end

node_configs -->|"controls"| node_inference
node_configs -->|"controls"| node_inference_rna_adt
node_configs -->|"controls"| node_trainer
node_inference -->|"loads data"| node_preprocess
node_inference_rna_adt -->|"loads data"| node_preprocess
node_preprocess -->|"uses"| node_process_utils
node_process_utils -->|"prepares"| node_tokenizer
node_tokenizer -->|"feeds"| node_dataset
node_dataset -->|"inputs"| node_scmamba
node_model_config -->|"defines"| node_scmamba
node_block -->|"builds"| node_scmamba
node_module -->|"builds"| node_scmamba
node_scmamba -->|"optimizes"| node_loss
node_loss -->|"drives"| node_trainer
node_trainer -->|"verifies"| node_evaluation
node_evaluation -->|"scores"| node_metrics
node_evaluation -->|"renders"| node_plot
node_evaluation -->|"analyzes"| node_trajectory
node_inference -.->|"outputs"| node_evaluation
node_inference_rna_adt -.->|"outputs"| node_evaluation

click node_inference "https://github.com/23aibox/scmamba/blob/main/inference.py"
click node_inference_rna_adt "https://github.com/23aibox/scmamba/blob/main/inference_rna_adt.py"
click node_configs "https://github.com/23aibox/scmamba/blob/main/config_files/scmamba2_config.json"
click node_preprocess "https://github.com/23aibox/scmamba/blob/main/scmamba2/preprocess.py"
click node_process_utils "https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/process.py"
click node_tokenizer "https://github.com/23aibox/scmamba/blob/main/scmamba2/tokenizer/tokenizer.py"
click node_dataset "https://github.com/23aibox/scmamba/blob/main/scmamba2/dataset/dataset.py"
click node_scmamba "https://github.com/23aibox/scmamba/blob/main/scmamba2/models/scmamba.py"
click node_block "https://github.com/23aibox/scmamba/blob/main/scmamba2/models/block.py"
click node_module "https://github.com/23aibox/scmamba/blob/main/scmamba2/models/module.py"
click node_model_config "https://github.com/23aibox/scmamba/blob/main/scmamba2/models/config_scmamba.py"
click node_loss "https://github.com/23aibox/scmamba/blob/main/scmamba2/loss.py"
click node_trainer "https://github.com/23aibox/scmamba/blob/main/scmamba2/trainer.py"
click node_evaluation "https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/evaluation.py"
click node_metrics "https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/metrics.py"
click node_plot "https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/plot.py"
click node_trajectory "https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/trajectory.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_inference,node_inference_rna_adt,node_configs toneBlue
class node_preprocess,node_process_utils,node_tokenizer,node_dataset toneAmber
class node_scmamba,node_block,node_module,node_model_config,node_loss,node_trainer toneMint
class node_evaluation,node_metrics,node_plot,node_trajectory toneRose
```

## Components

### Orchestration

Entry scripts and runtime configuration for inference and training

- [Inference](https://github.com/23aibox/scmamba/blob/main/inference.py): entrypoint · Main pretrained inference path for single- or multi-omics inputs · `inference.py`
- [RNA+ADT Inference](https://github.com/23aibox/scmamba/blob/main/inference_rna_adt.py): entrypoint · Specialized inference path for RNA and ADT inputs · `inference_rna_adt.py`
- [Configs](https://github.com/23aibox/scmamba/blob/main/config_files/scmamba2_config.json): runtime presets · Model and execution presets for standard, RNA+ADT, and scale settings · `config_files/scmamba2_config.json`

### Data pipeline

Preprocessing, tokenization, and dataset assembly

- [Preprocess](https://github.com/23aibox/scmamba/blob/main/scmamba2/preprocess.py): data prep · Feature filtering, normalization, and dataset preparation · `scmamba2/preprocess.py`
- [Process utils](https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/process.py): prep helpers · Shared preprocessing and dataset processing utilities · `scmamba2/utils/process.py`
- [Tokenizer](https://github.com/23aibox/scmamba/blob/main/scmamba2/tokenizer/tokenizer.py): tokenization · Converts omics features into token sequences for the model · `scmamba2/tokenizer/tokenizer.py`
- [Dataset](https://github.com/23aibox/scmamba/blob/main/scmamba2/dataset/dataset.py): sample builder · Reads, indexes, and batches samples for training and inference · `scmamba2/dataset/dataset.py`

### Core model

Neural backbone, configuration, and optimization

- [scMamba](https://github.com/23aibox/scmamba/blob/main/scmamba2/models/scmamba.py): model core · Main Mamba-based sequence model for omics representations · `scmamba2/models/scmamba.py`
- [Blocks](https://github.com/23aibox/scmamba/blob/main/scmamba2/models/block.py): model blocks · Reusable architectural blocks used inside the main model · `scmamba2/models/block.py`
- [Modules](https://github.com/23aibox/scmamba/blob/main/scmamba2/models/module.py): model modules · Additional reusable neural modules and sublayers · `scmamba2/models/module.py`
- [Model config](https://github.com/23aibox/scmamba/blob/main/scmamba2/models/config_scmamba.py): architecture config · Hyperparameter and shape definitions for scMamba variants · `scmamba2/models/config_scmamba.py`
- [Losses](https://github.com/23aibox/scmamba/blob/main/scmamba2/loss.py): training objective · Contrastive and similarity-based objectives for alignment · `scmamba2/loss.py`
- [Trainer](https://github.com/23aibox/scmamba/blob/main/scmamba2/trainer.py): training loop · Coordinates forward passes, optimization, validation, and checkpointing · `scmamba2/trainer.py`

### Analysis

Evaluation, metrics, plotting, and trajectory analysis

- [Evaluation](https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/evaluation.py): assessment · Benchmarks embeddings and predictions after inference or training · `scmamba2/utils/evaluation.py`
- [Metrics](https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/metrics.py): scoring · Quantitative evaluation and comparison utilities · `scmamba2/utils/metrics.py`
- [Plots](https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/plot.py): visualization · Visualization helpers for results and embedding summaries · `scmamba2/utils/plot.py`
- [Trajectory](https://github.com/23aibox/scmamba/blob/main/scmamba2/utils/trajectory.py): analysis · Trajectory and biological progression analysis utilities · `scmamba2/utils/trajectory.py`

## Connections

- Configs → Inference: controls · Selects inference mode, model size, and runtime settings
- Configs → RNA+ADT Inference: controls · Selects RNA+ADT-specific parameters and checkpoints
- Configs → Trainer: controls · Configures training, scaling, and distributed execution
- Inference → Preprocess: loads data · Starts the preprocessing path for inference inputs
- RNA+ADT Inference → Preprocess: loads data · Starts the preprocessing path for paired RNA+ADT inputs
- Preprocess → Process utils: uses · Delegates shared filtering and dataset preparation logic
- Process utils → Tokenizer: prepares · Shapes features for token conversion
- Tokenizer → Dataset: feeds · Creates model-ready sequences and sample structures
- Dataset → scMamba: inputs · Supplies tokenized batches to the model
- Model config → scMamba: defines · Provides architecture and size parameters
- Blocks → scMamba: builds · Implements internal model building blocks
- Modules → scMamba: builds · Provides reusable neural submodules
- scMamba → Losses: optimizes · Produces representations used by the training objectives
- Losses → Trainer: drives · Computes supervision signals for optimization
- Trainer → Evaluation: verifies · Runs validation and downstream assessment
- Evaluation → Metrics: scores · Computes benchmark and similarity metrics
- Evaluation → Plots: renders · Generates figures for results inspection
- Evaluation → Trajectory: analyzes · Supports trajectory and progression analysis
- Inference → Evaluation: outputs · Writes results for post-run analysis
- RNA+ADT Inference → Evaluation: outputs · Writes paired-modality results for analysis
