# bowang-lab/scgpt architecture

> Architecture diagram of the GitHub repository bowang-lab/scgpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bowang-lab/scgpt
- Repository: https://github.com/bowang-lab/scgpt

## Overview

##### Project Type and Purpose
- scGPT is a Python package and toolkit for building and fine-tuning a foundation model for single-cell multi-omics data.
- It provides data-processing scripts, a databank module, preprocessing and tokenization layers, a model core (PyTorch), a training engine, examples, tutorials, and documentation.

##### File Structure
- Top-level directories:
- data/ – cellxgene data ingestion & preprocessing scripts (shell + Python)
- scgpt/ – main library, with subpackages for databank, tokenizer, model, tasks, utils, and training
- examples/ – sample scripts using the API (e.g., finetune_integration.py)
- docs/ & tutorials/ – documentation and Jupyter notebooks demonstrating use cases
- tests/ – unit tests validating each component
- Key configuration and packaging files: pyproject.toml, .readthedocs.yaml, LICENSE.

##### README for Additional Insights
- Core components:
- Pretrained model zoo and loading utilities (scgpt/utils/util.py)
- Optional dependencies: flash-attn, faiss, wandb, HuggingFace integration
- Online apps (Superbio.ai) that wrap the core library for inference services.

##### Main Components and Their Interactions
a. Data Ingestion & Preprocessing Subsystem
- data/cellxgene/*.sh and *.py scripts: download, partition, process raw single-cell data
- data_config.py, metainfo.json, cancer_list.txt, query_list.txt
- Outputs: processed count matrices and metadata files

b. Databank Module (scgpt/scbank)
- Data classes (data.py), Databank manager (databank.py), Monitor (monitor.py), Settings (setting.py)
- Abstracts loading, caching, indexing of large single-cell datasets (e.g., FAISS indexes)

c. Preprocessing & Tokenization
- scgpt/preprocess.py: transforms matrices into token sequences
- scgpt/tokenizer: default vocab JSONs + GeneTokenizer class

d. Model Core (scgpt/model)
- Base model and variants (model.py, generation_model.py, multiomic_model.py)
- Layers: DSBN (dsbn.py), gradient reversal (grad_reverse.py)

e. Training Engine & Task Layer
- scgpt/trainer.py: orchestrates training loops, callbacks
- scgpt/data_collator.py & data_sampler.py: batching strategies
- scgpt/loss.py: custom objective functions
- scgpt/tasks/: implementations for cell embedding, GRN inference, perturbation prediction, reference mapping

f. API, Examples & Deployment
- examples/finetune_integration.py: demonstrates API usage for fine-tuning
- Packaging (pyproject.toml) for PyPI distribution
- CLI wrappers or entrypoints (if any)

g. External Services & Tooling
- PyTorch (model training/inference)
- flash-attn (optional speedup)
- faiss (reference mapping index)
- wandb (logging/visualization)
- HuggingFace integration branch (future)
- Superbio.ai webapps for hosted inference

h. Documentation & Testing
- docs/: Sphinx configs and .rst files (architecture, tutorials, API references)
- tests/: unit tests for sampler, tokenizer, scbank, model

- Blue = core library modules (scgpt/…)
- Green = data processing & storage
- Orange = training & tasks
- Purple = external dependencies/services
- Raw data → Data Ingestion scripts → Databank → Preprocessing → Tokenization → Model → Trainer → Outputs
- Trainer → Wandb (logging)
- Trainer → faiss (indexing)
- Model/API → Superbio.ai (deployed service)

##### Architectural Patterns & Principles
- Modular layering: clear separation of data handling, preprocessing, modeling, and training
- Pipeline orchestration: sequential dataflow from raw matrices to model outputs
- Plug-in tasks: Task modules implement interchangeable learning objectives
- Extensibility: easy addition of new tokenizers, models, and tasks

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing" 
        DI["Data Ingestion Scripts\n(data/cellxgene/)"]:::data
    end

    %% Databank Module
    subgraph "Databank Module (scgpt/scbank)" 
        SCBANK_DATA["data.py"]:::core
        SCBANK_DATABANK["databank.py"]:::core
        SCBANK_MONITOR["monitor.py"]:::core
        SCBANK_SETTING["setting.py"]:::core
    end

    %% Preprocessing & Tokenization
    subgraph "Preprocessing & Tokenization"
        PREP["preprocess.py"]:::core
        subgraph "Tokenizer"
            GENE_TOKENIZER["gene_tokenizer.py"]:::core
            DEFAULT_VOCAB1["default_gene_vocab.json"]:::core
            DEFAULT_VOCAB2["default_census_vocab.json"]:::core
        end
    end

    %% Model Core
    subgraph "Model Core (scgpt/model)"
        MODEL_BASE["model.py"]:::core
        MODEL_GEN["generation_model.py"]:::core
        MODEL_MULTI["multiomic_model.py"]:::core
        MODEL_DSBN["dsbn.py"]:::core
        MODEL_GR{"grad_reverse.py"}:::core
    end

    %% Training Engine & Task Layer
    subgraph "Training Engine & Task Layer"
        TRAINER["trainer.py"]:::training
        COLLATOR["data_collator.py"]:::training
        SAMPLER["data_sampler.py"]:::training
        LOSS["loss.py"]:::training
        subgraph "Tasks (scgpt/tasks)"
            TASK_CELL["cell_emb.py"]:::training
            TASK_GRN["grn.py"]:::training
        end
    end

    %% API, Examples & Deployment
    subgraph "API, Examples & Deployment"
        EXAMPLE["finetune_integration.py"]:::core
        PKG["pyproject.toml"]:::core
    end

    %% Documentation & Tutorials
    subgraph "Documentation & Tutorials"
        DOCS["docs/"]:::docs
        TUT["tutorials/"]:::docs
    end

    %% Testing Suite
    subgraph "Testing Suite"
        TEST_SAMPLER["test_sampler.py"]:::tests
        TEST_SCBANK["test_scbank.py"]:::tests
        TEST_SCFORMER["test_scformer.py"]:::tests
        TEST_TOKENIZER["test_tokenizer.py"]:::tests
    end

    %% External Dependencies & Services
    Pytorch(("PyTorch")):::external
    FlashAttn(("flash-attn")):::external
    FAISS(("faiss")):::external
    Wandb(("wandb")):::external
    HF(("HuggingFace")):::external
    Superbio(("Superbio.ai")):::external

    %% Data Flows & Dependencies
    DI -->|"processed matrices"| SCBANK_DATA
    SCBANK_DATA --> SCBANK_DATABANK
    SCBANK_DATABANK --> SCBANK_MONITOR
    SCBANK_MONITOR --> SCBANK_SETTING
    SCBANK_SETTING -->|"datasets"| PREP
    PREP -->|"raw tokens"| GENE_TOKENIZER
    GENE_TOKENIZER --> DEFAULT_VOCAB1
    GENE_TOKENIZER --> DEFAULT_VOCAB2
    GENE_TOKENIZER --> MODEL_BASE
    MODEL_BASE --> MODEL_GEN
    MODEL_GEN --> MODEL_MULTI
    MODEL_MULTI --> MODEL_DSBN
    MODEL_DSBN --> MODEL_GR
    MODEL_GR --> TRAINER
    TRAINER --> COLLATOR
    TRAINER --> SAMPLER
    TRAINER --> LOSS
    LOSS --> TASK_CELL
    LOSS --> TASK_GRN
    TRAINER -->|"logs"| Wandb
    TRAINER -->|“FAISS index”| FAISS
    TRAINER -.-> FlashAttn
    MODEL_BASE -->|"API request"| Superbio
    HF -.-> MODEL_BASE

    EXAMPLE --> MODEL_BASE
    EXAMPLE --> PKG

    DOCS -.-> MODEL_BASE
    TUT -.-> EXAMPLE

    TEST_SAMPLER --> SAMPLER
    TEST_SCBANK --> SCBANK_DATABANK
    TEST_SCFORMER --> MODEL_BASE
    TEST_TOKENIZER --> GENE_TOKENIZER

    %% Click Events
    click DI "https://github.com/bowang-lab/scgpt/tree/main/data/cellxgene/"
    click SCBANK_DATA "https://github.com/bowang-lab/scgpt/blob/main/scgpt/scbank/data.py"
    click SCBANK_DATABANK "https://github.com/bowang-lab/scgpt/blob/main/scgpt/scbank/databank.py"
    click SCBANK_MONITOR "https://github.com/bowang-lab/scgpt/blob/main/scgpt/scbank/monitor.py"
    click SCBANK_SETTING "https://github.com/bowang-lab/scgpt/blob/main/scgpt/scbank/setting.py"
    click PREP "https://github.com/bowang-lab/scgpt/blob/main/scgpt/preprocess.py"
    click GENE_TOKENIZER "https://github.com/bowang-lab/scgpt/blob/main/scgpt/tokenizer/gene_tokenizer.py"
    click DEFAULT_VOCAB1 "https://github.com/bowang-lab/scgpt/blob/main/scgpt/tokenizer/default_gene_vocab.json"
    click DEFAULT_VOCAB2 "https://github.com/bowang-lab/scgpt/blob/main/scgpt/tokenizer/default_census_vocab.json"
    click MODEL_BASE "https://github.com/bowang-lab/scgpt/blob/main/scgpt/model/model.py"
    click MODEL_GEN "https://github.com/bowang-lab/scgpt/blob/main/scgpt/model/generation_model.py"
    click MODEL_MULTI "https://github.com/bowang-lab/scgpt/blob/main/scgpt/model/multiomic_model.py"
    click MODEL_DSBN "https://github.com/bowang-lab/scgpt/blob/main/scgpt/model/dsbn.py"
    click MODEL_GR "https://github.com/bowang-lab/scgpt/blob/main/scgpt/model/grad_reverse.py"
    click TRAINER "https://github.com/bowang-lab/scgpt/blob/main/scgpt/trainer.py"
    click COLLATOR "https://github.com/bowang-lab/scgpt/blob/main/scgpt/data_collator.py"
    click SAMPLER "https://github.com/bowang-lab/scgpt/blob/main/scgpt/data_sampler.py"
    click LOSS "https://github.com/bowang-lab/scgpt/blob/main/scgpt/loss.py"
    click TASK_CELL "https://github.com/bowang-lab/scgpt/blob/main/scgpt/tasks/cell_emb.py"
    click TASK_GRN "https://github.com/bowang-lab/scgpt/blob/main/scgpt/tasks/grn.py"
    click EXAMPLE "https://github.com/bowang-lab/scgpt/blob/main/examples/finetune_integration.py"
    click PKG "https://github.com/bowang-lab/scgpt/blob/main/pyproject.toml"
    click DOCS "https://github.com/bowang-lab/scgpt/tree/main/docs/"
    click TUT "https://github.com/bowang-lab/scgpt/tree/main/tutorials/"
    click TEST_SAMPLER "https://github.com/bowang-lab/scgpt/blob/main/tests/test_sampler.py"
    click TEST_SCBANK "https://github.com/bowang-lab/scgpt/blob/main/tests/test_scbank.py"
    click TEST_SCFORMER "https://github.com/bowang-lab/scgpt/blob/main/tests/test_scformer.py"
    click TEST_TOKENIZER "https://github.com/bowang-lab/scgpt/blob/main/tests/test_tokenizer.py"

    %% Styles
    classDef core fill:#cfe2f3,stroke:#0366d6,stroke-width:1px
    classDef data fill:#d9ead3,stroke:#6aa84f,stroke-width:1px
    classDef training fill:#fce5cd,stroke:#e69138,stroke-width:1px
    classDef external fill:#ddd0ff,stroke:#674ea7,stroke-width:1px
    classDef docs fill:#f2f2f2,stroke:#999999,stroke-width:1px
    classDef tests fill:#f4cccc,stroke:#cc0000,stroke-width:1px
```
