# tencentailabhealthcare/scbert architecture

> Architecture diagram of the GitHub repository tencentailabhealthcare/scbert, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tencentailabhealthcare/scbert
- Repository: https://github.com/tencentailabhealthcare/scbert

## Overview

##### Main Components

a) Data Processing Layer:
- Input data flow starting with raw scRNA-seq data
- Preprocessing components (reference preprocess.py):
- NCBI Gene database validation
- Gene symbol revision
- Normalization (scanpy integration)
- Log transformation

b) Model Architecture Layer:
- Performer Transformer Architecture (from performer_pytorch/):
- Main Performer-LM component with its key parameters:
- Token embedding (num_tokens=7)
- Dimensional embedding (dim=200)
- Multi-head attention (heads=10)
- Encoder layers (depth=6)
- Reversible layers (indicated by reversible.py)

c) Training Pipeline:
- Pre-training phase (pretrain.py)
- Fine-tuning phase (finetune.py)
- Prediction pipeline (predict.py)

##### Flow and Relationships

- Data flow from preprocessing → model input
- Pre-trained model → Fine-tuning process
- Fine-tuned model → Prediction system
- Novel cell type detection branch (with threshold mechanism)

##### Additional Elements

- Component types
- Key parameters and thresholds

##### Technical Notes

Annotations for:
- Required dependencies (PyTorch, transformers, scanpy, etc.)
- Computational requirements
- Distributed training capabilities
- Memory considerations

Remember to indicate that this is a research tool and include appropriate attribution to Tencent AI Lab.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Legend
        L1[Process]
        L2[(Database)]
        L3([External Integration])
        L4{{Model Component}}
    end

    subgraph "Data Processing Layer"
        Input[Raw scRNA-seq Data]
        Preprocess[Preprocessing]:::process
        NCBI[(NCBI Gene DB)]
        Scanpy([Scanpy Integration])
    end

    subgraph "Model Architecture Layer"
        ModelCore{{Performer-LM}}:::model
        subgraph "Performer Architecture"
            TokenEmbed[Token Embedding<br/>num_tokens=7]:::model
            DimEmbed[Dimensional Embedding<br/>dim=200]:::model
            MultiHead[Multi-head Attention<br/>heads=10]:::model
            EncoderLayers[Encoder Layers<br/>depth=6]:::model
            RevLayers[Reversible Layers]:::model
        end
    end

    subgraph "Training and Inference Layer"
        PreTrain[Pre-training]:::training
        FineTune[Fine-tuning]:::training
        Predict[Prediction System]:::training
        NovelDetect{Novel Cell Type<br/>Detection}:::decision
        Threshold{Threshold<br/>Check 0.5}:::decision
    end

    subgraph "Output Layer"
        CellType[Cell Type Annotation]:::output
        NovelCell[Novel Cell Type]:::output
    end

    %% Connections
    Input --> Preprocess
    Preprocess --> ModelCore
    NCBI --> Preprocess
    Scanpy --> Preprocess
    
    ModelCore --> PreTrain
    PreTrain --> FineTune
    FineTune --> Predict
    Predict --> NovelDetect
    NovelDetect --> Threshold
    Threshold -->|>0.5| NovelCell
    Threshold -->|≤0.5| CellType

    %% Component Relationships
    TokenEmbed --> DimEmbed
    DimEmbed --> MultiHead
    MultiHead --> EncoderLayers
    EncoderLayers --> RevLayers
    RevLayers --> ModelCore

    %% Styles
    classDef process fill:#a8d1ff
    classDef model fill:#90EE90
    classDef training fill:#FFB347
    classDef decision fill:#FFE4B5
    classDef output fill:#98FB98
    classDef external fill:#D3D3D3

    %% Click Events
    click Preprocess "https://github.com/TencentAILabHealthcare/scBERT/blob/master/preprocess.py"
    click ModelCore "https://github.com/TencentAILabHealthcare/scBERT/blob/master/performer_pytorch/performer_pytorch.py"
    click RevLayers "https://github.com/TencentAILabHealthcare/scBERT/blob/master/performer_pytorch/performer_pytorch/reversible.py"
    click PreTrain "https://github.com/TencentAILabHealthcare/scBERT/blob/master/pretrain.py"
    click FineTune "https://github.com/TencentAILabHealthcare/scBERT/blob/master/finetune.py"
    click Predict "https://github.com/TencentAILabHealthcare/scBERT/blob/master/predict.py"
```
