# scverse/scvi-tools architecture

> Architecture diagram of the GitHub repository scverse/scvi-tools, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/scverse/scvi-tools
- Repository: https://github.com/scverse/scvi-tools

## Overview

This project is scvi-tools – a Python library designed for probabilistic analysis of single-cell omics data. It is not a traditional full-stack application with separate frontend and backend services; rather, it is a modular, research-focused library built on top of PyTorch (and other frameworks like Pyro and PyTorch Lightning) that provides a collection of building blocks for dimensionality reduction, data integration, automated annotation, and more.

##### Main components
- Core Library (src/scvi): This is the heart of the package. It is organized into submodules that include:
– Data Handling: Components in src/scvi/data and its nested directories manage data input/output, preprocessing, built-in datasets, and data utility functions.
– Dataloaders: Located in src/scvi/dataloaders, these classes are responsible for batching, sampling, and splitting the data.
– Models and Base Classes: Found in src/scvi/model and src/scvi/model/base; these contain various model implementations (e.g., scVI, TotalVI, AutoZI) and the underlying classes that govern model architecture and training logic.
– Modules: In src/scvi/module and its base submodules, which implement neural network components, VAE infrastructure, and specialized model-specific functions.
– Training Infrastructure: Under src/scvi/train, these components incorporate training plans, logging, progress tracking, and callbacks essential for model training.
– Distributions & Statistics: In src/scvi/distributions, which implement probabilistic distributions (negative binomial, beta binomial, etc.) used in the models.
– Criticism and External Integrations: Modules such as src/scvi/criticism and src/scvi/external provide additional model criticism tools and integrations with approaches like cellassign, contrastivevi, and others.
– Hub: Under src/scvi/hub, which likely facilitates model sharing, metadata management, and possibly model templates.

##### Relationships and interactions
- Data flows into the system via the data modules and built-in datasets. The raw inputs are processed and then fed into dataloaders.
- Dataloaders interface with model training components to supply data in a suitable format (batching, splitting) during the training process.
- Model modules leverage core base classes and architecture mixins (embeddings, training mixins, differential functionalities) to build various probabilistic models.
- The training package orchestrates training routines by invoking loggers, callbacks, and metric computations, incorporating the distributions defined separately.
- External modules (located in src/scvi/external) extend the capabilities of core models by introducing alternative modeling strategies and specialized functionality.
- Overarching configuration and core settings (e.g., src/scvi/_settings.py and _constants.py) standardize the runtime behavior and default parameters across components.

##### Architectural patterns and design principles
- Modular and Layered Design: The codebase is segregated into distinct modules (data, models, training, distributions, etc.), which communicate through clear interfaces.
- Reusability and Composability: The base classes in model and module directories are designed to be extended, fostering rapid development of new probabilistic models.
- Abstraction and Separation of Concerns: Each directory (e.g., data, model, train) encapsulates specific functionality, making it easier to test, extend, and maintain.
- Integration with External Frameworks: By building on top of PyTorch, Pyro, and PyTorch Lightning, scvi-tools leverages established high-performance libraries and design patterns.

## Diagram (Mermaid)

```mermaid
flowchart TD
    Core["Core Library"]:::core

    subgraph "Data Layer"
        DI["Data Ingestion & Preprocessing"]:::data
        T["Testing (Data)"]:::data
    end

    DL["Data Loaders"]:::dataLoader

    subgraph "Modeling Layer"
        BM["Base Models"]:::models
        SM["Specific Models"]:::models
        M["Modules (NN Components & VAE)"]:::models
    end

    PD["Probabilistic Distributions"]:::distributions
    TI["Training Infrastructure"]:::training

    subgraph "Extensions/External"
        CT["Criticism Tools"]:::external
        EM["External/Extended Models"]:::external
        HUB["Hub & Metadata Management"]:::external
    end

    %% Relationships
    Core -->|"includes"| DI
    Core -->|"includes"| DL
    Core -->|"includes"| BM
    Core -->|"includes"| TI

    DI -->|"feeds"| DL
    DL -->|"supplies data to"| BM
    DL -->|"supplies data to"| SM
    DL -->|"supplies data to"| M

    BM -->|"triggers training"| TI
    SM -->|"triggers training"| TI
    M  -->|"triggers training"| TI

    BM -->|"utilizes"| PD
    SM -->|"utilizes"| PD
    M  -->|"utilizes"| PD

    BM -->|"analyzed by"| CT
    SM -->|"extended by"| EM
    EM -->|"integrates with"| HUB

    TI -->|"uses"| PD

    %% Styles
    classDef core fill:#ffedcc,stroke:#d18b00,stroke-width:2px;
    classDef data fill:#ccf2ff,stroke:#0077b3,stroke-width:2px;
    classDef dataLoader fill:#ccffcc,stroke:#228B22,stroke-width:2px;
    classDef models fill:#ffe6ff,stroke:#a020f0,stroke-width:2px;
    classDef training fill:#ffcccc,stroke:#ff0000,stroke-width:2px;
    classDef distributions fill:#f2ccff,stroke:#8a2be2,stroke-width:2px;
    classDef external fill:#e6e6e6,stroke:#808080,stroke-width:2px;

    %% Click Events
    click Core "https://github.com/scverse/scvi-tools/tree/main/src/scvi"
    click DI "https://github.com/scverse/scvi-tools/tree/main/src/scvi/data"
    click T "https://github.com/scverse/scvi-tools/tree/main/tests/test_data"
    click DL "https://github.com/scverse/scvi-tools/tree/main/src/scvi/dataloaders"
    click BM "https://github.com/scverse/scvi-tools/tree/main/src/scvi/model/base"
    click SM "https://github.com/scverse/scvi-tools/tree/main/src/scvi/model"
    click M "https://github.com/scverse/scvi-tools/tree/main/src/scvi/module"
    click TI "https://github.com/scverse/scvi-tools/tree/main/src/scvi/train"
    click PD "https://github.com/scverse/scvi-tools/tree/main/src/scvi/distributions"
    click CT "https://github.com/scverse/scvi-tools/tree/main/src/scvi/criticism"
    click EM "https://github.com/scverse/scvi-tools/tree/main/src/scvi/external"
    click HUB "https://github.com/scverse/scvi-tools/tree/main/src/scvi/hub"
```
