How scverse/squidpy works
1. What this repository is
Squidpy is a Python library for spatial omics analysis. The central architecture is a set of analysis and visualization modules that operate mainly on anndata.AnnData-style data, with a parallel image-processing layer for tissue microscopy. It is not an app; it is a scientific computation toolkit.
2. Core data model boundary
The main boundary is between spatial transcriptomics/cell data and image data. Transcriptomic/spatial metadata is handled through read/analysis routines, while microscopy images are handled through the im package. Results are typically attached back to the input object and then consumed by plotting or downstream analysis.
3. Input and data acquisition layer
The read package contains file readers, with explicit support for Visium-oriented spatial inputs. The datasets package is separate and handles example-data discovery, registry lookup, and downloading of packaged datasets from remote sources. This isolates “load user data” from “fetch tutorial/test data.”
4. Graph and spatial statistics subsystem
The gr namespace is the main analytical engine. It groups algorithms for spatial neighbor graphs and higher-level analyses such as neighborhood enrichment, co-occurrence/patterns, Ripley-style spatial statistics, niche analysis, ligand-receptor analysis, and SEPAL-related routines. These functions consume spatial coordinates, graph structures, and annotations.
5. Tooling / feature extraction layer
The tl namespace contains more method-like analysis utilities, such as sliding-window computations and distance-based variable summaries. This layer sits between raw input/graph construction and the plotting layer, providing reusable computed features for downstream analysis.
6. Image processing subsystem
The im package is the dedicated image pipeline. It includes container abstractions, coordinate handling, I/O, preprocessing, segmentation, and feature extraction. This module is where high-resolution tissue images are normalized, tiled, segmented, and transformed into derived features used by the graph/statistics layers.
7. Experimental image extensions
experimental/im contains newer or provisional image workflows such as tissue detection and tile generation. Keeping these under an experimental namespace is an architectural boundary: stable APIs remain in im, while incubating image methods are isolated from the core contract.
8. Visualization subsystem
The pl package contains plotting code for graphs, spatial views, ligand-receptor results, and variable-by-distance outputs. It also includes interactive plotting support in a nested _interactive area. Plotting is a consumer layer: it renders results produced by gr, tl, im, and read without owning the analysis logic.
9. Shared infrastructure and utilities
The repository has a small set of shared modules for constants, compatibility shims, validators, docs helpers, and generic utilities. These are cross-cutting support pieces that reduce duplication across the analysis, I/O, and plotting packages.
10. Documentation and examples as a first-class surface
docs/ is extensive and includes API reference, class docs, tutorials, notebooks, release notes, and custom Sphinx extensions. This indicates the library’s architecture is meant to be learned through executable examples, not only through API reference.
11. Tests mirror the package boundaries
tests/ mirrors the main subsystems: datasets, read, image, graph, plotting, tools, experimental, and utility helpers. This is a strong sign that each top-level package is an independently testable subsystem with its own contracts and fixtures.
12. Build and release pipeline
GitHub Actions workflows cover build, test, notebook execution, and release. Supporting scripts in .scripts/ci download data and run notebooks during CI. ReadTheDocs configuration and code coverage integration show that documentation and reproducibility are part of the release architecture.
13. External ecosystem dependencies that shape the design
The package is built around the scverse stack, especially scanpy and anndata, and uses scientific image tooling such as scikit-image. The README also points users to napari-spatialdata for interactive visualization, which suggests Squidpy’s own interactive layer is being aligned with the broader SpatialData ecosystem rather than acting as the primary viewer.
14. How to draw the architecture
A good diagram would show: external spatial data and image files flowing into read/datasets and im; analysis branching into gr and tl; derived results flowing into pl; and experimental/im as a side branch for emerging image features. Surrounding all of that are shared utilities, docs, tests, and CI/release automation.
Main entry points: read Read · im Images · gr Graph stats · pl Plotting
Components 15
Core library
Primary analysis, I/O, image, and plotting subsystems.
- Read · Loads spatial inputs such as Visium-style data into the library's data model.
src/squidpy/read - Datasets · Resolves, downloads, and serves packaged example datasets separate from user data loading.
src/squidpy/datasets - Images · Core microscopy image container, coordinate handling, I/O, preprocessing, segmentation, and feature extraction.
src/squidpy/im - Exp. images · Incubating image workflows such as tissue detection and tile generation.
src/squidpy/experimental/im - Graph stats · Neighbor graphs, enrichment, co-occurrence, Ripley-style statistics, niche analysis, ligand-receptor analysis, and SEPAL routines.
src/squidpy/gr - Tools · Sliding-window and distance-based summaries used to prepare analysis features.
src/squidpy/tl - Plotting · Static and interactive plotting for graphs, spatial views, ligand-receptor results, and variable-by-distance outputs.
src/squidpy/pl
Shared support
Cross-cutting constants, utilities, validators, and docs helpers.
- Constants · Package-wide constants and low-level shared definitions.
src/squidpy/_constants - Utils · General-purpose helper functions used across the package.
src/squidpy/_utils.py - Validators · Validation helpers that enforce contracts across readers, analysis, and plotting.
src/squidpy/_validators.py - Docs helpers · Documentation-facing helpers shared by API docs and generated references.
src/squidpy/_docs.py
Docs
Documentation, tutorials, and release notes.
- Docs · Reference docs, tutorials, notebooks, release notes, and custom Sphinx extensions.
docs
Tests
Subsystem-level regression tests and fixtures.
- Tests · Package-aligned tests for datasets, read, image, graph, plotting, tools, experimental code, and utilities.
tests
Automation
CI, notebook execution, and release workflows.
- CI workflows · GitHub Actions workflows for build, test, notebook execution, and release.
.github/workflows - CI scripts · Helper scripts that download data and execute notebooks during CI.
.scripts/ci
Connections 24
- Datasets → Read: example data · Provides packaged datasets used alongside real user inputs.
- Read → Graph stats: coordinates · Feeds spatial observations and annotations into graph/statistical analyses.
- Read → Tools: input tables · Supplies imported data for derived feature computation.
- Read → Plotting: loaded data · Visualization consumes the imported spatial objects.
- Images → Graph stats: image-derived features · Image features and segmentation outputs can inform spatial analysis.
- Images → Plotting: image views · Plotting renders image-based outputs and overlays.
- Exp. images → Images: emerging APIs · Experimental image workflows extend the stable image subsystem.
- Graph stats → Plotting: results · Graph and spatial-statistics outputs are rendered by plotting.
- Tools → Plotting: summaries · Derived summaries become plotting inputs.
- Constants → Read: shared config · Provides package-level constants used by readers.
- Constants → Images: shared config · Provides package-level constants used by image code.
- Utils → Graph stats: helpers · Shared helpers support graph construction and analysis routines.
- Utils → Plotting: helpers · Shared helpers support plotting utilities.
- Validators → Read: validation · Checks reader inputs and file-specific assumptions.
- Validators → Images: validation · Checks image container and preprocessing inputs.
- Validators → Graph stats: validation · Checks analysis parameters and outputs.
- Docs helpers → Docs: doc build · Supports generated API/reference documentation.
- Tests → Read: coverage · Exercises reader contracts and fixtures.
- Tests → Images: coverage · Exercises image pipeline behavior.
- Tests → Graph stats: coverage · Exercises spatial-statistics algorithms.
- Tests → Plotting: coverage · Exercises plotting behavior.
- CI scripts → CI workflows: support · Feeds workflow steps for data download and notebook execution.
- CI workflows → Docs: publish · Build and release automation keeps docs and notebooks reproducible.
- CI workflows → Tests: verify · Workflows run the test matrix and notebook checks.