# scverse/spatialdata architecture

> Architecture diagram of the GitHub repository scverse/spatialdata, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/scverse/spatialdata
- Repository: https://github.com/scverse/spatialdata

## Overview

##### Project Type and Purpose
- SpatialData is a Python data‐framework library for spatial omics: it provides an in‐memory object model, I/O to/from disk in OME-NGFF/Zarr, high-level operations (map, aggregate, rasterize, vectorize), and transformation utilities.
- It ships a CLI entrypoint, a core API, multiple sub-packages (dataloader, io, models, transformations), plus docs, benchmarks, and tests.

A. CLI/API Layer
- src/spatialdata/__main__.py → “spatialdata” command-line entrypoint
- High-level API functions in src/spatialdata/\*.py (e.g., config.py, testing.py)
B. Core Library Layer
- src/spatialdata/_core/ — spatialdata objects, element definitions, centroids, concatenation, extent, validation
- src/spatialdata/_core/operations/ — spatial operations (aggregate, map, rasterize, transform, vectorize)
- src/spatialdata/_core/query/ — relational and spatial query engines
C. I/O Layer
- src/spatialdata/_io/ — format definitions, utilities
- Readers/Writers: io_points, io_raster, io_shapes, io_table, io_zarr
D. Dataloader & Dataset Layer
- src/spatialdata/dataloader/ → “datasets.py” for standardized dataset ingestion
- src/spatialdata/datasets.py → helper functions to construct common spatialdata instances
E. Models & Transformations Layer
- src/spatialdata/models/ — statistical or ML-style models on spatialdata
- src/spatialdata/transformations/ — coordinate transforms, NGFF coordinate systems, operations utils
F. Storage & External Dependencies
- OME-NGFF/Zarr on disk
- External libraries: xarray, zarr, numpy, pandas, pyproj, dask (implied by I/O)
G. Extensions & Ecosystem
- Integration points with spatialdata-io, spatialdata-plot, napari-spatialdata
H. Tooling & CI/CD
- GitHub workflows (build_image.yml, test.yaml, release.yaml), Dockerfile, benchmarks/asv, docs build

##### Relationships and Data Flows
- CLI/API → Core Library (invokes operations, queries)
- Core Library → I/O Layer (load/save datasets)
- Core Library ↔ Dataloader (to assemble SpatialData objects from raw inputs)
- Core Library → Models & Transformations (applies transforms and model fits)
- I/O Layer → Zarr/OME-NGFF (disk or cloud storage backend)
- Ecosystem packages → I/O Layer & Core Library (plug-ins use the same interfaces)

##### Architectural Patterns & Design Principles
- Layered architecture (API → Core → I/O → Storage)
- Plugin-style extensibility: new I/O formats, new transform modules, new dataloaders
- Separation of concerns: “_core” is pure in-memory data logic; “_io” handles persistence; “dataloader” ingest; “models” & “transformations” live on top
- Use of object-oriented design for data elements and functional decomposition for operations

##### Optional CI/CD & Documentation Layer
- At the side or bottom, represent GitHub Actions workflows, Dockerfile, docs build pipeline
- This informs maintainers how code changes propagate into releases and docs

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI/API Layer
    subgraph "CLI/API Layer"
        CLI["spatialdata (src/spatialdata/__main__.py)"]:::cli
        Config["config.py"]:::cli
        Testing["testing.py"]:::cli
    end

    %% Core Library Layer
    subgraph "Core Library Layer"
        CorePkg["_core/"]:::core
        Elements["_elements.py"]:::core
        Validation["validation.py"]:::core
        DataExtent["data_extent.py"]:::core
        Centroids["centroids.py"]:::core
        Concatenate["concatenate.py"]:::core
        CoreUtils["_utils.py"]:::core
        Deepcopy["_deepcopy.py"]:::core
        SpatialData["spatialdata.py"]:::core

        subgraph "Operations"
            OpsPkg["operations/"]:::core
            Agg["aggregate.py"]:::core
            MapOp["map.py"]:::core
            Rasterize["rasterize.py"]:::core
            RasterBins["rasterize_bins.py"]:::core
            TransformOp["transform.py"]:::core
            Vectorize["vectorize.py"]:::core
            OpsUtils["_utils.py"]:::core
        end

        subgraph "Query Engines"
            QueryPkg["query/"]:::core
            Relational["relational_query.py"]:::core
            SpatialQ["spatial_query.py"]:::core
            QueryUtils["_utils.py"]:::core
        end
    end

    %% I/O Layer
    subgraph "I/O Layer"
        IOPkg["_io/"]:::io
        Format["format.py"]:::io
        IOPoints["io_points.py"]:::io
        IORaster["io_raster.py"]:::io
        IOShapes["io_shapes.py"]:::io
        IOTable["io_table.py"]:::io
        IOZarr["io_zarr.py"]:::io
        IOUtils["_utils.py"]:::io
    end

    %% Dataloader & Dataset Layer
    subgraph "Dataloader & Dataset Layer"
        Dataloader["dataloader/datasets.py"]:::dataloader
        Datasets["datasets.py"]:::dataloader
    end

    %% Models & Transformations Layer
    subgraph "Models Layer"
        ModelsPkg["models/"]:::models
        ModelsFile["models.py"]:::models
        ModelsUtils["_utils.py"]:::models
    end
    subgraph "Transformations Layer"
        TransPkg["transformations/"]:::transform
        TransFile["transformations.py"]:::transform
        TransOps["operations.py"]:::transform
        TransUtils["_utils.py"]:::transform
        subgraph "NGFF Coordinate Systems"
            NGFFPkg["ngff/"]:::transform
            NGFFCoord["ngff_coordinate_system.py"]:::transform
            NGFFTrans["ngff_transformations.py"]:::transform
            NGFFUtils["_utils.py"]:::transform
        end
    end

    %% Extensions & Plugin Bridges
    subgraph "Extensions & Plugin Bridges"
        Bridges["_bridges/"]:::ecosystem
    end

    %% Storage & External Dependencies
    subgraph "Storage & External Dependencies"
        Storage["Zarr/OME-NGFF Storage"]:::external
        ExtLibs["xarray, numpy, pandas, pyproj, dask"]:::external
    end

    %% Ecosystem Packages
    subgraph "Ecosystem Packages"
        SPIo["spatialdata-io"]:::ecosystem
        SPPlot["spatialdata-plot"]:::ecosystem
        Napari["napari-spatialdata"]:::ecosystem
    end

    %% Tooling & CI/CD
    subgraph "Tooling & CI/CD"
        Workflows[".github/workflows/"]:::ci
        Dockerfile["Dockerfile"]:::ci
        Benchmarks["benchmarks/"]:::ci
        Docs["docs/"]:::ci
    end

    %% Relationships
    CLI -->|"invokes"| CorePkg
    Config --> CorePkg
    Testing --> CorePkg

    CorePkg -->|"load()/save()"| IOPkg
    CorePkg <-->|"assemble/serialize"| Dataloader
    CorePkg -->|"apply()"| ModelsPkg
    CorePkg -->|"transform()"| TransPkg

    IOPkg -->|"persist"| Storage
    IOPkg -->|"uses"| ExtLibs

    SPIo -.->|"plugin"| IOPkg
    SPPlot -.->|"uses"| CorePkg
    Napari -.->|"plugin"| CorePkg

    Dataloader --> CorePkg
    ModelsPkg --> CorePkg
    TransPkg --> CorePkg

    Workflows --> Dockerfile
    Workflows --> Benchmarks
    Workflows --> Docs

    %% Click Events
    click CLI "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/__main__.py"
    click Config "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/config.py"
    click Testing "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/testing.py"
    click CorePkg "https://github.com/scverse/spatialdata/tree/main/src/spatialdata/_core/"
    click Elements "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/_elements.py"
    click Validation "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/validation.py"
    click DataExtent "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/data_extent.py"
    click Centroids "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/centroids.py"
    click Concatenate "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/concatenate.py"
    click CoreUtils "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/_utils.py"
    click Deepcopy "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/_deepcopy.py"
    click SpatialData "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/spatialdata.py"
    click OpsPkg "https://github.com/scverse/spatialdata/tree/main/src/spatialdata/_core/operations/"
    click Agg "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/operations/aggregate.py"
    click MapOp "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/operations/map.py"
    click Rasterize "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/operations/rasterize.py"
    click RasterBins "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/operations/rasterize_bins.py"
    click TransformOp "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/operations/transform.py"
    click Vectorize "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/operations/vectorize.py"
    click OpsUtils "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/operations/_utils.py"
    click QueryPkg "https://github.com/scverse/spatialdata/tree/main/src/spatialdata/_core/query/"
    click Relational "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/query/relational_query.py"
    click SpatialQ "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/query/spatial_query.py"
    click QueryUtils "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/query/_utils.py"
    click IOPkg "https://github.com/scverse/spatialdata/tree/main/src/spatialdata/_io/"
    click Format "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_io/format.py"
    click IOPoints "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_io/io_points.py"
    click IORaster "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_io/io_raster.py"
    click IOShapes "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_io/io_shapes.py"
    click IOTable "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_io/io_table.py"
    click IOZarr "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_io/io_zarr.py"
    click IOUtils "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_io/_utils.py"
    click Dataloader "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/dataloader/datasets.py"
    click Datasets "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/datasets.py"
    click ModelsPkg "https://github.com/scverse/spatialdata/tree/main/src/spatialdata/models/"
    click ModelsFile "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/models/models.py"
    click ModelsUtils "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/models/_utils.py"
    click TransPkg "https://github.com/scverse/spatialdata/tree/main/src/spatialdata/transformations/"
    click TransFile "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/transformations/transformations.py"
    click TransOps "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/transformations/operations.py"
    click TransUtils "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/transformations/_utils.py"
    click NGFFPkg "https://github.com/scverse/spatialdata/tree/main/src/spatialdata/transformations/ngff/"
    click NGFFCoord "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/transformations/ngff/ngff_coordinate_system.py"
    click NGFFTrans "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/transformations/ngff/ngff_transformations.py"
    click NGFFUtils "https://github.com/scverse/spatialdata/blob/main/src/spatialdata/transformations/ngff/_utils.py"
    click Bridges "https://github.com/scverse/spatialdata/tree/main/src/spatialdata/_bridges/"

    %% Styles
    classDef cli fill:#cccccc,stroke:#333,stroke-width:1px
    classDef core fill:#cce5ff,stroke:#004085,stroke-width:1px
    classDef io fill:#d4edda,stroke:#155724,stroke-width:1px
    classDef dataloader fill:#ffe5b4,stroke:#996600,stroke-width:1px
    classDef models fill:#e2c2f1,stroke:#6f42c1,stroke-width:1px
    classDef transform fill:#f5e6cc,stroke:#b8860b,stroke-width:1px
    classDef ecosystem fill:#f8d7da,stroke:#721c24,stroke-width:1px
    classDef external fill:#fff3cd,stroke:#856404,stroke-width:1px
    classDef ci fill:#e2e3e5,stroke:#6c757d,stroke-width:1px
```
