# gustaveroussy/sopa architecture

> Architecture diagram of the GitHub repository gustaveroussy/sopa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gustaveroussy/sopa
- Repository: https://github.com/gustaveroussy/sopa

## Overview

##### Project type and purpose
- Sopa is a hybrid full-stack spatial-omics pipeline:
- A Python library (API) for processing spatial transcriptomics and imaging data.
- A CLI frontend for ad-hoc runs.
- A Snakemake pipeline for batch processing on clusters.
- It ingests raw spatial‐omics data (Xenium, Visium HD, CosMX, etc.), performs patching, segmentation (via multiple algorithms), aggregation, spatial analyses, and outputs standardized data and reports.

##### High-level file structure and architectural patterns
- Top-level directories:
- `sopa/` – core library modules (I/O, CLI, segmentation, patches, aggregation, spatial, utils).
- `workflow/` – Snakemake orchestrator (`Snakefile`, `rules/`, `config/`, `profile/`).
- `docker/` – Dockerfiles packaging segmentation engines.
- `docs/`, `mkdocs.yml` – documentation site.
- CI/CD: `.github/workflows/ci.yml`, GitHub Actions.
- Architectural pattern: modular pipeline. Core library layers correspond to sequential steps. Snakemake defines dependencies between these modules as rules.

##### Key README insights
- Sopa supports three “flavours”: API, CLI, Snakemake.
- Main processing steps:
1) Data reading & standardization (`sopa.io.reader`, `sopa.io.standardize`)
2) Patch creation (`sopa.patchify`)
3) Segmentation (`sopa.segmentation.methods…`)
4) Conflict resolution (`sopa.patches.resolve`)
5) Aggregation (`sopa.aggregation`)
6) Spatial analysis (`sopa.spatial`)
7) Reporting & interactive explorer output (`sopa.io.report`, `sopa.cli.explorer`)

##### Components
a. User interfaces
– API client (Python script/module)
– CLI (`sopa` commands)
– Snakemake orchestrator (config files + rules)
1. I/O & standardization
2. Patch generation
3. Segmentation engines (Cellpose, Baysor, Stardist, Proseg, etc.), each in its own Docker container or Conda env
4. Patch inference & resolution
5. Aggregation (transcripts/channels)
6. Spatial analytics (graph, distance, join)
7. Reporting & explorer conversion
c. Data storage
– Raw input (files on disk)
– Intermediate storage (Zarr on disk or memory‐mapped)
– Final outputs (zarr directories, HTML reports, explorer files)
d. Orchestration and deployment
– Snakemake engine invoking each module as a rule, using profiles (local, SLURM, LSF)
– Docker images for isolation of segmentation methods
– CI/CD pipeline for tests, linting, packaging

##### Relationships and interactions
- Documentation site (MkDocs) as an ancillary component reading from `docs/` and `mkdocs.yml`.

##### Architectural patterns & principles
- Modular single‐responsibility components
- Pipeline orchestration via Snakemake (declarative DAG)
- Containerization for reproducibility (Docker per engine)
- Separation of concerns: API vs CLI vs Workflow
- Plugin‐style segmentation methods loaded via factory pattern

##### Detail without over-splitting
- How the CLI and API wrap the same core modules.
- Snakemake profiles (local vs cluster) under an “Execution Environment” section.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "User Interfaces"
        API["API Client"]:::interface
        subgraph "CLI Commands"
            CLI_APP["app.py"]:::interface
            CLI_PATCH["patchify.py"]:::interface
            CLI_SEG["segmentation.py"]:::interface
            CLI_RESOLVE["resolve.py"]:::interface
            CLI_EXPLORER["explorer.py"]:::interface
            CLI_ANNOTATE["annotate.py"]:::interface
        end
        SNKW["Snakemake Orchestrator"]:::orchestration
    end

    subgraph "Configuration"
        CFG_FILES["workflow/config/*.yaml"]:::config
        PROFILES["workflow/profile/*"]:::config
    end

    subgraph "Core Library"
        subgraph "1. I/O & Standardization"
            IO_READER["sopa.io.reader"]:::io
            IO_STD["sopa.io.standardize"]:::io
        end
        subgraph "2. Patch Generation"
            PATCH_GEN["sopa.patches"]:::process
        end
        subgraph "3. Segmentation Engines"
            SEG_CELL["Cellpose"]:::container
            SEG_BAY["Baysor"]:::container
            SEG_STAR["Stardist"]:::container
            SEG_PRO["Proseg"]:::container
            SEG_COM["Comseg"]:::container
        end
        subgraph "4. Conflict Resolution"
            RESOLVE["sopa.patches._inference"]:::process
        end
        subgraph "5. Aggregation"
            AGG["sopa.aggregation"]:::process
        end
        subgraph "6. Spatial Analysis"
            SPATIAL["sopa.spatial"]:::process
        end
        subgraph "7. Reporting & Explorer"
            REPORT["sopa.io.report"]:::io
            EXPL_CONV["sopa.io.explorer.converter"]:::io
        end
    end

    subgraph "Data Storage"
        RAW["Raw Input Files"]:::storage
        INTERM["Intermediate Zarr Store"]:::storage
        FINAL["Final Outputs (zarr, HTML)"]:::storage
    end

    subgraph "Deployment / CI"
        DOCKER_BASE["docker/sopa/"]:::container
        DOCKER_CP["docker/sopa-cellpose/"]:::container
        DOCKER_BS["docker/sopa-baysor/"]:::container
        DOCKER_SD["docker/sopa-stardist/"]:::container
        DOCKER_PG["docker/sopa-proseg/"]:::container
        CI[".github/workflows/ci.yml"]:::ci
    end

    DOC["MkDocs Site"]:::doc

    %% Flows
    API --> IO_READER
    CLI_APP --> IO_READER
    CLI_PATCH --> PATCH_GEN
    CLI_SEG --> SEG_CELL
    CLI_SEG --> SEG_BAY
    CLI_SEG --> SEG_STAR
    CLI_SEG --> SEG_PRO
    CLI_SEG --> SEG_COM
    CLI_RESOLVE --> RESOLVE
    CLI_EXPLORER --> EXPL_CONV
    SNKW --> CFG_FILES
    SNKW --> PROFILES
    SNKW --> IO_READER
    SNKW --> PATCH_GEN
    SNKW --> SEG_CELL
    SNKW --> RESOLVE
    SNKW --> AGG
    SNKW --> SPATIAL
    SNKW --> REPORT

    IO_READER --> IO_STD
    IO_STD --> PATCH_GEN
    PATCH_GEN --> SEG_CELL
    PATCH_GEN --> SEG_BAY
    PATCH_GEN --> SEG_STAR
    PATCH_GEN --> SEG_PRO
    PATCH_GEN --> SEG_COM
    SEG_CELL --> RESOLVE
    SEG_BAY --> RESOLVE
    SEG_STAR --> RESOLVE
    SEG_PRO --> RESOLVE
    SEG_COM --> RESOLVE
    RESOLVE --> AGG
    AGG --> SPATIAL
    SPATIAL --> REPORT
    REPORT --> EXPL_CONV

    IO_READER --> RAW
    IO_STD --> INTERM
    EXPL_CONV --> FINAL

    %% Docker branches
    SEG_CELL --> DOCKER_CP
    SEG_BAY --> DOCKER_BS
    SEG_STAR --> DOCKER_SD
    SEG_PRO --> DOCKER_PG
    DOCKER_BASE --> SEG_CELL
    DOCKER_BASE --> SEG_BAY
    DOCKER_BASE --> SEG_STAR
    DOCKER_BASE --> SEG_PRO
    DOCKER_BASE --> SEG_COM

    CI --> DOCKER_BASE
    CI --> SNKW
    CI --> API
    CI --> CLI_APP

    DOC -->|"reads from"| docs_dir["docs/ + mkdocs.yml"]:::doc

    %% Click events
    click API "https://github.com/gustaveroussy/sopa/tree/main/sopa/"
    click CLI_APP "https://github.com/gustaveroussy/sopa/blob/main/sopa/cli/app.py"
    click CLI_PATCH "https://github.com/gustaveroussy/sopa/blob/main/sopa/cli/patchify.py"
    click CLI_SEG "https://github.com/gustaveroussy/sopa/blob/main/sopa/cli/segmentation.py"
    click CLI_RESOLVE "https://github.com/gustaveroussy/sopa/blob/main/sopa/cli/resolve.py"
    click CLI_EXPLORER "https://github.com/gustaveroussy/sopa/blob/main/sopa/cli/explorer.py"
    click CLI_ANNOTATE "https://github.com/gustaveroussy/sopa/blob/main/sopa/cli/annotate.py"
    click SNKW "https://github.com/gustaveroussy/sopa/tree/main/workflow/Snakefile"
    click CFG_FILES "https://github.com/gustaveroussy/sopa/tree/main/workflow/config/"
    click PROFILES "https://github.com/gustaveroussy/sopa/tree/main/workflow/profile/"
    click IO_READER "https://github.com/gustaveroussy/sopa/tree/main/sopa/io/reader/"
    click IO_STD "https://github.com/gustaveroussy/sopa/blob/main/sopa/io/standardize.py"
    click PATCH_GEN "https://github.com/gustaveroussy/sopa/tree/main/sopa/patches/"
    click RESOLVE "https://github.com/gustaveroussy/sopa/blob/main/sopa/patches/_inference.py"
    click SEG_CELL "https://github.com/gustaveroussy/sopa/blob/main/sopa/segmentation/methods/_cellpose.py"
    click SEG_BAY "https://github.com/gustaveroussy/sopa/blob/main/sopa/segmentation/methods/_baysor.py"
    click SEG_STAR "https://github.com/gustaveroussy/sopa/blob/main/sopa/segmentation/methods/_stardist.py"
    click SEG_PRO "https://github.com/gustaveroussy/sopa/blob/main/sopa/segmentation/methods/_proseg.py"
    click SEG_COM "https://github.com/gustaveroussy/sopa/blob/main/sopa/segmentation/methods/_comseg.py"
    click AGG "https://github.com/gustaveroussy/sopa/blob/main/sopa/aggregation/aggregation.py"
    click SPATIAL "https://github.com/gustaveroussy/sopa/blob/main/sopa/spatial/_build.py"
    click REPORT "https://github.com/gustaveroussy/sopa/blob/main/sopa/io/report/generate.py"
    click EXPL_CONV "https://github.com/gustaveroussy/sopa/blob/main/sopa/io/explorer/converter.py"
    click DOCKER_CP "https://github.com/gustaveroussy/sopa/tree/main/docker/sopa-cellpose/"
    click DOCKER_BS "https://github.com/gustaveroussy/sopa/tree/main/docker/sopa-baysor/"
    click DOCKER_SD "https://github.com/gustaveroussy/sopa/tree/main/docker/sopa-stardist/"
    click DOCKER_PG "https://github.com/gustaveroussy/sopa/tree/main/docker/sopa-proseg/"
    click DOCKER_BASE "https://github.com/gustaveroussy/sopa/tree/main/docker/sopa/"
    click CI "https://github.com/gustaveroussy/sopa/blob/main/.github/workflows/ci.yml"
    click docs_dir "https://github.com/gustaveroussy/sopa/tree/main/docs/"

    %% Styles
    classDef interface fill:#cccccc,stroke:#333,stroke-width:1px
    classDef orchestration fill:#b19cd9,stroke:#333,stroke-width:1px
    classDef config fill:#ffe4b5,stroke:#333,stroke-width:1px
    classDef io fill:#98fb98,stroke:#333,stroke-width:1px
    classDef process fill:#add8e6,stroke:#333,stroke-width:1px
    classDef container fill:#ffa500,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
    classDef storage fill:#90ee90,stroke:#333,stroke-width:1px,shape:cylinder
    classDef ci fill:#d3d3d3,stroke:#333,stroke-width:1px
    classDef doc fill:#dcdcdc,stroke:#333,stroke-width:1px,stroke-dasharray: 2 2
```
