How gustaveroussy/sopa works
Project type and purpose
• Sopa is a hybrid full-stack spatial-omics pipeline:
• A Python library (API) for processing spatial transcriptomics and imaging data.
• A CLI frontend for ad-hoc runs.
• A Snakemake pipeline for batch processing on clusters.
• It ingests raw spatial‐omics data (Xenium, Visium HD, CosMX, etc.), performs patching, segmentation (via multiple algorithms), aggregation, spatial analyses, and outputs standardized data and reports.
High-level file structure and architectural patterns
• Top-level directories:
• sopa/ – core library modules (I/O, CLI, segmentation, patches, aggregation, spatial, utils).
• workflow/ – Snakemake orchestrator (Snakefile, rules/, config/, profile/).
• docker/ – Dockerfiles packaging segmentation engines.
• docs/, mkdocs.yml – documentation site.
• CI/CD: .github/workflows/ci.yml, GitHub Actions.
• Architectural pattern: modular pipeline. Core library layers correspond to sequential steps. Snakemake defines dependencies between these modules as rules.
Key README insights
• Sopa supports three “flavours”: API, CLI, Snakemake.
• Main processing steps:
1) Data reading & standardization (sopa.io.reader, sopa.io.standardize)
2) Patch creation (sopa.patchify)
3) Segmentation (sopa.segmentation.methods…)
4) Conflict resolution (sopa.patches.resolve)
5) Aggregation (sopa.aggregation)
6) Spatial analysis (sopa.spatial)
7) Reporting & interactive explorer output (sopa.io.report, sopa.cli.explorer)
Components
a. User interfaces
– API client (Python script/module)
– CLI (sopa commands)
– Snakemake orchestrator (config files + rules)
1. I/O & standardization
2. Patch generation
3. Segmentation engines (Cellpose, Baysor, Stardist, Proseg, etc.), each in its own Docker container or Conda env
4. Patch inference & resolution
5. Aggregation (transcripts/channels)
6. Spatial analytics (graph, distance, join)
7. Reporting & explorer conversion
c. Data storage
– Raw input (files on disk)
– Intermediate storage (Zarr on disk or memory‐mapped)
– Final outputs (zarr directories, HTML reports, explorer files)
d. Orchestration and deployment
– Snakemake engine invoking each module as a rule, using profiles (local, SLURM, LSF)
– Docker images for isolation of segmentation methods
– CI/CD pipeline for tests, linting, packaging
Relationships and interactions
• Documentation site (MkDocs) as an ancillary component reading from docs/ and mkdocs.yml.
Architectural patterns & principles
• Modular single‐responsibility components
• Pipeline orchestration via Snakemake (declarative DAG)
• Containerization for reproducibility (Docker per engine)
• Separation of concerns: API vs CLI vs Workflow
• Plugin‐style segmentation methods loaded via factory pattern
Detail without over-splitting
• How the CLI and API wrap the same core modules.
• Snakemake profiles (local vs cluster) under an “Execution Environment” section.