# katherlab/cobra architecture

> Architecture diagram of the GitHub repository katherlab/cobra, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/katherlab/cobra
- Repository: https://github.com/katherlab/cobra

## Overview

1. **Repository purpose**
   - COBRA is a Python package for learning and using slide-level pathology representations from precomputed tile embeddings, not from raw WSIs.
   - The architecture centers on a pretrained representation model, with separate entrypoints for self-supervised pretraining, embedding extraction, downstream cross-validation, and heatmap visualization.

2. **Primary data boundary**
   - The core input is an HDF5-backed collection of tile embeddings produced upstream by external patch encoders and tooling such as STAMP.
   - Metadata flows through CSV or Excel tables for patient/slide IDs and labels.
   - Raw whole-slide images are only needed for visualization in the heatmap path.

3. **Core model subsystem**
   - `cobra/model/cobra.py` defines the main COBRA representation model.
   - It is configured to combine multiple feature streams and uses a Mamba-2-based architecture, with attention/aggregation heads and configurable dimensions.
   - This is the model used both during pretraining and later inference to produce slide or patient embeddings.

4. **Self-supervised pretraining subsystem**
   - `cobra/ssl/pretrain.py` is the training entrypoint for contrastive SSL pretraining.
   - `cobra/ssl/data.py` handles loading tile features from one or more feature bases and presenting them as training samples.
   - `cobra/ssl/model.py` wraps the training objective, which the README describes as MoCo-v3 style contrastive learning.
   - This stage learns COBRA weights from feature-space views rather than from images.

5. **Inference and embedding extraction**
   - `cobra/inference/extract_feats.py` loads a trained checkpoint and turns tile-level features into slide-level or patient-level embeddings.
   - It supports a primary feature directory plus an optional auxiliary feature directory, matching tiles by coordinates before aggregation.
   - Output is written as HDF5 embeddings for later evaluation or deployment.
   - The config-driven interface allows the same code path to serve slide-level and patient-level extraction.

6. **Heatmap generation path**
   - `cobra/inference/heatmaps.py` is the visualization boundary between learned embeddings and WSI rendering.
   - It reads tile feature files and corresponding slide images, computes attention or relevance scores, and overlays them on a slide thumbnail.
   - `cobra/utils/get_mpp.py` supports correct spatial scaling for the heatmap and scale bar.

7. **Downstream evaluation subsystem**
   - `cobra/crossval/train.py` trains a downstream MLP classifier on extracted COBRA embeddings using stratified K-fold cross-validation.
   - `cobra/crossval/deploy.py` evaluates saved fold checkpoints on unseen data and aggregates predictions/metrics.
   - These scripts consume metadata plus HDF5 embeddings, making them the main supervised evaluation boundary of the repo.

8. **Utility layer**
   - `cobra/utils/load_cobra.py` centralizes checkpoint loading and model instantiation.
   - `cobra/utils/mamba2.py` contains the Mamba-2 building blocks used by the core model.
   - `cobra/utils/abmil.py` provides attention-based MIL utilities that support aggregation/interpretability-related logic.

9. **Configuration and orchestration**
   - The repository is organized around YAML-driven workflows, with `cobra/configs/example.yml` as the canonical template.
   - The same pattern is used across pretraining, extraction, heatmaps, and cross-validation, which makes the pipeline reproducible and easy to diagram as config → entrypoint → model/data → artifact.

10. **External dependencies and distribution**
   - Pretrained checkpoints are distributed via Hugging Face, while tile feature generation is expected to come from external encoders.
   - The repo itself is the COBRA layer that sits between upstream feature extraction and downstream slide-level tasks.
   - `pyproject.toml` and `uv.lock` indicate a Python packaging/setup workflow managed with `uv`.

11. **End-to-end architectural flow**
   - Upstream: WSI → external patch encoder(s) → tile embeddings.
   - Middle: tile embeddings → COBRA pretraining or checkpoint inference.
   - Downstream: slide/patient embeddings → cross-validation classifier or heatmap overlay.
   - The main architectural seam is that COBRA never requires raw WSI for representation learning, only for visualization.

Main entry points: [`cobra/inference/heatmaps.py`](https://github.com/katherlab/cobra/blob/main/cobra/inference/heatmaps.py) (Heatmaps), [`cobra/model/cobra.py`](https://github.com/katherlab/cobra/blob/main/cobra/model/cobra.py) (COBRA model), [`cobra/configs/example.yml`](https://github.com/katherlab/cobra/blob/main/cobra/configs/example.yml) (Example config), [`cobra/inference/extract_feats.py`](https://github.com/katherlab/cobra/blob/main/cobra/inference/extract_feats.py) (Extract feats)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core model"]
  node_cobra_model["COBRA model<br/>representation model<br/>[cobra.py]"]
  node_mamba2["Mamba-2 blocks<br/>sequence blocks<br/>[mamba2.py]"]
  node_abmil["MIL utils<br/>attention pooling<br/>[abmil.py]"]
end

subgraph group_ssl["Self-supervised pretraining"]
  node_ssl_pretrain(("Pretrain<br/>ssl entrypoint<br/>[pretrain.py]"))
  node_ssl_data["SSL data<br/>tile loader<br/>[data.py]"]
  node_ssl_objective["MoCo-v3<br/>contrastive objective<br/>[model.py]"]
end

subgraph group_serve["Inference and visualization"]
  node_extract_feats["Extract feats<br/>embedding extraction<br/>[extract_feats.py]"]
  node_heatmaps["Heatmaps<br/>visualization<br/>[heatmaps.py]"]
end

subgraph group_eval["Downstream evaluation"]
  node_crossval_train(("CV train<br/>classifier training<br/>[train.py]"))
  node_crossval_deploy(("CV deploy<br/>evaluation runner<br/>[deploy.py]"))
end

subgraph group_support["Support"]
  node_load_cobra["Load COBRA<br/>checkpoint loader<br/>[load_cobra.py]"]
  node_get_mpp["MPP helper<br/>scale helper<br/>[get_mpp.py]"]
  node_example_cfg["Example config<br/>yaml template<br/>[example.yml]"]
end

node_ssl_pretrain -->|"loads"| node_ssl_data
node_ssl_pretrain -->|"optimizes"| node_ssl_objective
node_ssl_objective -->|"trains"| node_cobra_model
node_cobra_model -->|"uses"| node_mamba2
node_cobra_model -->|"aggregates"| node_abmil
node_extract_feats -->|"loads"| node_load_cobra
node_load_cobra -->|"instantiates"| node_cobra_model
node_extract_feats -->|"configured by"| node_example_cfg
node_heatmaps -->|"loads"| node_load_cobra
node_heatmaps -->|"scales"| node_get_mpp
node_heatmaps -->|"scores"| node_abmil
node_heatmaps -->|"configured by"| node_example_cfg
node_crossval_train -->|"consumes"| node_extract_feats
node_crossval_train -->|"configured by"| node_example_cfg
node_crossval_deploy -->|"loads folds"| node_crossval_train
node_crossval_deploy -->|"configured by"| node_example_cfg

click node_cobra_model "https://github.com/katherlab/cobra/blob/main/cobra/model/cobra.py"
click node_mamba2 "https://github.com/katherlab/cobra/blob/main/cobra/utils/mamba2.py"
click node_abmil "https://github.com/katherlab/cobra/blob/main/cobra/utils/abmil.py"
click node_ssl_pretrain "https://github.com/katherlab/cobra/blob/main/cobra/ssl/pretrain.py"
click node_ssl_data "https://github.com/katherlab/cobra/blob/main/cobra/ssl/data.py"
click node_ssl_objective "https://github.com/katherlab/cobra/blob/main/cobra/ssl/model.py"
click node_extract_feats "https://github.com/katherlab/cobra/blob/main/cobra/inference/extract_feats.py"
click node_heatmaps "https://github.com/katherlab/cobra/blob/main/cobra/inference/heatmaps.py"
click node_load_cobra "https://github.com/katherlab/cobra/blob/main/cobra/utils/load_cobra.py"
click node_get_mpp "https://github.com/katherlab/cobra/blob/main/cobra/utils/get_mpp.py"
click node_example_cfg "https://github.com/katherlab/cobra/blob/main/cobra/configs/example.yml"
click node_crossval_train "https://github.com/katherlab/cobra/blob/main/cobra/crossval/train.py"
click node_crossval_deploy "https://github.com/katherlab/cobra/blob/main/cobra/crossval/deploy.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cobra_model,node_mamba2,node_abmil toneBlue
class node_ssl_pretrain,node_ssl_data,node_ssl_objective toneAmber
class node_extract_feats,node_heatmaps toneMint
class node_crossval_train,node_crossval_deploy toneRose
class node_load_cobra,node_get_mpp,node_example_cfg toneIndigo
```

## Components

### Core model

Representation model and its building blocks.

- [COBRA model](https://github.com/katherlab/cobra/blob/main/cobra/model/cobra.py): representation model · Main slide/patient embedding model built around Mamba-2 and aggregation heads. · `cobra/model/cobra.py`
- [Mamba-2 blocks](https://github.com/katherlab/cobra/blob/main/cobra/utils/mamba2.py): sequence blocks · Core sequence modeling components used by the representation model. · `cobra/utils/mamba2.py`
- [MIL utils](https://github.com/katherlab/cobra/blob/main/cobra/utils/abmil.py): attention pooling · Attention-based multiple-instance learning utilities for aggregation and interpretability. · `cobra/utils/abmil.py`

### Self-supervised pretraining

Feature-space contrastive training pipeline.

- [Pretrain](https://github.com/katherlab/cobra/blob/main/cobra/ssl/pretrain.py): ssl entrypoint · Self-supervised contrastive pretraining entrypoint. · `cobra/ssl/pretrain.py`
- [SSL data](https://github.com/katherlab/cobra/blob/main/cobra/ssl/data.py): tile loader · Loads tile embeddings from one or more feature bases for SSL training. · `cobra/ssl/data.py`
- [MoCo-v3](https://github.com/katherlab/cobra/blob/main/cobra/ssl/model.py): contrastive objective · Training wrapper and objective used for self-supervised learning. · `cobra/ssl/model.py`

### Inference and visualization

Checkpoint loading, embedding extraction, and heatmap rendering.

- [Extract feats](https://github.com/katherlab/cobra/blob/main/cobra/inference/extract_feats.py): embedding extraction · Loads a checkpoint and aggregates tile features into slide or patient embeddings. · `cobra/inference/extract_feats.py`
- [Heatmaps](https://github.com/katherlab/cobra/blob/main/cobra/inference/heatmaps.py): visualization · Computes relevance scores and overlays them on slide thumbnails. · `cobra/inference/heatmaps.py`

### Downstream evaluation

Cross-validation training and deployment on extracted embeddings.

- [CV train](https://github.com/katherlab/cobra/blob/main/cobra/crossval/train.py): classifier training · Trains downstream MLP classifiers with stratified K-fold validation. · `cobra/crossval/train.py`
- [CV deploy](https://github.com/katherlab/cobra/blob/main/cobra/crossval/deploy.py): evaluation runner · Loads saved fold checkpoints and aggregates predictions and metrics. · `cobra/crossval/deploy.py`

### Support

Reusable utilities and configuration.

- [Load COBRA](https://github.com/katherlab/cobra/blob/main/cobra/utils/load_cobra.py): checkpoint loader · Centralized model instantiation and checkpoint loading helper. · `cobra/utils/load_cobra.py`
- [MPP helper](https://github.com/katherlab/cobra/blob/main/cobra/utils/get_mpp.py): scale helper · Resolves microns-per-pixel for correct spatial scaling in heatmaps. · `cobra/utils/get_mpp.py`
- [Example config](https://github.com/katherlab/cobra/blob/main/cobra/configs/example.yml): yaml template · Canonical configuration template for the repo workflows. · `cobra/configs/example.yml`

## Connections

- Pretrain → SSL data: loads · Training samples are built from tile embeddings.
- Pretrain → MoCo-v3: optimizes · Pretraining uses the contrastive objective wrapper.
- MoCo-v3 → COBRA model: trains · Learns the main COBRA representation model.
- COBRA model → Mamba-2 blocks: uses · Representation layers depend on Mamba-2 building blocks.
- COBRA model → MIL utils: aggregates · Aggregation and interpretability rely on MIL-style attention logic.
- Extract feats → Load COBRA: loads · Inference starts by loading a trained checkpoint and model.
- Load COBRA → COBRA model: instantiates · Creates the COBRA model for inference or deployment.
- Extract feats → Example config: configured by · Extraction is driven by the YAML workflow configuration.
- Heatmaps → Load COBRA: loads · Visualization can use a trained checkpoint for scoring.
- Heatmaps → MPP helper: scales · Heatmap overlays use microns-per-pixel for spatial calibration.
- Heatmaps → MIL utils: scores · Attention/relevance scoring uses aggregation logic from the MIL layer.
- Heatmaps → Example config: configured by · Heatmap generation follows the same config-driven pattern.
- CV train → Extract feats: consumes · Downstream training uses extracted COBRA embeddings as input.
- CV train → Example config: configured by · Cross-validation workflows are YAML-driven.
- CV deploy → CV train: loads folds · Deployment evaluates saved checkpoints from training folds.
- CV deploy → Example config: configured by · Deployment is also controlled by the shared config pattern.
