How katherlab/cobra works
1. Repository purpose
- COBRA is a Python package for learning and using slide-level pathology representations from precomputed tile embeddings, not from raw WSIs.
- The architecture centers on a pretrained representation model, with separate entrypoints for self-supervised pretraining, embedding extraction, downstream cross-validation, and heatmap visualization.
2. Primary data boundary
- The core input is an HDF5-backed collection of tile embeddings produced upstream by external patch encoders and tooling such as STAMP.
- Metadata flows through CSV or Excel tables for patient/slide IDs and labels.
- Raw whole-slide images are only needed for visualization in the heatmap path.
3. Core model subsystem
- cobra/model/cobra.py defines the main COBRA representation model.
- It is configured to combine multiple feature streams and uses a Mamba-2-based architecture, with attention/aggregation heads and configurable dimensions.
- This is the model used both during pretraining and later inference to produce slide or patient embeddings.
4. Self-supervised pretraining subsystem
- cobra/ssl/pretrain.py is the training entrypoint for contrastive SSL pretraining.
- cobra/ssl/data.py handles loading tile features from one or more feature bases and presenting them as training samples.
- cobra/ssl/model.py wraps the training objective, which the README describes as MoCo-v3 style contrastive learning.
- This stage learns COBRA weights from feature-space views rather than from images.
5. Inference and embedding extraction
- cobra/inference/extract_feats.py loads a trained checkpoint and turns tile-level features into slide-level or patient-level embeddings.
- It supports a primary feature directory plus an optional auxiliary feature directory, matching tiles by coordinates before aggregation.
- Output is written as HDF5 embeddings for later evaluation or deployment.
- The config-driven interface allows the same code path to serve slide-level and patient-level extraction.
6. Heatmap generation path
- cobra/inference/heatmaps.py is the visualization boundary between learned embeddings and WSI rendering.
- It reads tile feature files and corresponding slide images, computes attention or relevance scores, and overlays them on a slide thumbnail.
- cobra/utils/get_mpp.py supports correct spatial scaling for the heatmap and scale bar.
7. Downstream evaluation subsystem
- cobra/crossval/train.py trains a downstream MLP classifier on extracted COBRA embeddings using stratified K-fold cross-validation.
- cobra/crossval/deploy.py evaluates saved fold checkpoints on unseen data and aggregates predictions/metrics.
- These scripts consume metadata plus HDF5 embeddings, making them the main supervised evaluation boundary of the repo.
8. Utility layer
- cobra/utils/load_cobra.py centralizes checkpoint loading and model instantiation.
- cobra/utils/mamba2.py contains the Mamba-2 building blocks used by the core model.
- cobra/utils/abmil.py provides attention-based MIL utilities that support aggregation/interpretability-related logic.
9. Configuration and orchestration
- The repository is organized around YAML-driven workflows, with cobra/configs/example.yml as the canonical template.
- The same pattern is used across pretraining, extraction, heatmaps, and cross-validation, which makes the pipeline reproducible and easy to diagram as config → entrypoint → model/data → artifact.
10. External dependencies and distribution
- Pretrained checkpoints are distributed via Hugging Face, while tile feature generation is expected to come from external encoders.
- The repo itself is the COBRA layer that sits between upstream feature extraction and downstream slide-level tasks.
- pyproject.toml and uv.lock indicate a Python packaging/setup workflow managed with uv.
11. End-to-end architectural flow
- Upstream: WSI → external patch encoder(s) → tile embeddings.
- Middle: tile embeddings → COBRA pretraining or checkpoint inference.
- Downstream: slide/patient embeddings → cross-validation classifier or heatmap overlay.
- The main architectural seam is that COBRA never requires raw WSI for representation learning, only for visualization.
Main entry points: heatmaps.py Heatmaps · cobra.py COBRA model · example.yml Example config · extract_feats.py Extract feats
Components 13
Core model
Representation model and its building blocks.
- COBRA model · Main slide/patient embedding model built around Mamba-2 and aggregation heads.
cobra/model/cobra.py - Mamba-2 blocks · Core sequence modeling components used by the representation model.
cobra/utils/mamba2.py - MIL utils · Attention-based multiple-instance learning utilities for aggregation and interpretability.
cobra/utils/abmil.py
Self-supervised pretraining
Feature-space contrastive training pipeline.
Inference and visualization
Checkpoint loading, embedding extraction, and heatmap rendering.
- Extract feats · Loads a checkpoint and aggregates tile features into slide or patient embeddings.
cobra/inference/extract_feats.py - Heatmaps · Computes relevance scores and overlays them on slide thumbnails.
cobra/inference/heatmaps.py
Downstream evaluation
Cross-validation training and deployment on extracted embeddings.
Support
Reusable utilities and configuration.
- Load COBRA · Centralized model instantiation and checkpoint loading helper.
cobra/utils/load_cobra.py - MPP helper · Resolves microns-per-pixel for correct spatial scaling in heatmaps.
cobra/utils/get_mpp.py - Example config · Canonical configuration template for the repo workflows.
cobra/configs/example.yml
Connections 16
- Pretrain → SSL data: loads · Training samples are built from tile embeddings.
- Pretrain → MoCo-v3: optimizes · Pretraining uses the contrastive objective wrapper.
- MoCo-v3 → COBRA model: trains · Learns the main COBRA representation model.
- COBRA model → Mamba-2 blocks: uses · Representation layers depend on Mamba-2 building blocks.
- COBRA model → MIL utils: aggregates · Aggregation and interpretability rely on MIL-style attention logic.
- Extract feats → Load COBRA: loads · Inference starts by loading a trained checkpoint and model.
- Load COBRA → COBRA model: instantiates · Creates the COBRA model for inference or deployment.
- Extract feats → Example config: configured by · Extraction is driven by the YAML workflow configuration.
- Heatmaps → Load COBRA: loads · Visualization can use a trained checkpoint for scoring.
- Heatmaps → MPP helper: scales · Heatmap overlays use microns-per-pixel for spatial calibration.
- Heatmaps → MIL utils: scores · Attention/relevance scoring uses aggregation logic from the MIL layer.
- Heatmaps → Example config: configured by · Heatmap generation follows the same config-driven pattern.
- CV train → Extract feats: consumes · Downstream training uses extracted COBRA embeddings as input.
- CV train → Example config: configured by · Cross-validation workflows are YAML-driven.
- CV deploy → CV train: loads folds · Deployment evaluates saved checkpoints from training folds.
- CV deploy → Example config: configured by · Deployment is also controlled by the shared config pattern.