sbrussee/pathforge

Purpose and Entry Points

PathForge is a config-driven Python framework for computational-pathology workflows: WSI tiling/feature extraction, H5 artifact production, MIL training/benchmarking/optimization, slide retrieval, and packaged-model inference. The primary CLI surface is the umbrella app in src/pathforge/cli/app.py, with workflow commands under src/pathforge/cli/ and default runnable templates in default_config/.

Configuration, Policies, and Experiment Control

YAML is parsed and validated into the central configuration model, then mode-specific policies coordinate work. Feature extraction, benchmarking, inference, and Optuna optimization are policy boundaries rather than CLI business logic.

• src/pathforge/config/config.py

• src/pathforge/policy/feature_extraction.py

• src/pathforge/policy/benchmarking.py

• src/pathforge/policy/optimization.py

• src/pathforge/policy/inference.py

Dataset and Artifact Boundary

Annotation rows define datasets and slide identity. WSI datasets resolve slide files and metadata; bag datasets load feature bags and labels for MIL. Per-slide H5 artifacts are the durable interchange format between extraction, training, retrieval, reporting, and inference. Their core invariant is row alignment between tile coordinates and feature matrices.

• src/pathforge/core/datasets/wsi_dataset.py

• src/pathforge/core/datasets/bag_dataset.py

• src/pathforge/core/datasets/bag_schema.py

• src/pathforge/core/io/h5/features.py

• src/pathforge/core/io/h5/layout.py

Slide Processing and Feature Extraction

A registry-selected slide processor performs tissue segmentation, tiling, optional tile-overview generation, and feature extraction. The bundled LazySlide implementation is the main WSI backend; feature extractors are independently selected through factories/registries. Processing emits reusable H5 bags keyed by tiling and extractor choices.

• src/pathforge/core/slide_processing/factory.py

• src/pathforge/core/slide_processing/lazyslide/processor.py

• src/pathforge/core/slide_processing/lazyslide/feature_extraction_patch.py

• src/pathforge/core/feature_extractors/factory.py

• src/pathforge/core/feature_extractors/registry.py

MIL Training, Tasks, and Backend Adapters

Task definitions normalize classification, regression, survival, and retrieval behavior. Native models and trainers operate on canonical bag dictionaries; Lightning is the principal deep-learning runtime, with a sklearn path for slide-level models. Registries decouple policies from concrete models, losses, trainers, metrics, and explainers.

• src/pathforge/core/tasks/mil.py

• src/pathforge/core/models/mil_base.py

• src/pathforge/training/lightning.py

• src/pathforge/training/sklearn_trainer.py

• src/pathforge/utils/registries.py

Optional integrations are deliberately isolated: TorchMIL and MIL-Lab model adapters, plus TorchMetrics/TorchSurv metric/loss adapters, prevent optional dependencies from entering core domain and training code.

• src/pathforge/adapters/torchmil/backend.py

• src/pathforge/adapters/mil_lab/backend.py

• src/pathforge/adapters/metrics/classification.py

• src/pathforge/utils/optional/

Evaluation, Optimization, and Distributed Execution

Benchmark and optimization policies enumerate or sample pipeline combinations, train/evaluate them, and produce normalized ranked results. Evaluation and visualization use registries/orchestrators. Optuna is the search/pruning service; distributed planning/execution supports local or cluster-oriented staged workloads, with SLURM scripts supplied.

• src/pathforge/core/evaluation/orchestrator.py

• src/pathforge/core/experiments/combinations.py

• src/pathforge/execution/distributed.py

• scripts/run_distributed_smoke.sbatch

Retrieval, Inference, and Outputs

Slide retrieval consumes existing H5 bags, creates configurable representations, searches reference pools, aggregates rankings, evaluates metrics, and renders results—without training. Packaged-model inference reads H5 bags and can persist normalized per-instance heatmaps in the artifact prediction namespace.

• src/pathforge/core/tasks/slide_retrieval.py

• src/pathforge/slide_retrieval/representation_strategies/registry.py

• src/pathforge/slide_retrieval/search_strategies/registry.py

• src/pathforge/inference/model_package.py

• src/pathforge/inference/heatmaps.py