How stefanowoerner/medimeta-pytorch works
1. Repository purpose
- This is a small PyTorch library, not a model repository.
- Its main job is to expose the external MedIMeta medical imaging dataset through a unified Python API and make it usable for single-task, multi-task, few-shot, and cross-domain experiments.
2. Core boundary: code vs. data
- The repository contains the access layer only.
- The actual dataset lives outside the repo and is downloaded separately from Zenodo.
- The library expects a local dataset root path and reads standardized task data from disk.
3. Primary runtime dependency
- The package is built around PyTorch and specifically follows the torch.utils.data.Dataset pattern.
- This means the central abstraction is a dataset object that returns (image, label) samples in a form directly consumable by PyTorch training loops.
4. Main entry point
- medimeta/medimeta.py is the key module.
- It likely defines the MedIMeta dataset class described in the README, which provides a unified interface for selecting:
- a dataset/domain,
- a task within that dataset,
- and the corresponding labeled samples.
5. Task and metadata layer
- medimeta/tasks.py appears to encode the task catalog and task-specific metadata.
- This is where the repository likely standardizes the many underlying medical datasets into a common task format, including dataset names, task names, and class/label structure.
6. Batching and sampling utilities
- medimeta/batches.py and medimeta/task_presampling.py support multi-task and few-shot workflows.
- These modules are the bridge from “one dataset object” to “task-aware training episodes or batches,” which is important for cross-domain learning and meta-learning setups.
7. Logging/support code
- medimeta/logger.py is a small supporting subsystem for runtime messages or experiment feedback.
- It is auxiliary rather than architectural, but it likely helps surface dataset loading or sampling behavior during experiments.
8. Package surface
- medimeta/__init__.py is the public API boundary.
- It likely re-exports the main classes and helpers so users can import from medimeta directly instead of touching internal modules.
9. Example-driven usage model
- The examples/ directory shows the intended integration patterns:
- plain supervised learning,
- ImageNet-pretrained evaluation on few-shot tasks,
- cross-domain pretraining,
- cross-domain MAML.
- These examples demonstrate that MedIMeta is designed to plug into broader ML pipelines rather than provide a full training framework itself.
10. External integration point: TorchCross
- The README explicitly says MedIMeta integrates with TorchCross.
- Architecturally, MedIMeta provides the dataset/task abstractions, while TorchCross provides the learning algorithms and experiment scaffolding for cross-domain and meta-learning.
11. Standardization assumptions
- The dataset is already standardized to 224×224 images and comes with predefined splits.
- This reduces preprocessing complexity inside the library; the architecture assumes consistent task formatting rather than performing heavy on-the-fly data harmonization.
12. Overall data flow
- External dataset files on disk → MedIMeta dataset object → task-specific sample retrieval → optional task sampling/batching helpers → PyTorch/TorchCross training or evaluation code.
- The repository’s main architectural role is to normalize many medical image tasks into one consistent access pattern.
13. Suggested diagram focus
- Draw three main layers:
- Dataset storage: local MedIMeta files from Zenodo,
- MedIMeta package: dataset/task/sampling utilities,
- Experiment layer: PyTorch and TorchCross consumers in the example scripts.
- The most important boundaries are between stored data, standardized task metadata, and downstream learning workflows.
Main entry points: __init__.py API surface · medimeta.py MedIMeta · tasks.py Task catalog · batches.py Batches
Components 13
Dataset storage
External MedIMeta files downloaded from Zenodo and stored locally.
- Zenodo data · Local MedIMeta dataset root populated from Zenodo downloads.
MedIMeta package
Core dataset access, task metadata, batching, and support code.
- MedIMeta · Main torch.utils.data.Dataset interface for selecting datasets, tasks, and samples.
medimeta/medimeta.py - Task catalog · Standardized mapping of dataset names, tasks, and label structure.
medimeta/tasks.py - Batches · Helpers for task-aware batching and multi-task episode construction.
medimeta/batches.py - Presampling · Utilities for few-shot and task-presampled data access.
medimeta/task_presampling.py - Logger · Lightweight runtime messaging and loading feedback.
medimeta/logger.py - API surface · Public import boundary that re-exports the main package objects.
medimeta/__init__.py
Experiments
Example entry points showing how the library plugs into training and evaluation workflows.
- Supervised · Plain supervised learning usage pattern.
examples/fully_supervised.py - ImageNet eval · Few-shot evaluation with ImageNet-pretrained features.
examples/imagenet_pretrained.py - Pretraining · Cross-domain pretraining workflow.
examples/cross_domain_pretraining.py - MAML · Cross-domain MAML workflow.
examples/cross_domain_maml.py - PyTorch · Downstream torch training loops consuming dataset samples.
- TorchCross · External framework for cross-domain and meta-learning experiments.
Connections 19
- Zenodo data → MedIMeta: loaded by · The dataset object reads standardized samples from the local dataset root.
- Task catalog → MedIMeta: drives · Task metadata defines which dataset/task/labels the dataset object serves.
- MedIMeta → Batches: feeds · The base dataset is wrapped or organized into task-aware batches.
- MedIMeta → Presampling: feeds · Task-presampled access is built on top of the dataset interface.
- Task catalog → Batches: configures · Batching logic depends on standardized task definitions.
- Task catalog → Presampling: configures · Few-shot sampling uses task and class metadata.
- MedIMeta → Logger: reports · Loading and sampling events can be surfaced through logging.
- API surface → MedIMeta: exports · Public package imports expose the main dataset class.
- API surface → Task catalog: exports · Public package imports expose task metadata helpers.
- API surface → Batches: exports · Public package imports expose batching helpers.
- API surface → Presampling: exports · Public package imports expose task-presampling helpers.
- Supervised → PyTorch: uses · Example shows direct supervised consumption in PyTorch.
- ImageNet eval → PyTorch: uses · Example evaluates few-shot tasks inside a standard torch pipeline.
- Pretraining → TorchCross: uses · Example demonstrates cross-domain pretraining with TorchCross.
- MAML → TorchCross: uses · Example demonstrates meta-learning with TorchCross.
- Supervised → API surface: imports · Examples are intended to import from the package surface.
- ImageNet eval → API surface: imports · Examples are intended to import from the package surface.
- Pretraining → API surface: imports · Examples are intended to import from the package surface.
- MAML → API surface: imports · Examples are intended to import from the package surface.