How porterlin/cxr-clip-cgmh works
Project type and purpose
• This is a machine‐learning system for large‐scale chest X-ray language-image pre‐training (“CXR-CLIP-CGMH”), implemented in Python using PyTorch and Hydra.
• It supports:
• Zero-shot image–text contrastive pre-training (CLIP-style)
• Linear probing fine-tuning on downstream tasks (classification)
• Evaluation (zero-shot, fine-tune) and single-image inference
File structure
• Top‐level scripts (entrypoints):
• train.py (pre-training)
• finetune.py (linear probe training)
• evaluate_clip.py (zero-shot eval)
• evaluate_finetune.py (fine-tune eval)
• inference.py (single image)
• Configuration (Hydra + YAML):
• configs/: data_train / data_valid / data_test / dataloader / model / optimizer / scheduler / loss / tokenizer / transform / train.yaml
• Core code module (cxrclip/):
• cxrclip/data: data module, datasets, transforms, tokenization, prompt generation
• cxrclip/model: image_encoder, text_encoder, projection heads, classification module
• cxrclip/loss: contrastive loss, classification loss, combined loss
• cxrclip/optimizer & scheduler: custom wrappers (warmup cosine)
• cxrclip/prompt: constants & prompt templates
• cxrclip/trainer: orchestrates forward/backward, logging, checkpointing, distributed
• cxrclip/util: distributed utils, summary writer, global env
• datasets/: static JSON/README for prompt definitions
• text_augmentation/: optional back-translation for text
• requirements.txt, .gitignore, LICENSE
README insights
• Uses Hydra to modularly select data splits, models, optimizers, schedulers, losses.
• Supports single-GPU and multi-GPU (torchrun + NCCL)
• Prompts are loaded from JSON, can sample positive/negative/uncertain templates