How huyhieupham/midl2021-ct-classification works
Project Type and Purpose
• This is a research‐oriented machine‐learning pipeline for classifying multi‐phase abdominal CT scans. It provides both 2D slice‐wise and 3D scan‐level training, validation, and inference workflows, along with auxiliary study‐level evaluation scripts and notebooks.
High-Level Architecture Components
a. Entry Points
• main.py – launches 2D training/evaluation
• main3d.py – launches 3D training/evaluation
• study_evaluation/\*.py and notebooks – auxiliary analysis, sampling, and report generation
b. Core Library (core/)
• config/ – YAML configurations and default_cfg loader
• dataset/ – DICOM preprocessing, HU conversion, 2D & 3D Dataset classes
• model/ – model definitions, training/validation loops, grad-CAM, prediction logging
• utils/ – argument parsing, checkpointing, logging, loss/metric computations, optimizer/scheduler setup, RNG seeding
c. Data Sources & Outputs
• Raw DICOM files on disk
• CSV manifests for 2D and 3D experiments
• Preprocessed image tensors
• Model checkpoints, logs, metrics outputs
• Study-level reports and figures in images/ and study_evaluation/
d. External Dependencies
• PyTorch (training, tensor ops)
• PyYAML (config parsing)
• NumPy, Pandas (data handling)
• pydicom or custom DICOM reader (implied)
• Jupyter for notebooks
Relationships and Data Flow
1. User invokes CLI (main.py or main3d.py) with a path to a YAML config.
2. Config loader reads the YAML into default_cfg and overrides.
3. Dataset module loads CSV manifest, reads raw .dcm files, applies HU conversion and windowing.
4. DataLoader feeds batches to the model.
5. Training loop in model/train_model.py executes forward/backward passes. Checkpoints and logs are written via utils/checkpoint and utils/logging.
6. Validation/test loop (model/valid_model.py / test_model.py) computes metrics and writes summaries.
7. For 3D, main3d.py triggers core/model/train_model with 3D data pipeline (core/dataset/data_3d.py).
8. study_evaluation scripts/notebooks consume saved checkpoints to run inference (inference.py, infer_lits.py), random sampling (sampling.py), per-study aggregation (study_evaluate.py), and reporting (Report.ipynb).
Architectural Patterns and Principles
• Modularization: separation of config, data, model, and utilities
• Dependency Injection: config objects passed to dataset/model/train loops
• Single‐responsibility: each module focuses on a discrete task (e.g., data loading vs. model logic)
• Extensibility: New architectures added by new YAML config and subclassing model.py