# huyhieupham/midl2021-ct-classification architecture

> Architecture diagram of the GitHub repository huyhieupham/midl2021-ct-classification, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/huyhieupham/midl2021-ct-classification
- Repository: https://github.com/huyhieupham/midl2021-ct-classification

## Overview

##### Project Type and Purpose
- This is a research‐oriented machine‐learning pipeline for classifying multi‐phase abdominal CT scans. It provides both 2D slice‐wise and 3D scan‐level training, validation, and inference workflows, along with auxiliary study‐level evaluation scripts and notebooks.

##### High-Level Architecture Components
a. Entry Points
- main.py – launches 2D training/evaluation
- main3d.py – launches 3D training/evaluation
- study_evaluation/\*.py and notebooks – auxiliary analysis, sampling, and report generation
b. Core Library (core/)
- config/ – YAML configurations and default_cfg loader
- dataset/ – DICOM preprocessing, HU conversion, 2D & 3D Dataset classes
- model/ – model definitions, training/validation loops, grad-CAM, prediction logging
- utils/ – argument parsing, checkpointing, logging, loss/metric computations, optimizer/scheduler setup, RNG seeding
c. Data Sources & Outputs
- Raw DICOM files on disk
- CSV manifests for 2D and 3D experiments
- Preprocessed image tensors
- Model checkpoints, logs, metrics outputs
- Study-level reports and figures in images/ and study_evaluation/
d. External Dependencies
- PyTorch (training, tensor ops)
- PyYAML (config parsing)
- NumPy, Pandas (data handling)
- pydicom or custom DICOM reader (implied)
- Jupyter for notebooks

##### Relationships and Data Flow
1. User invokes CLI (main.py or main3d.py) with a path to a YAML config.
2. Config loader reads the YAML into default_cfg and overrides.
3. Dataset module loads CSV manifest, reads raw .dcm files, applies HU conversion and windowing.
4. DataLoader feeds batches to the model.
5. Training loop in model/train_model.py executes forward/backward passes. Checkpoints and logs are written via utils/checkpoint and utils/logging.
6. Validation/test loop (model/valid_model.py / test_model.py) computes metrics and writes summaries.
7. For 3D, main3d.py triggers core/model/train_model with 3D data pipeline (core/dataset/data_3d.py).
8. study_evaluation scripts/notebooks consume saved checkpoints to run inference (inference.py, infer_lits.py), random sampling (sampling.py), per-study aggregation (study_evaluate.py), and reporting (Report.ipynb).

##### Architectural Patterns and Principles
- Modularization: separation of config, data, model, and utilities
- Dependency Injection: config objects passed to dataset/model/train loops
- Single‐responsibility: each module focuses on a discrete task (e.g., data loading vs. model logic)
- Extensibility: New architectures added by new YAML config and subclassing model.py

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Entry Points
    subgraph "Entry Points"
        direction TB
        main2d["main.py"]:::entry
        main3d["main3d.py"]:::entry
    end

    %% Configuration
    configLoader["Config Loader\n(default_cfg & YAML)"]:::core

    %% 2D Pipeline
    subgraph "2D Pipeline"
        direction TB
        dataset2d["2D Dataset Preprocessor\n(data.py)"]:::core
        dataLoader2d["DataLoader"]:::core
        trainLoop2d["Train Loop 2D\n(train_model.py)"]:::core
        evalLoop2d["Eval/Test Loop\n(valid_model.py & test_model.py)"]:::core
    end

    %% 3D Pipeline
    subgraph "3D Pipeline"
        direction TB
        dataset3d["3D Dataset Preprocessor\n(data_3d.py)"]:::core
        dataLoader3d["DataLoader"]:::core
        trainLoop3d["Train Loop 3D\n(train_model.py)"]:::core
        evalLoop3d["Eval/Test Loop\n(valid_model.py & test_model.py)"]:::core
    end

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        rawDICOM["Raw DICOM Files"]:::datastore
        csvManifests["CSV Manifests"]:::datastore
        preprocessedTensors["Preprocessed Tensors"]:::datastore
        checkpointStore["Model Checkpoints"]:::datastore
        metricsStore["Metrics Store"]:::datastore
        reports["Reports & Figures"]:::datastore
    end

    %% Study Evaluation
    subgraph "Study Evaluation"
        direction TB
        studyEval["Study Evaluation\n(scripts & notebooks)"]:::entry
    end

    %% External Dependencies
    subgraph "External Libraries"
        direction TB
        pytorch["PyTorch"]:::external
        pyyaml["PyYAML"]:::external
        numpy["NumPy/Pandas"]:::external
        dicomReader["pydicom"]:::external
        jupyter["Jupyter"]:::external
    end

    %% Connections
    main2d -->|"load config"| configLoader
    main3d -->|"load config"| configLoader

    configLoader -->|"reads CSV & DICOM"| dataset2d
    configLoader -->|"reads CSV & DICOM"| dataset3d

    rawDICOM --> dataset2d
    csvManifests --> dataset2d
    dataset2d -->|"tensors"| preprocessedTensors
    preprocessedTensors --> dataLoader2d
    dataLoader2d --> trainLoop2d
    trainLoop2d --> checkpointStore
    trainLoop2d --> evalLoop2d
    evalLoop2d --> metricsStore

    rawDICOM --> dataset3d
    csvManifests --> dataset3d
    dataset3d -->|"tensors"| preprocessedTensors
    preprocessedTensors --> dataLoader3d
    dataLoader3d --> trainLoop3d
    trainLoop3d --> checkpointStore
    trainLoop3d --> evalLoop3d
    evalLoop3d --> metricsStore

    checkpointStore --> studyEval
    studyEval --> reports

    %% External deps arrows
    configLoader --> pyyaml
    dataset2d --> dicomReader
    dataset3d --> dicomReader
    trainLoop2d --> pytorch
    trainLoop3d --> pytorch
    evalLoop2d --> pytorch
    evalLoop3d --> pytorch
    dataLoader2d --> numpy
    dataLoader3d --> numpy

    %% Click Events
    click main2d "https://github.com/huyhieupham/midl2021-ct-classification/blob/main/main.py"
    click main3d "https://github.com/huyhieupham/midl2021-ct-classification/blob/main/main3d.py"
    click configLoader "https://github.com/huyhieupham/midl2021-ct-classification/blob/main/core/config/default_cfg.py"
    click dataset2d "https://github.com/huyhieupham/midl2021-ct-classification/blob/main/core/dataset/data.py"
    click dataset3d "https://github.com/huyhieupham/midl2021-ct-classification/blob/main/core/dataset/data_3d.py"
    click trainLoop2d "https://github.com/huyhieupham/midl2021-ct-classification/blob/main/core/model/train_model.py"
    click trainLoop3d "https://github.com/huyhieupham/midl2021-ct-classification/blob/main/core/model/train_model.py"
    click studyEval "https://github.com/huyhieupham/midl2021-ct-classification/blob/main/study_evaluation/study_evaluate.py"

    %% Styles
    classDef entry fill:#f39c12,stroke:#333,stroke-width:1px
    classDef core fill:#27ae60,stroke:#333,stroke-width:1px
    classDef datastore fill:#3498db,stroke:#333,stroke-width:1px
    classDef external fill:#95a5a6,stroke:#333,stroke-width:1px
```
