# sposso/deep_learning_to_improve_breast_cancer_pytorch architecture

> Architecture diagram of the GitHub repository sposso/deep_learning_to_improve_breast_cancer_pytorch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sposso/deep_learning_to_improve_breast_cancer_pytorch
- Repository: https://github.com/sposso/deep_learning_to_improve_breast_cancer_pytorch

## Overview

##### Project Type and Purpose
- This is a research‐oriented deep learning pipeline in Python (PyTorch) for breast cancer detection on mammograms.
- It implements a two‐stage classifier:
- Patch‐level classifier (ResNet50 backbone)
- Whole‐image classifier (fine-tuned with additional ResNet blocks)
- It also provides preprocessing (watermark removal, patch extraction), model training, inference, and heatmap generation.

##### High-Level Components
A. Data Ingestion & Splitting
- Folder: /data
- Key files: split.py, sample.csv, train.csv, validation.csv, test.csv
- Output: stratified CSV partitions
B. Preprocessing & Patch Generation
- Folder: /Patches
- tools.py (segment_breast), generating_patches.py
- Output: PNG patches saved to disk, organized by class
C. Patch-Level Classifier Training
- Folder: /patch_classifier
- train_function.py, tools.py
- Backbone: ResNet50 (pretrained on ImageNet)
- Staged training (three stages, gradual layer unfreezing)
- Output: best_s_patch_model.pt, best_s10_patch_model.pt in /trained_models
D. Whole-Image Classifier Training
- Folder: /whole_image_classifier
- whole_classifier_model.py, main.py, tools.py
- Reuses patch model without top layers + two ResNet blocks + new FC
- Two‐stage training strategy
- Output: best_whole_image_classifier.pt in /trained_models
E. Heatmap Generation (Inference)
- Folder: /Heatmaps
- main.py (my_prob_heatmap), compact_heatmap_func.py, tools.py
- Sliding‐window inference using patch classifier → heatmap matrix
F. Shared Utilities
- Folder: /utils
- train_util.py, classifier.py, tools.py
- Common data loaders, training loops, metric calculators
G. Orchestration & Entry Points
- top-level main.py
- README.md describes usage steps and dependencies

##### Relationships & Data Flow
1. Raw Mammogram Data (DICOM) → Data Ingestion & Splitting
2. Splitted Data CSVs → Preprocessing & Patch Generation → Patch Dataset (PNG patches)
3. Patch Dataset → Patch-Level Classifier Training → Patch Models in /trained_models
4. Patch Model → Whole-Image Classifier Training → Whole Image Model in /trained_models
5. Patch Model + Full Mammogram → Heatmap Generation → Heatmap Outputs

##### Architectural Patterns & Principles
- Modular pipeline: separate folders for each stage (Single Responsibility).
- Reusable utilities: common code in /utils.
- Layered training strategy: freeze/unfreeze layers for controlled fine-tuning.
- Clear data flow: raw → preprocess → train → inference.

##### Technologies & Frameworks
- PyTorch (model definitions, training loops)
- NumPy/Pandas (CSV handling, array ops)
- PIL or OpenCV (PNG I/O, image resizing)
- DICOM readers (in Patches/tools.py for watermark removal)
- CLI scripts (Python entry points).

##### Final Checklist
- [ ] All seven main components labeled
- [ ] Saving/loading of models represented explicitly
- [ ] Utility module connections shown
- [ ] Annotations for important parameters (batch sizes, epochs, learning rates)

## Diagram (Mermaid)

```mermaid
flowchart TD
    RawData(["Raw Mammogram Data\n(DICOM)"]):::data
    DataIngestion(["Data Ingestion & Splitting\n- split.py\n- sample.csv\n- train.csv\n- validation.csv\n- test.csv"]):::data
    Preprocessing(["Preprocessing & Patch Generation\n- tools.py\n- generating_patches.py"]):::data
    PatchTraining(["Patch-Level Classifier Training\n- train_function.py\n- tools.py"]):::training
    WholeImage(["Whole-Image Classifier Training\n- main.py\n- tools.py\n- whole_classifier_model.py"]):::training
    Heatmap(["Heatmap Generation\n- main.py\n- compact_heatmap_func.py\n- tools.py"]):::inference
    SharedUtils(["Shared Utilities\n- train_util.py\n- classifier.py\n- tools.py"]):::util
    Orchestration(["Orchestration & Entry Points\n- main.py\n- README.md"]):::orchestration
    TrainedModels[(Trained Models\n/ trained_models\n- best_s_patch_model.pt\n- best_s10_patch_model.pt\n- best_whole_image_classifier.pt)]:::util

    RawData -->|DICOM| DataIngestion
    DataIngestion -->|CSV splits| Preprocessing
    Preprocessing -->|224×224 patches| PatchTraining
    PatchTraining -->|best_s*.pt| TrainedModels
    PatchTraining -->|patch model| WholeImage
    WholeImage -->|best_whole_image_classifier.pt| TrainedModels
    RawData -->|Full image| Heatmap
    PatchTraining -->|patch model| Heatmap

    Orchestration --> DataIngestion
    Orchestration --> Preprocessing
    Orchestration --> PatchTraining
    Orchestration --> WholeImage
    Orchestration --> Heatmap

    SharedUtils -.-> DataIngestion
    SharedUtils -.-> Preprocessing
    SharedUtils -.-> PatchTraining
    SharedUtils -.-> WholeImage
    SharedUtils -.-> Heatmap

    click DataIngestion "https://github.com/sposso/deep_learning_to_improve_breast_cancer_pytorch/blob/main/data/split.py"
    click Preprocessing "https://github.com/sposso/deep_learning_to_improve_breast_cancer_pytorch/blob/main/Patches/tools.py"
    click PatchTraining "https://github.com/sposso/deep_learning_to_improve_breast_cancer_pytorch/blob/main/patch_classifier/train_function.py"
    click WholeImage "https://github.com/sposso/deep_learning_to_improve_breast_cancer_pytorch/blob/main/whole_image_classifier/main.py"
    click Heatmap "https://github.com/sposso/deep_learning_to_improve_breast_cancer_pytorch/blob/main/Heatmaps/main.py"
    click SharedUtils "https://github.com/sposso/deep_learning_to_improve_breast_cancer_pytorch/blob/main/utils/train_util.py"
    click Orchestration "https://github.com/sposso/deep_learning_to_improve_breast_cancer_pytorch/blob/main/main.py"

    classDef data fill:#add8e6,stroke:#333,stroke-width:1px
    classDef training fill:#98fb98,stroke:#333,stroke-width:1px
    classDef inference fill:#ffa500,stroke:#333,stroke-width:1px
    classDef util fill:#d3d3d3,stroke:#333,stroke-width:1px
    classDef orchestration fill:#c0c0c0,stroke:#333,stroke-width:1px
```
