# angione-lab/nsclc-survival-prediction-models architecture

> Architecture diagram of the GitHub repository angione-lab/nsclc-survival-prediction-models, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/angione-lab/nsclc-survival-prediction-models
- Repository: https://github.com/angione-lab/nsclc-survival-prediction-models

## Overview

##### Project Type and Purpose
- This is a research/experimental deep-learning project focused on survival prediction for non-small cell lung cancer (NSCLC). It integrates radiological imaging, gene expression, and clinical data.
- The project implements pathway-guided models (H-VAE-Cox and XAT-VAE-Cox) to produce prognostic indexes, and it includes additional steps like SHAP for model interpretability as well as biological pathway analysis using KEGG and Reactome.
- The code is modular, supporting preprocessing (e.g., U-Net segmentation), model training/inference, and post-processing (visualization and statistical tests performed in R).

##### Analyzing the File Structure
- Top-Level Directories:
- Data: Contains raw and preprocessed data (CT scans, gene expression, clinical datasets, pathway masks, etc.).
- R: Includes R scripts for generating final plots (bar graphs, Kaplan-Meier curves, etc.) used for biological interpretation.
- Results: Hosts output files from different model runs and analyses.
- baselineModels: Contains reference implementations for baseline survival prediction models (DeepCoxMixture, pycox).
- models: Contains the deep learning model implementations (e.g., HVAECoxmodel, XVAECoxmodel, UnetModel, etc.).
- run: Contains scripts for end-to-end execution including data preprocessing (e.g., U-Net segmentation), feature generation for genes and images, survival prediction models, and SHAP interpretation.
- utils: Includes helper functions for data loading, image processing, survival analysis, and plotting.
- Config Files and Environments:
- Requirements.txt covers environment dependencies.
- .spyproject files indicate this has been developed in Spyder (Python IDE), and although they are IDE configuration files, they hint at the development practices.
- Architectural Patterns:
- The project is modularized where distinct directories represent different stages of the workflow (data ingestion, preprocessing, model training/inference, results, and plotting).
- There’s a separation between research baseline implementations (in baselineModels) and proposed model implementations (in models), which indicates layered experimentation.

a. Main Components:
- Data Layer:
- Data sources (raw CT scan images, gene expression datasets, clinical data, pathway annotations).
- External data feeds (links provided in the README and instructions on how to populate the Data folder).
- Preprocessing Layer:
- Image processing using U-Net (run/U-net-ImageSegmentation.py).
- Data cleaning, normalization, and feature selection (performed within the preprocessing scripts in run and utils).
- Feature Extraction / Dimensionality Reduction:
- Gene latent feature extraction (run/GenerateGeneLatentFeatures.py).
- Image latent feature extraction (run/GenerateImageLatentFeatures.py).
- Model Training and Inference:
- Deep learning survival prediction models including H-VAE-Cox and XAT-VAE-Cox (located in models and executed via the run scripts).
- Baseline models (found under baselineModels) for comparative analyses.
- Model Interpretation:
- SHAP analysis for model interpretation (run/H_VAE_Shap_Interpretation.py).
- Additional visualization and interpretation using Grad-CAM.
- Evaluation and Visualization:
- Survival analysis and calculation of the C-index, Kaplan-Meier curves (results and plotting scripts using R).
- Final plots and statistical results stored in the Results folder.
- Supporting Utilities:
- Utility functions under utils for loading data, handling images, and computing survival metrics.

b. Relationships and Interactions:
- Interactions between components, for example:
- The output from the U-Net image segmentation feeds into the latent feature extraction.
- Latent features from both gene expression and imaging data are then combined with clinical data to perform survival prediction.
- Model outputs are then used by the interpretation modules (SHAP) and further visualized using R scripts.
- For baseline models, indicate a parallel flow for comparison purposes alongside the main H-VAE-Cox and XAT-VAE-Cox approaches.

c. Architectural Patterns / Design Principles:
- Modularity: Separation into discrete, loosely coupled modules (Data, Preprocessing, Models, Post-Processing) allows for individual upgrades and testing.
- Pipeline/Workflow design: Data flows successively through a series of transformation stages (preprocessing → feature generation → prediction → interpretation).
- Reusability: Utility modules and baseline models indicate an emphasis on reusable components.
- Integration of heterogeneous data types: The architecture accommodates multiple data modalities (images, gene expression, and clinical data) unified by the deep learning models.

d. Relevant Technologies and Frameworks:
- Python-based deep learning frameworks (likely using TensorFlow/Keras or PyTorch) indicated by .h5 model files and script naming conventions.
- R for specialized statistical analysis and plots.
- External libraries for survival analysis, SHAP explanations, and image segmentation.
- U-Net architecture for image segmentation and autoencoder-based models for feature extraction.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        DS["Data Sources (Raw CT, Gene Expression, Clinical, Annotations)"]:::data
    end

    %% Preprocessing Layer
    subgraph "Preprocessing Layer"
        US["U-Net Image Segmentation"]:::preprocess
        TCGA["TCGA Dataset Cleaning"]:::preprocess
    end

    %% Feature Extraction Layer
    subgraph "Feature Extraction Layer"
        GF["Gene Latent Feature Extraction"]:::feature
        IF["Image Latent Feature Extraction"]:::feature
    end

    %% Model Training Layer
    subgraph "Model Training Layer"
        HV["H-VAE-Cox Model"]:::model
        XV["X-VAE-Cox Model"]:::model
        BM["Baseline Models"]:::model
    end

    %% Interpretation Layer
    subgraph "Interpretation Layer"
        SHAP["SHAP Analysis"]:::interpretation
    end

    %% Evaluation and Visualization Layer
    subgraph "Evaluation & Visualization Layer"
        RV["R Visualization & Final Plots"]:::visualization
    end

    %% Supporting Utilities
    UTL["Utilities"]:::utility

    %% Flow connections
    DS --> US
    DS --> TCGA
    US --> IF
    TCGA --> GF
    GF --> HV
    GF --> XV
    IF --> HV
    IF --> XV
    DS --> BM
    HV --> SHAP
    XV --> SHAP
    BM --> RV
    SHAP --> RV

    %% Supporting Utilities connections (dashed)
    UTL --- DS
    UTL --- US
    UTL --- TCGA
    UTL --- GF
    UTL --- IF
    UTL --- HV
    UTL --- XV
    UTL --- BM
    UTL --- SHAP
    UTL --- RV

    %% Click Events
    click DS "https://github.com/angione-lab/nsclc-survival-prediction-models/tree/main/Data/"
    click US "https://github.com/angione-lab/nsclc-survival-prediction-models/blob/main/run/U-net-ImageSegmentation.py"
    click TCGA "https://github.com/angione-lab/nsclc-survival-prediction-models/blob/main/run/preprocess_TCGA_dataset.py"
    click GF "https://github.com/angione-lab/nsclc-survival-prediction-models/blob/main/run/GenerateGeneLatentFeatures.py"
    click IF "https://github.com/angione-lab/nsclc-survival-prediction-models/blob/main/run/GenerateImageLatentFeatures.py"
    click HV "https://github.com/angione-lab/nsclc-survival-prediction-models/blob/main/models/HVAECoxmodel.py"
    click HV "https://github.com/angione-lab/nsclc-survival-prediction-models/blob/main/run/RunHvaeCoxModel.py"
    click XV "https://github.com/angione-lab/nsclc-survival-prediction-models/blob/main/models/XVAECoxmodel.py"
    click XV "https://github.com/angione-lab/nsclc-survival-prediction-models/blob/main/run/RunXVaeCoxModel.py"
    click BM "https://github.com/angione-lab/nsclc-survival-prediction-models/tree/main/baselineModels/"
    click SHAP "https://github.com/angione-lab/nsclc-survival-prediction-models/blob/main/run/H_VAE_Shap_Interpretation.py"
    click RV "https://github.com/angione-lab/nsclc-survival-prediction-models/tree/main/R/"
    click RV "https://github.com/angione-lab/nsclc-survival-prediction-models/tree/main/Results/"
    click UTL "https://github.com/angione-lab/nsclc-survival-prediction-models/tree/main/utils/"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef preprocess fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef feature fill:#FFFFE0,stroke:#333,stroke-width:2px;
    classDef model fill:#F08080,stroke:#333,stroke-width:2px;
    classDef interpretation fill:#FFB6C1,stroke:#333,stroke-width:2px;
    classDef visualization fill:#D3D3D3,stroke:#333,stroke-width:2px;
    classDef utility fill:#E6E6FA,stroke:#333,stroke-width:2px,stroke-dasharray: 5 5;
```
