# lin-lcx/hgcn architecture

> Architecture diagram of the GitHub repository lin-lcx/hgcn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lin-lcx/hgcn
- Repository: https://github.com/lin-lcx/hgcn

## Overview

##### Project Type and Purpose
- This is a research‐oriented machine learning pipeline for cancer survival prediction.
- It integrates a Hybrid Graph Convolutional Network (HGCN) with an online masked autoencoder to handle multimodal patient data (pathological images, clinical records, genomic profiles).

##### High-Level Component Breakdown
A. Data Acquisition & Storage
- External Data Sources: cBioPortal (genomics), GDC (slides & clinical).
- Local Data Folder:
- data_split/ (precomputed train/test splits)
- Supplementary files (e.g., esca_split.pkl, etc.)
B. Data Preprocessing
- cut_and_pretrain/
- cut_and_pretrain.py: patches extraction & autoencoder pretraining
- utils/filters.py, utils/general.py: image filtering, augmentation, helper functions
- gendata.ipynb
- Notebook for merging modalities into unified graph input format
C. Model Definition & Utilities
- HGCN_code/
- mae_model.py: masked autoencoder and HGCN architecture
- mae_utils.py: masking strategy, reconstruction loss functions
- util.py: graph construction routines, feature normalization
D. Training & Evaluation
- train.py (entry point)
- Loads preprocessed graphs, applies data_split if enabled
- Instantiates model, optimizer, scheduler
- Runs training loop, validation, test inference
- Saves trained weights and prediction outputs

##### Relationships and Data Flow
- Raw data (images, genomics, clinical) → cut_and_pretrain → patch embeddings
- Patch embeddings + clinical + genomic features → gendata.ipynb → multimodal graph files
- Graph files + data_split → train.py → HGCN_code modules → model training
- Trained model ↔ online masked autoencoder paradigm for modality imputation at inference
- Final output: survival risk predictions written to disk

##### Architectural Patterns & Key Design Principles
- Modular Pipeline: clear separation of Data Acquisition → Preprocessing → Model → Training
- Graph-based Learning: uses PyTorch Geometric to build modality-specific graphs and hypergraphs
- Masked Autoencoding: integrated into the model for robust missing-modality handling
- Configurability: train.py parameters, if_fit_split flag, easily switch between full/missing modalities

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External / Data Storage
    subgraph "Data Acquisition & Storage"
        direction TB
        CB["cBioPortal (Genomics)"]:::external
        GD["GDC (Slides & Clinical)"]:::external
        DS["data_split/"]:::data
        SF["Supplementary.md"]:::data
        DS_ES["esca_split.pkl"]:::data
        DS_KI["kirc_split.pkl"]:::data
        DS_LI["lihc_split.pkl"]:::data
        DS_LU["luad_split.pkl"]:::data
        DS_LS["lusc_split.pkl"]:::data
        DS_UC["ucec_split.pkl"]:::data
    end

    %% Preprocessing Subsystem
    subgraph "Preprocessing Subsystem"
        direction TB
        CP["cut_and_pretrain.py"]:::preprocess
        F1["filters.py"]:::preprocess
        F2["general.py"]:::preprocess
        GN["gendata.ipynb"]:::preprocess
    end

    %% Graph Construction Engine
    subgraph "Graph Construction Engine"
        direction TB
        GC["util.py"]:::model
    end

    %% Model Core
    subgraph "Model Core"
        direction TB
        MM["mae_model.py"]:::model
        MU["mae_utils.py"]:::model
    end

    %% Training & Evaluation Loop
    subgraph "Training & Evaluation"
        direction TB
        TR["train.py"]:::training
    end

    %% Data Flows
    CB -->|"raw genomics"| CP
    GD -->|"raw images & clinical"| CP
    CP -->|"patch embeddings"| GN
    F1 --> CP
    F2 --> CP
    GN -->|"multimodal graph .pt"| GC
    DS --> TR
    DS_ES --> TR
    DS_KI --> TR
    DS_LI --> TR
    DS_LU --> TR
    DS_LS --> TR
    DS_UC --> TR
    GC -->|"graph files"| TR
    TR -->|"instantiates"| MM
    TR -->|"uses masking & loss"| MU
    MM <-->|"online MAE feedback"| MU
    TR -->|"saves predictions"| SF

    %% Click Events
    click DS "https://github.com/lin-lcx/hgcn/tree/main/data_split/"
    click DS_ES "https://github.com/lin-lcx/hgcn/blob/main/data_split/esca_split.pkl"
    click DS_KI "https://github.com/lin-lcx/hgcn/blob/main/data_split/kirc_split.pkl"
    click DS_LI "https://github.com/lin-lcx/hgcn/blob/main/data_split/lihc_split.pkl"
    click DS_LU "https://github.com/lin-lcx/hgcn/blob/main/data_split/luad_split.pkl"
    click DS_LS "https://github.com/lin-lcx/hgcn/blob/main/data_split/lusc_split.pkl"
    click DS_UC "https://github.com/lin-lcx/hgcn/blob/main/data_split/ucec_split.pkl"
    click SF "https://github.com/lin-lcx/hgcn/blob/main/Supplementary.md"
    click CP "https://github.com/lin-lcx/hgcn/blob/main/cut_and_pretrain/cut_and_pretrain.py"
    click F1 "https://github.com/lin-lcx/hgcn/blob/main/cut_and_pretrain/utils/filters.py"
    click F2 "https://github.com/lin-lcx/hgcn/blob/main/cut_and_pretrain/utils/general.py"
    click GN "https://github.com/lin-lcx/hgcn/blob/main/gendata.ipynb"
    click GC "https://github.com/lin-lcx/hgcn/blob/main/HGCN_code/util.py"
    click MM "https://github.com/lin-lcx/hgcn/blob/main/HGCN_code/mae_model.py"
    click MU "https://github.com/lin-lcx/hgcn/blob/main/HGCN_code/mae_utils.py"
    click TR "https://github.com/lin-lcx/hgcn/blob/main/HGCN_code/train.py"

    %% Styles
    classDef external fill:#AED6F1,stroke:#1F618D,color:#1B2631
    classDef data fill:#D6EAF8,stroke:#1B4F72,color:#1B2631
    classDef preprocess fill:#82E0AA,stroke:#196F3D,color:#1B2631
    classDef model fill:#F5B041,stroke:#B9770E,color:#1B2631
    classDef training fill:#D2B4DE,stroke:#6C3483,color:#1B2631
```
