# julian-borges/breast-cancer-ai-misclassification- architecture

> Architecture diagram of the GitHub repository julian-borges/breast-cancer-ai-misclassification-, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/julian-borges/breast-cancer-ai-misclassification-
- Repository: https://github.com/julian-borges/breast-cancer-ai-misclassification-

## Overview

##### Project type and purpose
- This is a data‐science/ML pipeline for auditing shortcut learning in breast cancer genomic subtyping. It is **not** a web service or full‐stack app but a **research/analysis** workflow that ingests raw clinical/genomic data, preprocesses it, trains & explains ML models, and outputs visualizations and reports.

##### File structure
- Top‐level files and their roles:
- `clinical_raw.dta` – Original raw clinical dataset in Stata format
- `AI-BIAS cleaned_TCGA_PAM50_model_dataset.csv` – Cleaned and combined dataset (gene expression + clinical features) for modeling
- `PAM50_Capstone_Consolidated_Pipeline.do` – Stata script for data cleaning/preprocessing steps (pseudo‐SHAP calculations, merging)
- `PAM50_SHAP_Colab_Notebook.ipynb` – Python notebook that trains XGBoost, runs true SHAP, and generates plots
- `requirements.txt` – Python library dependencies (e.g., xgboost, shap, pandas)
- `README.md` – Project overview, objectives, and instructions

##### Key insights from the README
- Pipeline stages:
a. Data cleaning (Stata)
b. Model training & validation (XGBoost in Python)
c. Explainability & auditing (SHAP)
d. Misclassification analysis by subgroup
- Execution environment: Google Colab for the notebook

##### Main components & interactions
a. **Data Ingestion & Preprocessing**
– Input: `clinical_raw.dta`
– Tool: Stata (`.do` script)
– Output: cleaned CSV (`AI-BIAS cleaned_TCGA_PAM50_model_dataset.csv`)
b. **Model Training & Validation**
– Input: cleaned CSV
– Tool: Python environment (Google Colab)
– Framework: XGBoost for classification (PAM50 subtypes)
– Output: trained model artifacts (in‐memory in notebook)
c. **Explainability**
– Input: trained model + dataset
– Library: SHAP (python package)
– Output: feature‐importance plots, global & local explanations
d. **Misclassification Analysis**
– Input: model predictions + actual subtypes + clinical covariates
– Tool: Python (pandas/matplotlib/seaborn)
– Output: analyses by age, hormone status, batch effects
e. **Reporting & Visualization**
– Notebook generates figures and tables for the capstone report

##### Architectural patterns & design principles
- **Sequential pipeline (ETL → ML → explain)**
- **Separation of concerns**: Stata handles raw data cleaning, Python handles modeling & explainability
- **Reproducibility**: version‐controlled scripts (`.do`, notebook) and `requirements.txt`

- Components:
- Rectangles for data stores (`.dta`, `.csv`)
- Directional data flow (e.g., raw→Stata→clean CSV→Python→model→explain→report)
- Blue for data artifacts
- Green for compute environments/tools
- Orange for outputs/visualizations
- Labels:
- File names (`clinical_raw.dta`, `AI-BIAS cleaned_TCGA_PAM50_model_dataset.csv`)
- Scripts (`PAM50_Capstone_Consolidated_Pipeline.do`, `PAM50_SHAP_Colab_Notebook.ipynb`)
- Libraries (`XGBoost`, `SHAP`)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Ingestion & Preprocessing"
        RAW["clinical_raw.dta"]:::data
        STATA["PAM50_Capstone_Consolidated_Pipeline.do"]:::tool
        CLEAN["AI-BIAS cleaned_TCGA_PAM50_model_dataset.csv"]:::data
        RAW -->|"Input to Stata script"| STATA
        STATA -->|"Outputs cleaned CSV"| CLEAN
    end

    subgraph "Python/Colab Environment"
        NB["PAM50_SHAP_Colab_Notebook.ipynb"]:::tool
        TRAIN["XGBoost Training"]:::tool
        MODEL["Trained Model"]:::model
        SHAP["SHAP Explainability"]:::tool
        REPORT["SHAP Reports"]:::output
        ANALYSIS["Misclassification Analysis"]:::tool
        PLOTS["Analysis Plots"]:::output
        CLEAN --> NB
        NB -->|"Trains model"| TRAIN
        TRAIN -->|"Generates"| MODEL
        MODEL -->|"Explained by"| SHAP
        SHAP -->|"Produces"| REPORT
        MODEL -->|"Analyzed with"| ANALYSIS
        ANALYSIS -->|"Produces"| PLOTS
    end

    README["README.md"]:::doc

    classDef data fill:#AED6F1,stroke:#1B4F72;
    classDef tool fill:#ABEBC6,stroke:#1D8348;
    classDef model fill:#F9E79F,stroke:#B7950B,shape:hexagon;
    classDef output fill:#F5CBA7,stroke:#CD6155;
    classDef doc fill:#D7BDE2,stroke:#6C3483,stroke-dasharray:2 2;

    click RAW "https://github.com/julian-borges/breast-cancer-ai-misclassification-/blob/main/clinical_raw.dta"
    click STATA "https://github.com/julian-borges/breast-cancer-ai-misclassification-/blob/main/PAM50_Capstone_Consolidated_Pipeline.do"
    click CLEAN "https://github.com/julian-borges/breast-cancer-ai-misclassification-/blob/main/AI-BIAS cleaned_TCGA_PAM50_model_dataset.csv"
    click NB "https://github.com/julian-borges/breast-cancer-ai-misclassification-/blob/main/PAM50_SHAP_Colab_Notebook.ipynb"
    click README "https://github.com/julian-borges/breast-cancer-ai-misclassification-/blob/main/README.md"
```
