# dilant123/lowcost-ml-cancer-padufes20 architecture

> Architecture diagram of the GitHub repository dilant123/lowcost-ml-cancer-padufes20, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dilant123/lowcost-ml-cancer-padufes20
- Repository: https://github.com/dilant123/lowcost-ml-cancer-padufes20

## Overview

##### Project type and purpose
- This is a research-grade, end-to-end multimodal machine-learning pipeline for classifying skin lesions (PAD-UFES-20).
- It ingests clinical metadata (CSV) and medical images, extracts features, applies preprocessing (SMOTE, PCA, feature selection), trains multiple models, builds a weighted ensemble, and outputs results and visualizations.

##### File structure
- Top-level artifacts:
- Notebooks (`.ipynb`) for exploration and post-article code
- `config.yaml` (configuration parameters)
- `requirements.txt` (dependencies)
- Directories: `graficos/`, `plots_enhanced/` (visual outputs)
- From the README, the canonical project layout under `PAD_UFES_Multimodal/` is:
- `data/` (raw inputs: `metadata.csv`, `imagenes/` folder)
- `resultados/` (combined dataset and exploration stats)
- `graficos/` (static plots)
- `plots_enhanced/` (interactive plots)
- `modelos/` (saved model files and metrics)

##### Key architecture insights from the README
- Single-process pipeline orchestrating:
a. Data ingestion (metadata + images)
b. Feature extraction (image processing + clinical features)
c. Class balancing (SMOTE)
d. Dimensionality reduction & feature selection (PCA, RFE, variance, mutual information)
e. Model training (XGBoost, Random Forest, Extra Trees, SVM)
f. Ensemble learning (custom weighted, stacking, voting)
g. Output generation (models, CSV/JSON results, PNG/HTML plots)
- Configuration and parameters centralized in `config.py` (paths, hyperparameters, test/val splits).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    subgraph "Data Sources"
        direction TB
        MetadataCSV["Metadata CSV (data/metadata.csv)"]:::dataStore
        ImageRepo["Image Repository (data/imagenes/)"]:::dataStore
    end

    %% Environment and Orchestration
    subgraph "Environment & Orchestration"
        direction TB
        EnvDef["requirements.txt"]:::env
        ConfigStore["config.yaml"]:::config
        Notebook["ULTIMATECODEFORPAD_UFES.ipynb"]:::process
        Alt1["ULTIMATECODEFORPAD_UFES - POSTARTICLE.ipynb"]:::process
        Alt2["ULTIMATECODEFORPAD_UFES copy.ipynb"]:::process
        Alt3["ULTIMATECODEFORPAD_UFES copy 2.ipynb"]:::process
        Alt4["ULTIMATECODEFORPAD_UFES.zip"]:::process
    end

    %% Preprocessing
    subgraph "Preprocessing Module" 
        direction TB
        ClinClean["Clinical Data Cleaning"]:::process
        ImgFeat["Image Feature Extraction"]:::process
        SMOTE["SMOTE Balancing"]:::process
        DimRed["PCA & Feature Selection"]:::process
    end

    %% Feature Store
    FeatureStore["Multimodal Dataset (multimodal_dataset.csv)"]:::dataStore

    %% Modeling Module
    subgraph "Model Training"
        direction TB
        XGB["XGBoost"]:::model
        RF["Random Forest"]:::model
        SVM["SVM"]:::model
        ET["Extra Trees"]:::model
    end

    %% Ensemble Module
    Ensemble["Dynamic Weighted Ensemble"]:::model

    %% Output Module
    subgraph "Outputs"
        direction TB
        ModelArt["Model Artifacts (.pkl)"]:::dataStore
        Results["Results CSV/JSON"]:::dataStore
        StaticVis["Static Visualizations (graficos/)"]:::dataStore
        InteractiveVis["Interactive Visualizations (plots_enhanced/)"]:::dataStore
        HTMLVis["Interactive Class Distribution (HTML)"]:::dataStore
    end

    %% Data Flow
    MetadataCSV -->|ingest| ClinClean
    ImageRepo -->|ingest| ImgFeat
    ClinClean -->|features| FeatureStore
    ImgFeat -->|features| FeatureStore
    FeatureStore -->|balanced| SMOTE
    SMOTE -->|reduced| DimRed
    DimRed -->|dataset| FeatureStore
    FeatureStore -->|training data| XGB
    FeatureStore -->|training data| RF
    FeatureStore -->|training data| SVM
    FeatureStore -->|training data| ET
    XGB -->|models| Ensemble
    RF -->|models| Ensemble
    SVM -->|models| Ensemble
    ET -->|models| Ensemble
    Ensemble -->|final| ModelArt
    Ensemble -->|metrics| Results
    Ensemble -->|plots| StaticVis
    Ensemble -->|plots| InteractiveVis
    Ensemble -->|html| HTMLVis

    %% Config Injection
    ConfigStore -.-> ClinClean
    ConfigStore -.-> ImgFeat
    ConfigStore -.-> SMOTE
    ConfigStore -.-> DimRed
    ConfigStore -.-> XGB
    ConfigStore -.-> RF
    ConfigStore -.-> SVM
    ConfigStore -.-> ET
    ConfigStore -.-> Ensemble

    %% Notebook orchestration
    Notebook --> ClinClean
    Notebook --> ImgFeat
    Notebook --> SMOTE
    Notebook --> DimRed
    Notebook --> XGB
    Notebook --> RF
    Notebook --> SVM
    Notebook --> ET
    Notebook --> Ensemble

    %% Environment container
    EnvDef -.-> Notebook

    %% Click Events
    click ConfigStore "https://github.com/dilant123/lowcost-ml-cancer-padufes20/blob/main/config.yaml"
    click EnvDef "https://github.com/dilant123/lowcost-ml-cancer-padufes20/blob/main/requirements.txt"
    click Notebook "https://github.com/dilant123/lowcost-ml-cancer-padufes20/blob/main/ULTIMATECODEFORPAD_UFES.ipynb"
    click Alt1 "https://github.com/dilant123/lowcost-ml-cancer-padufes20/blob/main/ULTIMATECODEFORPAD_UFES - POSTARTICLE.ipynb"
    click Alt2 "https://github.com/dilant123/lowcost-ml-cancer-padufes20/blob/main/ULTIMATECODEFORPAD_UFES copy.ipynb"
    click Alt3 "https://github.com/dilant123/lowcost-ml-cancer-padufes20/blob/main/ULTIMATECODEFORPAD_UFES copy 2.ipynb"
    click Alt4 "https://github.com/dilant123/lowcost-ml-cancer-padufes20/blob/main/ULTIMATECODEFORPAD_UFES.zip"
    click StaticVis "https://github.com/dilant123/lowcost-ml-cancer-padufes20/tree/main/graficos/"
    click InteractiveVis "https://github.com/dilant123/lowcost-ml-cancer-padufes20/tree/main/plots_enhanced/"
    click HTMLVis "https://github.com/dilant123/lowcost-ml-cancer-padufes20/blob/main/graficos/interactive_class_distribution.html"

    %% Styles
    classDef dataStore fill:#3b82f6,stroke:#1e40af,color:#ffffff
    classDef process fill:#10b981,stroke:#065f46,color:#ffffff
    classDef model fill:#f97316,stroke:#c2410c,color:#ffffff
    classDef config fill:#9ca3af,stroke:#6b7280,color:#000000
    classDef env fill:#d1d5db,stroke:#6b7280,color:#000000
```
