# h-ssiqueira/skinlesionai architecture

> Architecture diagram of the GitHub repository h-ssiqueira/skinlesionai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/h-ssiqueira/skinlesionai
- Repository: https://github.com/h-ssiqueira/skinlesionai

## Overview

##### Project type and purpose
- This is a research‐oriented machine learning pipeline for skin lesion classification (SkinLesionAI).
- It is organized as Jupyter/Colab notebooks for:
– Experiments (model development, GAN image generation, segmentation)
– Trainings (ConvNeXt, ResNetV2, Xception, Vision Transformer variants)
– Analysis (metrics extraction, graphs, statistical evaluation)
- Data is sourced from the HAM10000 dataset (Kaggle) and results are stored in files and a SQLite database.

##### File structure
– Top‐level directories:
- Notebooks/experiments – exploratory model development and augmentation
- Notebooks/trainings – systematic training runs, organized by architecture
- analysis/ – metrics extraction, database, visualizations
- docs/ – final PDF report
– Config files: requirements.txt specifies dependencies.
– Database: analysis/database.db and accompanying .sql indicate a metrics storage component.
– No service layers or web frontends – this is a batch‐style pipeline.

##### README for additional insights
- Objectives: compare models (CNNs and ViT), test techniques (data augmentation, segmentation, GANs, transfer learning)
- Outputs: models (≈22.4 GB), metrics (accuracy, loss, recall, specificity, F1, AUC, confusion matrices)

##### Main components
a. Data Sources & Storage
- HAM10000 dataset (Kaggle)
- Google Drive (notebooks & model artifacts)
- Local file system & SQLite database (metrics)
b. Notebook Workflows
- Experiment Notebooks (GAN generation, data augmentation, segmentation)
- Training Notebooks (organized by architecture subfolders)
- Analysis Notebooks (read metrics DB, produce graphs)
c. Compute Environments
- Google Colab / TPU/GPU runtimes
- Local Jupyter/Python environment
d. Outputs & Artifacts
- Trained model files
- Serialized weights & checkpoints
- SQLite DB with metrics
- Plots & final PDF report

##### Relationships and interactions
1. Data ingestion: download from Kaggle → store on Drive/local
2. Experiment phase: notebooks read raw data → generate augmented data (GAN output, segmentations) → save augmented images
3. Training phase: training notebooks read raw+augmented data → train models → write model checkpoints & log metrics to database
4. Analysis phase: analysis notebooks query SQLite DB → generate performance plots → export results to docs
5. Final report compiles graphs and metrics into PDF

##### Architectural patterns & design principles
- Modular pipeline: separate Experiment, Training, Analysis stages
- File‐based orchestration: Jupyter notebooks drive each stage
- Centralized metrics storage (SQLite) for reproducibility and easy querying
- Use of external compute and storage services (Colab, Drive) for scalability

##### Diagramming recommendations
– Rectangles for compute modules (notebook clusters)
– Clouds for external services (Kaggle, Google Drive, Colab)
– Raw data → Experiment → Augmented data → Training → Metrics DB → Analysis → Report
Layer 1: External Data & Services
Layer 2: Notebook Stages
Layer 3: Storage & Artifacts
Layer 4: Reporting

##### Detail without over-splitting
- Seven model families and GAN pipelines under the “Training Notebooks” group, rather than drawing each variant.
- Segmentation and augmentation as sub-components of the Experiment stage.
- Metrics DB is centrally placed to show bidirectional interaction with Training and Analysis.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Data & Services
    subgraph "External Services" 
        direction TB
        Kaggle((Kaggle Dataset)):::external
        Drive(("Google Drive Storage")):::external
        Colab((Google Colab/TPU/GPU)):::external
    end

    %% Ingestion Stage
    subgraph "Ingestion Stage" 
        direction TB
        Download["Download Dataset Notebook"]:::notebook
        Templates["Compute Environment Templates"]:::notebook
        RawData[(Raw Data Files)]:::storage
    end

    %% Experiment Stage
    subgraph "Experiment Stage"
        direction TB
        GANExp["GAN Experiments Notebooks"]:::notebook
        Segmentation["Segmentation Notebook"]:::notebook
        ArchExp["Architecture Exploration Notebook"]:::notebook
        MetricsExp["Experiment Metrics Notebook"]:::notebook
        AugData[(Augmented Data)]:::storage
    end

    %% Training Stage
    subgraph "Training Stage"
        direction TB
        subgraph "Standard Trainings"
            direction TB
            ConvNext["ConvNeXt Variants"]:::notebook
            ResNetV2["ResNetV2 Variants"]:::notebook
            Xception["Xception Variants"]:::notebook
            ViT["Vision Transformer Variants"]:::notebook
        end
        subgraph "GAN-Augmented Trainings"
            direction TB
            ConvNextGAN["ConvNeXt GAN Training"]:::notebook
            ResNetV2GAN["ResNetV2 GAN Training"]:::notebook
            XceptionGAN["Xception GAN Training"]:::notebook
        end
        Models[(Trained Model Checkpoints)]:::storage
        MetricsDB[(Metrics SQLite DB)]:::database
    end

    %% Analysis Stage
    subgraph "Analysis Stage"
        direction TB
        DBSQL["Database Schema (.sql)"]:::notebook
        DBFile["Metrics DB File"]:::database
        DBQuery["Database Query Notebook"]:::notebook
        Graphs["Visualization Notebooks"]:::notebook
        Stats["Statistical Analysis Notebooks"]:::notebook
    end

    %% Reporting Stage
    subgraph "Reporting Stage"
        direction TB
        Report[/"Final PDF Report"/]:::report
    end

    %% Data Flows
    Kaggle -->|download| Download
    Download -->|stores raw data| RawData
    Drive --> RawData
    Colab --> Download
    Colab --> Templates
    Templates --> Download

    RawData --> GANExp
    RawData --> Segmentation
    RawData --> ArchExp
    RawData --> MetricsExp
    GANExp --> AugData
    Segmentation --> AugData
    ArchExp --> AugData
    MetricsExp --> AugData

    AugData --> ConvNext
    AugData --> ResNetV2
    AugData --> Xception
    AugData --> ViT

    GANExp --> ConvNextGAN
    GANExp --> ResNetV2GAN
    GANExp --> XceptionGAN

    ConvNext --> Models
    ResNetV2 --> Models
    Xception --> Models
    ViT --> Models

    ConvNextGAN --> Models
    ResNetV2GAN --> Models
    XceptionGAN --> Models

    ConvNext --> MetricsDB
    ResNetV2 --> MetricsDB
    Xception --> MetricsDB
    ViT --> MetricsDB
    ConvNextGAN --> MetricsDB
    ResNetV2GAN --> MetricsDB
    XceptionGAN --> MetricsDB

    MetricsDB --> DBSQL
    MetricsDB --> DBFile
    DBFile --> DBQuery
    DBSQL --> DBQuery

    DBQuery --> Graphs
    Graphs --> Stats
    Stats --> Report

    %% Click Events
    click Download "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/experiments/downloadDataset.ipynb"
    click GANExp "https://github.com/h-ssiqueira/skinlesionai/tree/master/Notebooks/experiments/GANs"
    click Segmentation "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/experiments/Segmentation.ipynb"
    click ArchExp "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/experiments/architectures.ipynb"
    click MetricsExp "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/experiments/metrics.ipynb"
    click ConvNext "https://github.com/h-ssiqueira/skinlesionai/tree/master/Notebooks/trainings/convnext"
    click ResNetV2 "https://github.com/h-ssiqueira/skinlesionai/tree/master/Notebooks/trainings/resnetv2"
    click Xception "https://github.com/h-ssiqueira/skinlesionai/tree/master/Notebooks/trainings/xception"
    click ViT "https://github.com/h-ssiqueira/skinlesionai/tree/master/Notebooks/trainings/visionTransformers"
    click ConvNextGAN "https://github.com/h-ssiqueira/skinlesionai/tree/master/Notebooks/trainings/GANs/convnext"
    click ResNetV2GAN "https://github.com/h-ssiqueira/skinlesionai/tree/master/Notebooks/trainings/GANs/resnetv2"
    click XceptionGAN "https://github.com/h-ssiqueira/skinlesionai/tree/master/Notebooks/trainings/GANs/xception"
    click DBFile "https://github.com/h-ssiqueira/skinlesionai/blob/master/analysis/database.db"
    click DBSQL "https://github.com/h-ssiqueira/skinlesionai/blob/master/analysis/database.sql"
    click DBQuery "https://github.com/h-ssiqueira/skinlesionai/blob/master/analysis/database.ipynb"
    click Graphs "https://github.com/h-ssiqueira/skinlesionai/tree/master/analysis/graphs"
    click Stats "https://github.com/h-ssiqueira/skinlesionai/blob/master/analysis/statistics.ipynb"
    click Stats "https://github.com/h-ssiqueira/skinlesionai/blob/master/analysis/statisticsGANs.ipynb"
    click Stats "https://github.com/h-ssiqueira/skinlesionai/blob/master/analysis/investigationTopics.ipynb"
    click Report "https://github.com/h-ssiqueira/skinlesionai/blob/master/docs/Utilization_of_Different_Models_and_Convolutional_Neural_Network_Techniques_to_Skin_Lesion_Classification.pdf"
    click Templates "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/templateDownload.ipynb"
    click Templates "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/templateDrive.ipynb"
    click Templates "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/templateFormDownload.ipynb"
    click Templates "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/experiments/templateTPU.ipynb"
    click Templates "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/experiments/templateGPUwithTechniques.ipynb"
    click Download "https://github.com/h-ssiqueira/skinlesionai/blob/master/Notebooks/experiments/downloadDataset.ipynb"

    %% Styles
    classDef external fill:#BBDEFB,stroke:#64B5F6;
    classDef storage fill:#C8E6C9,stroke:#388E3C;
    classDef notebook fill:#FFE0B2,stroke:#FB8C00;
    classDef database fill:#A5D6A7,stroke:#2E7D32;
    classDef report fill:#D7CCC8,stroke:#5D4037;
```
