# gustavoeso/phishingurl-predictive-model architecture

> Architecture diagram of the GitHub repository gustavoeso/phishingurl-predictive-model, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gustavoeso/phishingurl-predictive-model
- Repository: https://github.com/gustavoeso/phishingurl-predictive-model

## Overview

##### Project Type and Purpose
- The project is a machine learning experiment to detect phishing URLs.
- Its main function is to extract features from URLs, train a predictive model (using techniques like Gradient Boosting Trees), and generate predictions with transparent probability outputs.
- The project makes use of key technologies like Python, PySpark, Databricks, and MLflow, which hint at a distributed data processing and experiment tracking workflow.

##### File Structure
- The project consists of a single Jupyter Notebook (Projeto_Modelo_Preditivo_Phishing_final.ipynb) alongside the README. This indicates that the main logic of the process—data ingestion, feature extraction, model training, and evaluation—is implemented in this notebook.
- The minimal file structure implies that there are no separate directories for frontend/backend or multiple modules; instead, the entire workflow is bundled into this notebook, common in experimental or academic projects.

##### README for Additional Insights
- The README clearly outlines the project’s purpose (phishing URL detection) and the ML tasks involved.
- It describes the technologies used (Python, PySpark, Databricks, MLflow) and provides an overview of the functionalities like feature extraction, training and validation, and detailed prediction probabilities.
- It references the Kaggle dataset “PhiUSIIL Phishing URL Dataset” as the source, which is a key external dependency.

a. Main Components to Include:
- Data Source: Indicate the external dataset (Kaggle – PhiUSIIL Phishing URL Dataset).
- Data Ingestion & Preprocessing: A module that handles reading the dataset and extracting relevant features (URL length, character continuity, TLD legitimacy probability, etc.).
- Feature Extraction Module: Transform raw URL data into features that the model can utilize.
- Model Training Pipeline: Show where the Gradient Boosting Trees (GBT) model is trained using PySpark.
- Experiment Tracking: Include MLflow to indicate the tracking and monitoring of different training runs.
- Execution Environment: Represent the usage of Databricks, highlighting that the notebook is run on a cloud-based distributed processing platform.
- Prediction/Inference Module: Illustrate the component that applies the trained model to generate predictions with associated probabilities.

b. Relationships and Interactions:

c. Highlighting Architectural Patterns/Design Principles:
- A pipeline or workflow design pattern where data flows sequentially through discrete processing stages.
- Modularity of tasks—each module (feature extraction, training, inference) can be represented as a separate block.
- Mention the use of distributed data processing with PySpark and a cloud-based execution environment at Databricks.
- That MLflow serves as a monitoring and logging layer, which is an essential design principle for reproducibility and experiment tracking.

d. Including Relevant Technologies & Frameworks:

##### Elements
- Each module/component must have a clear label.

## Diagram (Mermaid)

```mermaid
flowchart TD
    P["Project Documentation"]:::documentation
    DS["Dataset (Kaggle - Phishing URL Dataset)"]:::datasource

    subgraph "Notebook Workflow"
        B["Data Ingestion & Preprocessing (Notebook)"]:::processing
        C["Feature Extraction (Notebook)"]:::processing
        D["Model Training (GBT with PySpark) (Notebook)"]:::processing
        E["MLflow Experiment Tracking (Notebook)"]:::tracking
        F["Prediction/Inference (Notebook)"]:::processing
    end

    ENV["Execution Environment: Databricks Notebook"]:::environment

    P -->|"refers_to"| DS
    DS -->|"ingested_by"| B
    B -->|"features_extracted"| C
    C -->|"trains"| D
    D -->|"predicts_with"| F
    D -->|"logs_to"| E
    D -->|"runs_on"| ENV

    click P "https://github.com/gustavoeso/phishingurl-predictive-model/blob/main/README.md"
    click B "https://github.com/gustavoeso/phishingurl-predictive-model/blob/main/Projeto_Modelo_Preditivo_Phishing_final.ipynb"
    click C "https://github.com/gustavoeso/phishingurl-predictive-model/blob/main/Projeto_Modelo_Preditivo_Phishing_final.ipynb"
    click D "https://github.com/gustavoeso/phishingurl-predictive-model/blob/main/Projeto_Modelo_Preditivo_Phishing_final.ipynb"
    click E "https://github.com/gustavoeso/phishingurl-predictive-model/blob/main/Projeto_Modelo_Preditivo_Phishing_final.ipynb"
    click F "https://github.com/gustavoeso/phishingurl-predictive-model/blob/main/Projeto_Modelo_Preditivo_Phishing_final.ipynb"

    classDef datasource fill:#ffcccc,stroke:#cc0000,stroke-width:2px;
    classDef processing fill:#ccffcc,stroke:#008000,stroke-width:2px;
    classDef tracking fill:#ffffcc,stroke:#999900,stroke-width:2px;
    classDef documentation fill:#cce5ff,stroke:#0066cc,stroke-width:2px;
    classDef environment fill:#e0ccff,stroke:#6600cc,stroke-width:2px;
```
