How rimtouny/phishing-attack-detection-using-machine-learning works
Project type and purpose
• This is a data-science/ML pipeline implemented in a Jupyter Notebook. Its primary goal is to detect phishing URLs via binary classification, using extensive feature extraction, EDA, model training (multiple classifiers), ensemble stacking/voting, and evaluation.
File structure
• Top‐level files:
• Phishing-Attack-Detection.ipynb – contains all data ingestion, feature engineering, modeling, evaluation, and visualization.
• README.md – describes datasets, features, modeling steps, and results.
• LICENSE – project license.
• No separate “src” or “notebooks” directory: the notebook drives the entire pipeline.
README for additional insights
• Data sources: three external URL datasets (PhishStorm, ISCX-URL2016, Kaggle).
• Pipeline steps: concatenate raw datasets → feature extraction → EDA → data cleaning/SMOTE → feature selection → model training (multiple algorithms via LazyClassifier) → bias-variance analysis → ensemble stacking/voting → champion model evaluation.
• Key libraries: pandas, scikit-learn, matplotlib, imbalanced-learn (SMOTE), LazyClassifier.
Main components
a. External Data Sources
• PhishStorm-URL dataset
• ISCX-URL2016 dataset
• Malicious URL Kaggle dataset
b. Data Ingestion & Preprocessing
• Data concatenation
• Initial cleaning (nulls, duplicates)
c. Feature Extraction Module
• Length/count features
• Boolean features
• Calculation-based features (entropy, ratios, malicious probability)
d. Exploratory Data Analysis (EDA)
• Statistics & visualizations (boxplots, histograms, correlations)
e. Data Cleaning & Balancing
• Null/duplicate handling
• Train/test split (80/20)
• SMOTE oversampling on training set
f. Feature Selection
• 90% repetition threshold filter
g. Modeling & Evaluation
• LazyClassifier wrapper training multiple base models (LR, SVM, DT, RF, XGBoost, etc.)
• Bias-variance decomposition analysis
• Metrics calculation (confusion matrix, F1, classification report)
h. Ensemble Module
• Select top two TPR models & top two TNR models
• Stacking classifiers for each pair
• Soft voting between the two ensembles
i. Champion Model Evaluation
• Final performance metrics visualization
Relationships & interactions
• Data flows from external sources → ingestion → feature extraction → EDA & cleaning → feature selection → model training/evaluation → ensembling → champion model.
Architectural patterns & design principles
• Pipeline pattern: sequential transformation stages.
• Modular separation: ingestion, feature extraction, modeling, ensembling.
• Ensemble design: stacking + voting classifiers.
Relevant technologies & frameworks
• Jupyter Notebook (or Colab) as orchestration environment.
• pandas for data manipulation.
• scikit-learn for feature engineering, modeling, metrics.
• imbalanced-learn (SMOTE) for oversampling.
• matplotlib/seaborn for visualization.
• LazyClassifier for rapid benchmarking of classifiers.
Elements
• Processing modules (rectangles)
• Models (rounded rectangles)
• Feedback loops (e.g., EDA → data cleaning).