How jayantuppal/intrusion-detection-system works
Project type and purpose
• This is a machine-learning based Intrusion Detection System (IDS) implemented as a Jupyter notebook.
• Its goal is to process the KDD Cup ’99 network traffic dataset, train three classifiers (Decision Tree, Gaussian Naive Bayes, XGBoost), combine them via a voting ensemble, and produce predictions on malicious vs. normal connections.
File structure
• Top-level files only:
• IDS.ipynb – the end-to-end notebook containing data-load, preprocessing, EDA, model training, evaluation, and visualization.
• requirements.txt – Python dependencies (scikit-learn, xgboost, pandas, matplotlib, etc.).
• README.md – project overview and usage instructions.
• No separate “src” or “scripts” directories; everything lives in the notebook.
Key insights from the README
• Four sequential steps: Load & preprocess data → EDA → Model training → Ensemble classification.
• Technologies: Python, Jupyter, pandas, scikit-learn, XGBoost, matplotlib/seaborn for plots.
• External dependency: KDD dataset file.
Main components
a. Data Source
• KDD ’99 dataset (local CSV/GZ file).
b. Data Ingestion & Preprocessing
• Integration, cleaning (remove outliers/duplicates), encoding symbolic → numeric, scaling or normalization.
c. Exploratory Data Analysis (EDA)
• Statistical summaries, visualizations (histograms, boxplots, correlation heatmap).
d. Model Training
• Three parallel classifier modules:
• Decision Tree Trainer
• Gaussian Naive Bayes Trainer
• XGBoost Trainer
• Each module outputs a trained model.
e. Ensemble Module
• Voting aggregator that takes predictions from the three models and outputs final class.
f. Evaluation & Results
• Metrics calculator (accuracy, precision, recall, F1, confusion matrix) and result plots.
Architectural patterns/design principles
• Pipeline pattern: data flows through discrete, ordered stages.
• Parallel training sub-pipeline: three independent model pipelines feeding into a single ensemble.
• Separation of concerns: distinct modules for preprocessing, EDA, training, and evaluation.
Relevant technologies/frameworks
• Python & Jupyter Notebook environment.
• pandas for data manipulation.
• scikit-learn for Decision Tree & Gaussian NB, metrics.
• xgboost library for the XGBoost classifier.
• matplotlib/seaborn for visualizations.