# abdelrhman-abu-alhijaa/ids-ml-project architecture

> Architecture diagram of the GitHub repository abdelrhman-abu-alhijaa/ids-ml-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abdelrhman-abu-alhijaa/ids-ml-project
- Repository: https://github.com/abdelrhman-abu-alhijaa/ids-ml-project

## Overview

1. **Project purpose**
   - This repository implements a **machine-learning intrusion detection system (IDS)** for the **CICIDS2017** network traffic dataset.
   - The architecture is split into two clear phases: an **offline ML pipeline** for data preparation/training, and an **online Streamlit dashboard** for interactive detection.

2. **Primary runtime boundary**
   - The only user-facing runtime component is **`app.py`**, which serves as a **Streamlit web app**.
   - Everything else in the repo is part of the **analysis/training workflow**: data cleaning, EDA, and model training notebooks.

3. **Data source and ingestion**
   - The system starts from **raw CSV network-flow records** from **CICIDS2017**.
   - These records represent the external input boundary of the system and are the basis for both training and live-like classification in the dashboard.

4. **Data preparation subsystem**
   - **`01_data_cleaing.ipynb`** is the preprocessing stage.
   - Its role is to transform raw traffic data into a cleaned, model-ready dataset, likely handling missing values, feature normalization/selection, and label preparation.
   - This stage produces **processed data** and may also generate intermediate artifacts such as cleaned CSVs and charts.

5. **Exploratory analysis subsystem**
   - **`02_eda.ipynb`** is the analysis/understanding layer.
   - It is used to inspect class distribution, feature behavior, and traffic/attack patterns before training.
   - This notebook is not part of production inference; it supports dataset understanding and model design decisions.

6. **Model training and evaluation subsystem**
   - **`03_model_training.ipynb`** is the training pipeline.
   - It compares multiple classifiers: **Random Forest, XGBoost, and Decision Tree**.
   - The notebook evaluates models using metrics like **Accuracy, F1, and ROC-AUC**, and the README indicates the selected/validated model reaches perfect scores on the reported dataset.

7. **Model artifact boundary**
   - Trained model files are intended to be stored under a **`models/`** directory, even though the tree shown does not list the folder explicitly.
   - This is the bridge between offline training and online inference: the dashboard consumes the saved model artifact rather than retraining on startup.

8. **Interactive detection subsystem**
   - The Streamlit dashboard in **`app.py`** is the operational interface for detection.
   - It accepts traffic-like input, runs the trained classifier, and returns:
     - predicted attack/benign class,
     - confidence score,
     - and likely a user-visible alert state.

9. **Alert/export subsystem**
   - The app includes an export path for **detection alerts as CSV**.
   - This means the dashboard is not only a visualizer; it also produces persisted outputs for downstream review or incident response.

10. **Model capability and classification scope**
   - The system is positioned as a **multi-class IDS**, detecting **13 attack types** in addition to benign traffic.
   - That makes the core prediction boundary a **multi-class classification service**, not binary anomaly detection.

11. **Technology stack boundaries**
   - **Python** is the implementation language across notebooks and app.
   - Core libraries define the architecture:
     - **pandas / numpy** for data handling,
     - **matplotlib / seaborn** for analysis and chart generation,
     - **scikit-learn / xgboost** for model training and evaluation,
     - **Streamlit** for the front-end/runtime dashboard,
     - **joblib** for model persistence/loading.

12. **Repository structure and artifact flow**
   - The intended flow is:
     - raw dataset → cleaning notebook → processed data/charts → EDA notebook → training notebook → saved model(s) → Streamlit app.
   - The **`report.md`** file appears to capture the written project report, while the notebooks contain the executable research/training workflow.

13. **What is not in this repo**
   - There is no evidence of a separate backend API, message queue, database, or streaming ingestion pipeline.
   - The architecture is therefore a **single-process dashboard plus offline notebooks**, rather than a distributed production IDS stack.

14. **How to diagram it**
   - Draw four main blocks:
     - **External dataset input (CICIDS2017)**
     - **Offline ML pipeline** (cleaning, EDA, training, model persistence)
     - **Saved model artifact store**
     - **Streamlit dashboard** (interactive inference, confidence display, CSV export)
   - The key dependency arrow is from **trained model artifact → Streamlit app**, and from **raw data → notebooks → trained model**.

Main entry points: [`01_data_cleaing.ipynb`](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/01_data_cleaing.ipynb) (Data Cleaning), [`03_model_training.ipynb`](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/03_model_training.ipynb) (Model Training), [`report.md`](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/report.md) (Report), [`app.py`](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py) (Dashboard)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_external["External Data"]
  node_cicids2017[("CICIDS2017<br/>raw dataset")]
end

subgraph group_offline["Offline ML Pipeline"]
  node_cleaning["Data Cleaning<br/>notebook preprocessing"]
  node_feature_prep["Feature Prep<br/>data transform"]
  node_eda["EDA<br/>notebook analysis<br/>[02_eda.ipynb]"]
  node_class_study["Class Study<br/>analysis output<br/>[02_eda.ipynb]"]
  node_training["Model Training<br/>notebook training"]
  node_model_compare["Model Compare<br/>training comparison"]
  node_metrics["Metrics<br/>evaluation"]
  node_model_store[("Saved Model<br/>artifact store")]
end

subgraph group_runtime["Streamlit Runtime"]
  node_app["Dashboard<br/>streamlit app<br/>[app.py]"]
  node_input_form["Input Form<br/>ui input<br/>[app.py]"]
  node_predictor["Inference<br/>[app.py]"]
  node_confidence["Confidence<br/>prediction output<br/>[app.py]"]
  node_export_csv["CSV Export<br/>alert export<br/>[app.py]"]
end

subgraph group_docs["Documentation"]
  node_report["Report<br/>[report.md]"]
  node_readme["README<br/>overview doc<br/>[README.md]"]
end

node_cicids2017 -->|"raw flows"| node_cleaning
node_cleaning -->|"preprocess"| node_feature_prep
node_feature_prep -->|"prepared data"| node_eda
node_eda -->|"inspect"| node_class_study
node_class_study -->|"findings"| node_training
node_cleaning -->|"dataset"| node_training
node_training -->|"benchmark"| node_model_compare
node_model_compare -->|"score"| node_metrics
node_metrics -->|"persist best"| node_model_store
node_model_store -->|"load model"| node_app
node_app -->|"collect input"| node_input_form
node_input_form -->|"features"| node_predictor
node_predictor -->|"predict"| node_confidence
node_confidence -->|"alert record"| node_export_csv
node_report -.->|"document"| node_readme
node_cleaning -.->|"results"| node_report
node_training -.->|"results"| node_report
node_app -.->|"behavior"| node_report

click node_cleaning "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/01_data_cleaing.ipynb"
click node_feature_prep "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/01_data_cleaing.ipynb"
click node_eda "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/02_eda.ipynb"
click node_class_study "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/02_eda.ipynb"
click node_training "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/03_model_training.ipynb"
click node_model_compare "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/03_model_training.ipynb"
click node_metrics "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/03_model_training.ipynb"
click node_app "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py"
click node_input_form "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py"
click node_predictor "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py"
click node_confidence "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py"
click node_export_csv "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py"
click node_report "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/report.md"
click node_readme "https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/README.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cicids2017 toneBlue
class node_cleaning,node_feature_prep,node_eda,node_class_study,node_training,node_model_compare,node_metrics,node_model_store toneAmber
class node_app,node_input_form,node_predictor,node_confidence,node_export_csv toneMint
class node_report,node_readme toneRose
```

## Components

### External Data

Raw CICIDS2017 network-flow records entering the project

- **CICIDS2017**: raw dataset · Source network-traffic CSV records used for training and dashboard inputs

### Offline ML Pipeline

Notebook-driven preparation, analysis, training, and model packaging

- [Data Cleaning](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/01_data_cleaing.ipynb): notebook preprocessing · Cleans raw traffic data and prepares model-ready records · `01_data_cleaing.ipynb`
- [Feature Prep](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/01_data_cleaing.ipynb): data transform · Representative preprocessing step for normalization, encoding, and label shaping · `01_data_cleaing.ipynb`
- [EDA](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/02_eda.ipynb): notebook analysis · Explores class balance, feature behavior, and attack patterns · `02_eda.ipynb`
- [Class Study](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/02_eda.ipynb): analysis output · Representative inspection of traffic classes and attack distributions · `02_eda.ipynb`
- [Model Training](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/03_model_training.ipynb): notebook training · Trains and compares Random Forest, XGBoost, and Decision Tree models · `03_model_training.ipynb`
- [Model Compare](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/03_model_training.ipynb): training comparison · Representative classifier benchmarking and selection step · `03_model_training.ipynb`
- [Metrics](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/03_model_training.ipynb): evaluation · Accuracy, F1, and ROC-AUC evaluation for candidate models · `03_model_training.ipynb`
- **Saved Model**: artifact store · Persisted trained model artifact consumed by the dashboard

### Streamlit Runtime

Interactive detection and alert export in the only user-facing app

- [Dashboard](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py): streamlit app · User-facing Streamlit interface for interactive intrusion detection · `app.py`
- [Input Form](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py): ui input · Collects traffic-like feature values from the user · `app.py`
- [Inference](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py): model inference · Loads the saved classifier and predicts benign or attack class · `app.py`
- [Confidence](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py): prediction output · Displays prediction confidence and alert state · `app.py`
- [CSV Export](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/app.py): alert export · Exports detection alerts for review or incident response · `app.py`

### Documentation

Project narrative and usage notes

- [Report](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/report.md): project report · Written summary of the project, results, and methodology · `report.md`
- [README](https://github.com/abdelrhman-abu-alhijaa/ids-ml-project/blob/main/README.md): overview doc · Repository overview and setup guidance · `README.md`

## Connections

- CICIDS2017 → Data Cleaning: raw flows · Raw CSV traffic records are cleaned for downstream analysis
- Data Cleaning → Feature Prep: preprocess · Cleaning produces model-ready feature and label data
- Feature Prep → EDA: prepared data · Prepared data is inspected in exploratory analysis
- EDA → Class Study: inspect · EDA surfaces class distribution and traffic patterns
- Class Study → Model Training: findings · Analysis informs model training choices
- Data Cleaning → Model Training: dataset · Processed data feeds the training notebook
- Model Training → Model Compare: benchmark · Multiple classifiers are compared during training
- Model Compare → Metrics: score · Candidate models are evaluated with classification metrics
- Metrics → Saved Model: persist best · Selected trained model is saved for inference
- Saved Model → Dashboard: load model · The Streamlit app loads the persisted artifact at runtime
- Dashboard → Input Form: collect input · The dashboard captures traffic-like feature values
- Input Form → Inference: features · Entered values are passed to the inference step
- Inference → Confidence: predict · Inference returns class and confidence information
- Confidence → CSV Export: alert record · Predictions can be exported as CSV alerts
- Report → README: document · Project write-up complements repository usage notes
- Data Cleaning → Report: results · Preprocessing and experiment outcomes are summarized in the report
- Model Training → Report: results · Training and evaluation results are captured in the report
- Dashboard → Report: behavior · The report documents the dashboard and detection workflow