# jaslyn-j/concept-drift-detecting_nids architecture

> Architecture diagram of the GitHub repository jaslyn-j/concept-drift-detecting_nids, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jaslyn-j/concept-drift-detecting_nids
- Repository: https://github.com/jaslyn-j/concept-drift-detecting_nids

## Overview

This repository is a research NIDS demonstration: it trains flow classifiers, simulates concept drift, evaluates detection and adaptation strategies, and presents saved experiment results in a local Streamlit dashboard. The main user workflow is to explore the simulated stream and model performance, replay drift events, and submit a flow for prediction. The README documents the experiment stages and dashboard capabilities; the sampled app excerpt shows dashboard visualizations and prediction/performance UI, but not its data-loading code, so artifact links rely on README statements.

Main entry points: [`notebooks/concept_drift_detection_handling.ipynb`](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb) (Stream simulation), [`app.py`](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py) (Streamlit dashboard), [`notebooks/anomaly_normal_traffic_detection.ipynb`](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/anomaly_normal_traffic_detection.ipynb) (Data exploration), [`data/preprocess.joblib`](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/preprocess.joblib) (Preprocessing artifact)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experiments["Modeling Experiments"]
  node_exploration["Data exploration"]
  node_preprocessing["Preprocessing pipeline"]
  node_baseline["Baseline classifiers"]
end

subgraph group_drift["Drift Experiments"]
  node_streambuild["Stream simulation"]
  node_staticmodel["Static model"]
  node_detectors["Drift detectors"]
  node_retraining["Triggered retraining"]
  node_arf["Online ARF"]
  node_evaluation["Prequential evaluation"]
end

subgraph group_dashboard["Research Dashboard"]
  node_dashboardapp["Streamlit dashboard<br/>[app.py]"]
  node_explorer["Data explorer<br/>[app.py]"]
  node_replay["Stream replay<br/>[app.py]"]
  node_prediction["Manual prediction<br/>[app.py]"]
  node_capacity["Capacity analysis<br/>[app.py]"]
end

subgraph group_artifacts["Saved Artifacts"]
  node_preprocessfile["Preprocessing artifact<br/>[preprocess.joblib]"]
  node_initialmodel["Initial model"]
  node_adaptedmodel["Adapted model"]
  node_streamfile[("Simulated stream<br/>[stream.parquet]")]
  node_metrics["Window metrics<br/>[window_metrics.csv]"]
  node_driftevents["Drift event log<br/>[drift_events.json]"]
  node_performance["Performance report"]
  node_metadata["Experiment metadata<br/>[meta.json]"]
end

node_researcher(("Researcher"))
node_unsw[("UNSW-NB15 data")]

node_researcher -->|"opens"| node_dashboardapp
node_unsw -->|"examines"| node_exploration
node_exploration -->|"informs"| node_preprocessing
node_preprocessing -->|"saves pipeline"| node_preprocessfile
node_exploration -->|"trains and scores"| node_baseline
node_baseline -.->|"saves model"| node_initialmodel
node_unsw -->|"provides rows"| node_streambuild
node_streambuild -.->|"builds stream"| node_streamfile
node_preprocessfile -->|"transforms data"| node_streambuild
node_streambuild -->|"evaluates on stream"| node_staticmodel
node_streambuild -->|"supplies windows"| node_detectors
node_detectors -->|"triggers updates"| node_retraining
node_streambuild -->|"feeds rows"| node_arf
node_staticmodel -->|"contributes results"| node_evaluation
node_retraining -->|"contributes results"| node_evaluation
node_arf -->|"contributes results"| node_evaluation
node_evaluation -.->|"reports metrics"| node_metrics
node_retraining -.->|"produces model"| node_adaptedmodel
node_dashboardapp -->|"presents"| node_explorer
node_dashboardapp -->|"presents"| node_replay
node_dashboardapp -->|"presents"| node_prediction
node_dashboardapp -->|"presents"| node_capacity
node_streamfile -.->|"provides data"| node_explorer
node_streamfile -.->|"provides windows"| node_replay
node_metrics -.->|"shows performance"| node_replay
node_driftevents -.->|"marks alarms"| node_replay
node_preprocessfile -->|"transforms input"| node_prediction
node_initialmodel -.->|"classifies input"| node_prediction
node_performance -->|"provides measurements"| node_capacity
node_metadata -.->|"provides defaults"| node_prediction

click node_exploration "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/anomaly_normal_traffic_detection.ipynb"
click node_preprocessing "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/anomaly_normal_traffic_detection.ipynb"
click node_baseline "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/anomaly_normal_traffic_detection.ipynb"
click node_streambuild "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb"
click node_staticmodel "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb"
click node_detectors "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb"
click node_retraining "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb"
click node_arf "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb"
click node_evaluation "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb"
click node_dashboardapp "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py"
click node_explorer "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py"
click node_replay "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py"
click node_prediction "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py"
click node_capacity "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py"
click node_preprocessfile "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/preprocess.joblib"
click node_initialmodel "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/model_initial.joblib"
click node_adaptedmodel "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/model_adapted.joblib"
click node_streamfile "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/stream.parquet"
click node_metrics "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/window_metrics.csv"
click node_driftevents "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/drift_events.json"
click node_performance "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/performance_report.csv"
click node_metadata "https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/meta.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_exploration,node_preprocessing,node_baseline toneBlue
class node_streambuild,node_staticmodel,node_detectors,node_retraining,node_arf,node_evaluation,node_unsw toneAmber
class node_dashboardapp,node_explorer,node_replay,node_prediction,node_capacity toneMint
class node_preprocessfile,node_initialmodel,node_adaptedmodel,node_streamfile,node_metrics,node_driftevents,node_performance,node_metadata toneRose
class node_researcher toneIndigo
```

## Components

### Modeling Experiments

- [Data exploration](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/anomaly_normal_traffic_detection.ipynb): `notebooks/anomaly_normal_traffic_detection.ipynb`
- [Preprocessing pipeline](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/anomaly_normal_traffic_detection.ipynb): `notebooks/anomaly_normal_traffic_detection.ipynb`
- [Baseline classifiers](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/anomaly_normal_traffic_detection.ipynb): `notebooks/anomaly_normal_traffic_detection.ipynb`

### Drift Experiments

- [Stream simulation](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb): `notebooks/concept_drift_detection_handling.ipynb`
- [Static model](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb): `notebooks/concept_drift_detection_handling.ipynb`
- [Drift detectors](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb): `notebooks/concept_drift_detection_handling.ipynb`
- [Triggered retraining](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb): `notebooks/concept_drift_detection_handling.ipynb`
- [Online ARF](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb): `notebooks/concept_drift_detection_handling.ipynb`
- [Prequential evaluation](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/notebooks/concept_drift_detection_handling.ipynb): `notebooks/concept_drift_detection_handling.ipynb`

### Research Dashboard

- [Streamlit dashboard](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py): `app.py`
- [Data explorer](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py): `app.py`
- [Stream replay](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py): `app.py`
- [Manual prediction](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py): `app.py`
- [Capacity analysis](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/app.py): `app.py`

### Saved Artifacts

- [Preprocessing artifact](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/preprocess.joblib): `data/preprocess.joblib`
- [Initial model](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/model_initial.joblib): `data/model_initial.joblib`
- [Adapted model](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/model_adapted.joblib): `data/model_adapted.joblib`
- [Simulated stream](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/stream.parquet): `data/stream.parquet`
- [Window metrics](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/window_metrics.csv): `data/window_metrics.csv`
- [Drift event log](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/drift_events.json): `data/drift_events.json`
- [Performance report](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/performance_report.csv): `data/performance_report.csv`
- [Experiment metadata](https://github.com/jaslyn-j/concept-drift-detecting_nids/blob/main/data/meta.json): `data/meta.json`

### Other components

- **Researcher**
- **UNSW-NB15 data**

## Connections

- Researcher → Streamlit dashboard: opens (evidence: `README.md`)
- UNSW-NB15 data → Data exploration: examines (evidence: `README.md`)
- Data exploration → Preprocessing pipeline: informs (evidence: `README.md`)
- Preprocessing pipeline → Preprocessing artifact: saves pipeline (evidence: `README.md`)
- Data exploration → Baseline classifiers: trains and scores (evidence: `README.md`)
- Baseline classifiers → Initial model: saves model
- UNSW-NB15 data → Stream simulation: provides rows (evidence: `README.md`)
- Stream simulation → Simulated stream: builds stream
- Preprocessing artifact → Stream simulation: transforms data (evidence: `README.md`)
- Stream simulation → Static model: evaluates on stream (evidence: `README.md`)
- Stream simulation → Drift detectors: supplies windows (evidence: `README.md`)
- Drift detectors → Triggered retraining: triggers updates (evidence: `README.md`)
- Stream simulation → Online ARF: feeds rows (evidence: `README.md`)
- Static model → Prequential evaluation: contributes results (evidence: `README.md`)
- Triggered retraining → Prequential evaluation: contributes results (evidence: `README.md`)
- Online ARF → Prequential evaluation: contributes results (evidence: `README.md`)
- Prequential evaluation → Window metrics: reports metrics
- Triggered retraining → Adapted model: produces model
- Streamlit dashboard → Data explorer: presents (evidence: `README.md`)
- Streamlit dashboard → Stream replay: presents (evidence: `README.md`)
- Streamlit dashboard → Manual prediction: presents (evidence: `README.md`)
- Streamlit dashboard → Capacity analysis: presents (evidence: `README.md`)
- Simulated stream → Data explorer: provides data
- Simulated stream → Stream replay: provides windows
- Window metrics → Stream replay: shows performance
- Drift event log → Stream replay: marks alarms
- Preprocessing artifact → Manual prediction: transforms input (evidence: `README.md`)
- Initial model → Manual prediction: classifies input
- Performance report → Capacity analysis: provides measurements (evidence: `README.md`)
- Experiment metadata → Manual prediction: provides defaults