How jaslyn-j/concept-drift-detecting_nids works
This repository is a research NIDS demonstration: it trains flow classifiers, simulates concept drift, evaluates detection and adaptation strategies, and presents saved experiment results in a local Streamlit dashboard. The main user workflow is to explore the simulated stream and model performance, replay drift events, and submit a flow for prediction. The README documents the experiment stages and dashboard capabilities; the sampled app excerpt shows dashboard visualizations and prediction/performance UI, but not its data-loading code, so artifact links rely on README statements.
Main entry points: concept_drift_detection_handling.ipynb Stream simulation · app.py Streamlit dashboard · anomaly_normal_traffic_detection.ipynb Data exploration · preprocess.joblib Preprocessing artifact
Components 24
Modeling Experiments
- Data exploration
notebooks/anomaly_normal_traffic_detection.ipynb - Preprocessing pipeline
notebooks/anomaly_normal_traffic_detection.ipynb - Baseline classifiers
notebooks/anomaly_normal_traffic_detection.ipynb
Drift Experiments
- Stream simulation
notebooks/concept_drift_detection_handling.ipynb - Static model
notebooks/concept_drift_detection_handling.ipynb - Drift detectors
notebooks/concept_drift_detection_handling.ipynb - Triggered retraining
notebooks/concept_drift_detection_handling.ipynb - Online ARF
notebooks/concept_drift_detection_handling.ipynb - Prequential evaluation
notebooks/concept_drift_detection_handling.ipynb
Research Dashboard
- Streamlit dashboard
app.py - Data explorer
app.py - Stream replay
app.py - Manual prediction
app.py - Capacity analysis
app.py
Saved Artifacts
- Preprocessing artifact
data/preprocess.joblib - Initial model
data/model_initial.joblib - Adapted model
data/model_adapted.joblib - Simulated stream
data/stream.parquet - Window metrics
data/window_metrics.csv - Drift event log
data/drift_events.json - Performance report
data/performance_report.csv - Experiment metadata
data/meta.json
Other components
- Researcher
- UNSW-NB15 data
Connections 30
- Researcher → Streamlit dashboard: opens
README.md - UNSW-NB15 data → Data exploration: examines
README.md - Data exploration → Preprocessing pipeline: informs
README.md - Preprocessing pipeline → Preprocessing artifact: saves pipeline
README.md - Data exploration → Baseline classifiers: trains and scores
README.md - Baseline classifiers → Initial model: saves model
- UNSW-NB15 data → Stream simulation: provides rows
README.md - Stream simulation → Simulated stream: builds stream
- Preprocessing artifact → Stream simulation: transforms data
README.md - Stream simulation → Static model: evaluates on stream
README.md - Stream simulation → Drift detectors: supplies windows
README.md - Drift detectors → Triggered retraining: triggers updates
README.md - Stream simulation → Online ARF: feeds rows
README.md - Static model → Prequential evaluation: contributes results
README.md - Triggered retraining → Prequential evaluation: contributes results
README.md - Online ARF → Prequential evaluation: contributes results
README.md - Prequential evaluation → Window metrics: reports metrics
- Triggered retraining → Adapted model: produces model
- Streamlit dashboard → Data explorer: presents
README.md - Streamlit dashboard → Stream replay: presents
README.md - Streamlit dashboard → Manual prediction: presents
README.md - Streamlit dashboard → Capacity analysis: presents
README.md - Simulated stream → Data explorer: provides data
- Simulated stream → Stream replay: provides windows
- Window metrics → Stream replay: shows performance
- Drift event log → Stream replay: marks alarms
- Preprocessing artifact → Manual prediction: transforms input
README.md - Initial model → Manual prediction: classifies input
- Performance report → Capacity analysis: provides measurements
README.md - Experiment metadata → Manual prediction: provides defaults