How isratmou/ids-moi works
1. Repository shape
- This is primarily a research/training notebook project with a small deployment app for a live demo.
- The core artifacts are ton-iot-moi-cse-414.ipynb for end-to-end experimentation, app.py for the interactive Hugging Face Spaces UI, and hf_artifacts.zip as the packaged model/runtime assets used by the demo.
2. Primary system purpose
- The system is a hierarchical intrusion detection pipeline for IoT traffic.
- It first performs binary classification: Normal vs. Attack.
- If traffic is malicious, it performs a second multiclass attack-type classification.
3. Model family boundary
- The repository compares three architectures: DNN, CNN, and MOI-Lite.
- MOI-Lite is the main contribution and the default lightweight architecture.
- The architecture is tuned for constrained devices, with the README emphasizing small parameter count and INT8 quantization for edge deployment.
4. Training/data preparation pipeline
- The notebook owns the full data pipeline from the TON-IoT dataset through preprocessing to model training.
- Preprocessing includes sparse feature removal, engineered ratio/log features, one-hot encoding, normalization, and class-balancing strategy.
- This is a classic offline ML boundary: raw dataset in, cleaned feature tensors out.
5. Hierarchical inference flow
- The functional data flow is: IoT network features → Stage 1 binary detector → Stage 2 attack classifier → final label.
- Stage 2 is only entered when Stage 1 predicts attack traffic.
- This design reduces unnecessary multiclass inference on normal traffic and is the main architectural boundary between detection and attribution.
6. MOI-Lite internals
- MOI-Lite combines multi-scale dilated 1D CNN blocks with lightweight self-attention.
- It also uses spectral normalization and DropPath/stochastic depth.
- Architecturally, it sits between a pure CNN and a transformer-like model, but optimized for small-footprint IoT classification.
7. E-SATF training layer
- E-SATF is not a separate model; it is a training framework/loss augmentation layered on top of the base architectures.
- Its goal is to improve SHAP explanation stability under perturbation.
- This makes the training boundary important: the same model family can be trained with or without E-SATF, affecting both interpretability and robustness.
8. Evaluation and analysis subsystems
- The notebook includes dedicated phases for metrics, SHAP stability analysis, FGSM adversarial testing, quantization, and latency benchmarking.
- These are evaluation subsystems rather than runtime features, but they feed the paper-ready result tables and figures.
- The repo treats interpretability and adversarial robustness as first-class evaluation outputs, not just accuracy.
9. Deployment/runtime boundary
- app.py represents the inference-facing dashboard exposed through Hugging Face Spaces.
- The live demo lets users supply IoT feature values, choose a model variant, and view predictions plus SHAP explanations.
- This is separate from training: the app is the lightweight consumer of precomputed artifacts rather than the place where the models are trained.
10. Packaged artifacts
- hf_artifacts.zip likely contains the serialized models, preprocessing objects, or other assets required by the Hugging Face demo.
- This creates a clear split between offline research assets and online demo assets.
- For an architecture diagram, it should appear as a deployment bundle feeding the app/inference path.
11. Technology stack
- The project is Python-based and centered on TensorFlow 2.x, with SHAP for explanations and scikit-learn / pandas / numpy for data handling and evaluation.
- The README also ties execution to Kaggle notebooks and Hugging Face Spaces, which are the main execution and hosting environments.
12. What to show in a diagram
- The most important boxes are: TON-IoT dataset, preprocessing/feature engineering, Stage 1 binary model, Stage 2 attack classifier, E-SATF training layer, explainability/robustness evaluation, and deployment app with packaged artifacts.
- The key arrows are: dataset → preprocessing → training/evaluation; and packaged artifacts → app → user-facing predictions/explanations.
Main entry points: app.py Hugging Face app · ton-iot-moi-cse-414.ipynb Training notebook · requirements.txt Python ML stack · hf_artifacts.zip HF bundle
Components 14
Research / training
Offline experimentation, preprocessing, model training, and analysis.
- TON-IoT · Raw IoT traffic dataset used as the source for the offline pipeline.
- Training notebook · End-to-end experimentation workflow for preprocessing, training, and evaluation.
ton-iot-moi-cse-414.ipynb - Preprocessing · Sparse feature removal, engineered ratios/logs, one-hot encoding, normalization, and balancing.
- Stage 1 · First-level Normal vs Attack classifier.
- Stage 2 · Attack-type attribution model used only after malicious traffic is detected.
- MOI-Lite · Main contribution: multi-scale dilated 1D CNN with lightweight attention, spectral normalization, and DropPath.
- DNN · Dense neural baseline used for comparison.
- CNN · Convolutional baseline used for comparison.
- E-SATF · Training-time augmentation/loss layer for SHAP stability and robustness.
- Evaluation · Metrics, SHAP stability, FGSM adversarial testing, quantization, and latency benchmarking.
- Python ML stack · TensorFlow, SHAP, scikit-learn, pandas, and numpy support training and inference.
requirements.txt
Deployment / demo
Lightweight inference path for the live Hugging Face Space.
- Hugging Face app · Interactive inference dashboard for user-provided IoT features, model selection, predictions, and SHAP explanations.
app.py - User · Demo user supplying features and viewing predictions/explanations.
Packaged assets
Serialized artifacts shared between training and the demo app.
- HF bundle · Serialized models and preprocessing assets packaged for the demo runtime.
hf_artifacts.zip
Connections 24
- TON-IoT → Preprocessing: raw data · TON-IoT flows into the feature pipeline.
- Preprocessing → Stage 1: train/infer · Cleaned feature tensors feed the first-stage detector.
- Stage 1 → Stage 2: if attack · Only malicious traffic continues to attack attribution.
- Stage 1 → Evaluation: metrics · Stage 1 is evaluated for accuracy, robustness, and latency.
- Stage 2 → Evaluation: metrics · Stage 2 contributes classification and explanation results.
- MOI-Lite → Stage 1: architecture · MOI-Lite can implement the binary detector.
- MOI-Lite → Stage 2: architecture · MOI-Lite can also implement the attack classifier.
- DNN → Stage 1: baseline · DNN serves as a comparison architecture.
- CNN → Stage 1: baseline · CNN serves as a comparison architecture.
- E-SATF → MOI-Lite: training · E-SATF augments MOI-Lite during training.
- E-SATF → DNN: training · E-SATF can also wrap the DNN baseline.
- E-SATF → CNN: training · E-SATF can also wrap the CNN baseline.
- Evaluation → HF bundle: export · Trained models and preprocessing assets are packaged for demo use.
- HF bundle → Hugging Face app: load · The demo app consumes the packaged runtime assets.
- User → Hugging Face app: inputs · Users provide IoT feature values and choose a model variant.
- Hugging Face app → User: predictions · The app returns labels, probabilities, and SHAP explanations.
- Python ML stack → Training notebook: supports · The notebook uses the Python ML stack for training and analysis.
- Python ML stack → Hugging Face app: supports · The demo runtime depends on the same inference libraries.
- Training notebook → Preprocessing: orchestrates · The notebook owns the offline data preparation pipeline.
- Training notebook → MOI-Lite: trains · The notebook trains and compares the main model family.
- Training notebook → E-SATF: analyzes · The notebook evaluates training with and without E-SATF.
- Training notebook → Evaluation: reports · The notebook produces metrics, robustness, quantization, and latency results.
- Hugging Face app → Stage 1: inference path · The live demo routes inputs through the hierarchical detector.
- Hugging Face app → Stage 2: inference path · Attack cases continue to the second-stage classifier.