# glcrealm/ids-anomaly-detection-with-nsl-kdd architecture

> Architecture diagram of the GitHub repository glcrealm/ids-anomaly-detection-with-nsl-kdd, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/glcrealm/ids-anomaly-detection-with-nsl-kdd
- Repository: https://github.com/glcrealm/ids-anomaly-detection-with-nsl-kdd

## Overview

##### Project type and purpose
- This is an open-source machine-learning tool for network intrusion detection.
- It trains and evaluates an anomaly detection model (Random Forest) on the NSL-KDD dataset and provides a simple prediction interface via a test script.

##### File structure
- Top-level assets:
- IDS_train.ipynb – interactive notebook for data preparation, training, evaluation
- test_model.py – standalone inference script
- ebest_model.pkl – serialized trained model
- dataset/ – raw data files (KDDTrain+.txt, KDDTest+.txt, kddcup.names, attack.types)
- requirements.txt – Python dependencies
- No separate frontend or backend; it is a data-pipeline + model serving tool.

##### README for additional insights
- Confirms the use of NSL-KDD dataset, preprocessing steps, feature engineering, hyperparameter tuning, metrics.
- No external services or UI components; it is a CLI/notebook-driven workflow.

a. Main components
1. Data Storage
– Raw dataset folder with training and test data
2. Data Preprocessing & Feature Engineering
– Encoding categorical features, computing protocol/attack probabilities
3. Model Training Pipeline
– Randomized search, cross-validation, Random Forest fitting
4. Model Persistence
– Export/serialize best model to ebest_model.pkl via joblib
5. Inference Engine
– test_model.py loads ebest_model.pkl, applies same preprocessing, outputs normal/anomaly
6. Evaluation Module (within notebook)
– Accuracy, precision, recall, F1, confusion matrix
b. Relationships and interactions
c. Architectural patterns and principles
– Pipeline pattern: clear separation of data ingestion, transformation, training, serving
– Modular design: reusable preprocessing functions imported both by training notebook and test script
– Versioning of artifacts: model serialized with a deterministic name for reproducibility
d. Relevant technologies/libraries
– pandas, numpy for data handling
– scikit-learn for modeling (RandomForestClassifier, model_selection)
– joblib for model I/O
– matplotlib (notebook) for metric visualization
– Python scripts/notebook as orchestration layer

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        DataStorage["Dataset (KDDTrain+, KDDTest+)"]:::data
        Requirements["requirements.txt"]:::doc
    end

    %% Preprocessing Layer
    subgraph "Data Preprocessing & Feature Engineering"
        Preprocess["Preprocessor"]:::compute
        subgraph "Preprocessing Steps"
            Enc["Categorical Encoding"]:::compute
            Prob["Compute Probabilities"]:::compute
            Scale["Scaling/Normalization"]:::compute
        end
    end

    FeatureStore["Feature Store (DataFrame)"]:::data

    %% Training Layer
    subgraph "Model Training Pipeline"
        Trainer["Trainer"]:::compute
        HP["Hyperparameter Search & Cross-Validation"]:::compute
    end

    %% Model Persistence
    ModelFile["ebest_model.pkl"]:::artifact

    %% Evaluation Module
    Eval["Evaluation Module\n(Metrics & Visualization)"]:::artifact

    %% Inference Layer
    subgraph "Inference Engine"
        Inference["Inference Engine"]:::compute
    end
    Prediction["Predictions\n(Normal/Anomaly)"]:::artifact

    %% Connections
    DataStorage -->|"raw records"| Preprocess
    Preprocess -->|"feature vectors"| FeatureStore
    FeatureStore -->|"feature vectors"| Trainer
    Trainer -->|"trained model"| ModelFile
    Trainer -->|"metrics data"| Eval
    Preprocess -->|"reuse preprocessing"| Inference
    ModelFile -->|"load model"| Inference
    Inference -->|"predictions"| Prediction
    Requirements -.-> Preprocess
    Requirements -.-> Trainer
    Requirements -.-> Inference

    %% Click Events
    click DataStorage "https://github.com/glcrealm/ids-anomaly-detection-with-nsl-kdd/tree/main/dataset/"
    click Preprocess "https://github.com/glcrealm/ids-anomaly-detection-with-nsl-kdd/blob/main/IDS_train.ipynb"
    click Trainer "https://github.com/glcrealm/ids-anomaly-detection-with-nsl-kdd/blob/main/IDS_train.ipynb"
    click ModelFile "https://github.com/glcrealm/ids-anomaly-detection-with-nsl-kdd/blob/main/ebest_model.pkl"
    click Inference "https://github.com/glcrealm/ids-anomaly-detection-with-nsl-kdd/blob/main/test_model.py"
    click Eval "https://github.com/glcrealm/ids-anomaly-detection-with-nsl-kdd/blob/main/IDS_train.ipynb"
    click Requirements "https://github.com/glcrealm/ids-anomaly-detection-with-nsl-kdd/blob/main/requirements.txt"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000;
    classDef compute fill:#90EE90,stroke:#000;
    classDef artifact fill:#FFA500,stroke:#000;
    classDef doc fill:#D3D3D3,stroke:#000;
```
