How zhenlus/gnn-ids works
Main Components
• Data Sources:
• Datasets: There are two distinct datasets. One is synthetic (located in datasets/synt) and the other is based on the public CICIDS-2017 dataset (located in datasets/public).
• Attack Graph: The attack graph is provided in the mulval_attack_graph folder (both as textual representation via a .dot file and as a visual PDF). This graph defines the structure of the network attacks used for analysis.
• Data Preprocessing & Loading:
• Jupyter Notebooks (gnn_ids_dataset1.ipynb and gnn_ids_dataset2.ipynb) act as the entry point for loading and preprocessing the datasets.
• Python modules such as public_data.py and synthetic_data.py handle the specifics of loading and possibly transforming or normalizing the data.
• Model and Inference Engine:
• The model layer is implemented using PyTorch and PyTorch Geometric, as indicated by the specific version requirements in the README.
• The src/models.py, model_utils.py, and ag_utils.py files indicate that there are custom utilities for constructing and managing graph neural network models.
• Experiment and Evaluation:
• The notebooks also serve as environments to run the experiments, showing how data flows from ingestion, graph construction, through to training, inference, and evaluation.
Relationships and Interactions
a. Data Input Layer:
• Two primary sources: public and synthetic datasets.
• The attack graph component providing static network structure information.
b. Data Processing Layer:
• Notebooks and data loader modules that pre-process the raw dataset files and integrate the attack graph information.
c. Model Layer:
• Graph Neural Network (GNN) model implementation utilizing PyTorch and PyTorch Geometric.
• Utility modules (model_utils.py and ag_utils.py) that interface between the raw data processing and model training.
d. Output/Results:
• The inference outputs (predictions or detected anomalies) which would then be further analyzed in the notebook for evaluation.
• Data flows from the datasets to the preprocessing modules.
• Preprocessed data and the attack graph are merged into a unified data representation.
• That representation flows into the GNN inference engine for model training and detection.
• Results are then sent back to the experiment notebooks for visualization and analysis.
Architectural Patterns and Design Principles
• Separation of concerns is clearly visible: data handling, graph integration, and model execution are modular and segmented into separate files and directories.
• Use of pipelines: preprocessing in notebooks, discrete modules for processing and transformation, and finally, the GNN model acting as the inferential engine.
• The use of graph structures (via the attack graph and PyTorch Geometric) is central to integrating static network vulnerabilities with dynamic data.
Relevant Technologies
• Python ecosystem with emphasis on PyTorch for deep learning.
• PyTorch Geometric for handling graph-structured data.
• Jupyter Notebooks as interactive experiment interfaces.
• Dot file visualization (provided in mulval_attack_graph) for network attack graph representation.
• Blue shades for data sources.
• Green for preprocessing/integration modules.
• Orange or yellow for the core model/inference engine.
• Purple for output and evaluation components.