# anish-dev09/ics-networks architecture

> Architecture diagram of the GitHub repository anish-dev09/ics-networks, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anish-dev09/ics-networks
- Repository: https://github.com/anish-dev09/ics-networks

## Overview

1. This repository is organized as a full ICS intrusion-detection pipeline, not just a single model. The main flow is: HAI dataset ingestion from `data/raw/` → preprocessing and feature/sequence generation in `src/` → model training/evaluation scripts at the repo root → persisted artifacts in `results/` → Streamlit-based demo UI.

2. The data boundary is clear. Raw HAI dataset files live under `data/raw/hai`, while derived CNN-ready arrays are stored in `data/processed/cnn_sequences` as NumPy tensors plus metadata JSON. That split indicates two parallel preparation paths: one for tabular ML models and one for temporal deep-learning models.

3. `src/data` is the ingestion layer. `hai_loader.py` loads and slices the HAI dataset, `data_loader.py` provides general loading utilities, and `sequence_generator.py` transforms sensor time series into sliding windows for the CNN pipeline. This is the handoff point between raw CSV data and model-ready inputs.

4. `src/features` is the engineered-feature subsystem for classical ML. `feature_engineering.py` and `ics_feature_extractor.py` build statistical, temporal, lag, and correlation-based features from sensor streams. The repo’s Random Forest and XGBoost models depend on this layer rather than on raw sequences.

5. `src/models` contains the detector implementations themselves. `baseline_detector.py` provides anomaly baselines such as Isolation Forest, Z-score, and IQR rules; `ml_models.py` wraps Random Forest and XGBoost; `cnn_models.py` defines the 1D-CNN architecture; `deep_anomaly_detector.py`, `ensemble_detector.py`, and `encrypted_traffic_detector.py` suggest additional higher-level detection variants; and `protocol_validator.py` likely enforces protocol-aware sanity checks before scoring.

6. Protocol awareness is a separate subsystem under `src/protocols`. Parsers for Modbus, DNP3, and S7comm, plus `protocol_detector.py`, show that the project can inspect industrial network traffic at the application/protocol layer rather than treating all flows as generic packets. This is an important architecture boundary because protocol parsing feeds specialized validation and encrypted-traffic handling.

7. The orchestration layer lives in `src/detection`. `integrated_detector.py` and `advanced_integrated_detector.py` are the likely top-level coordinators that combine protocol parsing, feature extraction, temporal context, model inference, and post-processing/thresholding into a single detection decision. This is where the repository moves from “individual models” to “system-level detector.”

8. `src/utils` holds the adaptive and operational logic around the detectors. Modules such as `adaptive_threshold.py`, `temporal_analyzer.py`, `advanced_temporal_context.py`, `online_learning_system.py`, `automatic_pattern_learning.py`, `benign_pattern_learner.py`, `class_imbalance_handler.py`, and `performance_optimizer.py` indicate support for concept drift, imbalance handling, temporal context, and runtime tuning. These utilities sit beside the models rather than inside them, which keeps adaptation concerns separate from core inference code.

9. Training and evaluation are driven by repository-level scripts. `train_ml_models.py`, `train_cnn_model.py`, `prepare_cnn_data.py`, `compare_models.py`, `optimize_thresholds.py`, `train_zero_day_detector.py`, and `quick_test_baseline.py` form the offline pipeline entry points. They are responsible for generating artifacts and metrics rather than serving live traffic.

10. Persisted outputs are split into model artifacts and evaluation artifacts under `results/`. `results/models` contains trained binaries and their companion metadata, including the CNN Keras model, ML pickles, and feature-engineering objects; `results/metrics` contains CSV summaries for model comparison, baseline results, and feature importance. That makes the repository reproducible: the training code produces both inference assets and auditability data.

11. The demo boundary is separate from training. `streamlit_app.py` and `demo/app.py` provide the UI layer, while `demo/mock_data.py` and `demo/mock_hai_data.py` supply synthetic/sample traffic for interactive exploration. This suggests the live dashboard is designed to visualize detector behavior without requiring the full training pipeline to run inside the app.

12. Configuration and environment management are lightweight but explicit. `configs/config.yaml` centralizes settings, `src/utils/config_utils.py` loads them, and `.devcontainer` plus `.streamlit/config.toml` show the project is set up for both containerized development and Streamlit deployment. The repository is therefore structured for offline research workflows and a separate presentation/demo runtime.

13. Verification is covered by focused tests in the repo root: `test_feature_engineering.py`, `test_model_loading.py`, and `test_protocols.py`. These correspond to the key architecture seams: feature generation, serialized model compatibility, and protocol parsing correctness.

14. In diagram terms, the most important boundaries are: raw HAI data vs. derived feature/sequence data, tabular ML vs. sequence-based CNN processing, protocol parsing vs. generic sensor modeling, and offline training/evaluation vs. interactive Streamlit demo. Those are the core subsystems that define how data moves through the repository.

Main entry points: [`src/detection/integrated_detector.py`](https://github.com/anish-dev09/ics-networks/blob/main/src/detection/integrated_detector.py) (Integrated detector), [`results/models`](https://github.com/anish-dev09/ics-networks/tree/main/results/models) (Model artifacts), [`src/protocols/protocol_detector.py`](https://github.com/anish-dev09/ics-networks/blob/main/src/protocols/protocol_detector.py) (Protocol detector), [`src/data/hai_loader.py`](https://github.com/anish-dev09/ics-networks/blob/main/src/data/hai_loader.py) (HAI loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data pipeline"]
  node_data_raw_hai[("HAI raw data<br/>dataset")]
  node_data_loader["Data loader<br/>ingestion<br/>[data_loader.py]"]
  node_hai_loader["HAI loader<br/>ingestion<br/>[hai_loader.py]"]
  node_sequence_generator["Sequence gen<br/>sequence prep"]
  node_cnn_sequences[("CNN sequences<br/>processed data")]
end

subgraph group_features["Feature engineering"]
  node_feature_engineering["Feature engineering<br/>feature builder"]
  node_ics_feature_extractor["ICS extractor<br/>feature builder"]
end

subgraph group_models["Detectors"]
  node_baseline_detector["Baseline detector<br/>anomaly rules"]
  node_ml_models["ML models<br/>classical ML<br/>[ml_models.py]"]
  node_cnn_models["CNN models<br/>deep learning<br/>[cnn_models.py]"]
  node_ensemble_detector["Ensemble detector<br/>fusion"]
  node_deep_anomaly_detector["Deep anomaly"]
end

subgraph group_protocols["Protocol layer"]
  node_protocol_detector["Protocol detector<br/>protocol routing"]
  node_modbus_parser["Modbus parser<br/>protocol parser<br/>[modbus_parser.py]"]
  node_dnp3_parser["DNP3 parser<br/>protocol parser<br/>[dnp3_parser.py]"]
  node_s7comm_parser["S7comm parser<br/>protocol parser<br/>[s7comm_parser.py]"]
end

subgraph group_orchestration["Orchestration"]
  node_integrated_detector["Integrated detector<br/>orchestrator"]
  node_advanced_integrated_detector["Advanced detector<br/>orchestrator"]
  node_temporal_analyzer["Temporal analyzer<br/>runtime context"]
  node_adaptive_threshold["Adaptive threshold<br/>runtime tuning"]
  node_online_learning_system["Online learning<br/>adaptation"]
end

subgraph group_training["Training and eval"]
  node_train_ml_models["Train ML<br/>script<br/>[train_ml_models.py]"]
  node_train_cnn_model["Train CNN<br/>script<br/>[train_cnn_model.py]"]
  node_prepare_cnn_data["Prepare CNN<br/>script"]
  node_compare_models["Compare models<br/>evaluation<br/>[compare_models.py]"]
  node_optimize_thresholds["Optimize thresholds<br/>evaluation"]
  node_train_zero_day_detector["Zero-day train<br/>script"]
end

subgraph group_demo["Demo UI"]
  node_streamlit_app["Streamlit app<br/>ui<br/>[streamlit_app.py]"]
  node_demo_app["Demo app<br/>ui module<br/>[app.py]"]
end

subgraph group_artifacts["Persisted artifacts"]
  node_results_models[("Model artifacts<br/>artifact store")]
  node_results_metrics[("Metrics store<br/>artifact store")]
end

node_data_raw_hai -->|"load"| node_data_loader
node_data_loader -->|"specialize"| node_hai_loader
node_hai_loader -->|"tabular path"| node_feature_engineering
node_hai_loader -->|"sequence path"| node_sequence_generator
node_sequence_generator -->|"persist"| node_cnn_sequences
node_feature_engineering -->|"extend"| node_ics_feature_extractor
node_ics_feature_extractor -->|"features"| node_ml_models
node_feature_engineering -->|"features"| node_ml_models
node_sequence_generator -->|"sequences"| node_cnn_models
node_baseline_detector -->|"baseline input"| node_ensemble_detector
node_ml_models -->|"model input"| node_ensemble_detector
node_cnn_models -->|"deep input"| node_deep_anomaly_detector
node_protocol_detector -->|"route"| node_modbus_parser
node_protocol_detector -->|"route"| node_dnp3_parser
node_protocol_detector -->|"route"| node_s7comm_parser
node_modbus_parser -->|"parse"| node_integrated_detector
node_dnp3_parser -->|"parse"| node_integrated_detector
node_s7comm_parser -->|"parse"| node_integrated_detector
node_temporal_analyzer -->|"context"| node_integrated_detector
node_adaptive_threshold -->|"thresholds"| node_integrated_detector
node_online_learning_system -->|"adapt"| node_advanced_integrated_detector
node_integrated_detector -->|"compose"| node_advanced_integrated_detector
node_prepare_cnn_data -->|"generate"| node_cnn_sequences
node_train_ml_models -->|"save"| node_results_models
node_train_cnn_model -->|"save"| node_results_models
node_compare_models -->|"write"| node_results_metrics
node_optimize_thresholds -->|"write"| node_results_metrics
node_train_zero_day_detector -->|"save"| node_results_models
node_results_models -->|"load"| node_streamlit_app
node_results_models -->|"load"| node_demo_app
node_streamlit_app -.->|"share UI"| node_demo_app

click node_data_raw_hai "https://github.com/anish-dev09/ics-networks/tree/main/data/raw/hai"
click node_data_loader "https://github.com/anish-dev09/ics-networks/blob/main/src/data/data_loader.py"
click node_hai_loader "https://github.com/anish-dev09/ics-networks/blob/main/src/data/hai_loader.py"
click node_sequence_generator "https://github.com/anish-dev09/ics-networks/blob/main/src/data/sequence_generator.py"
click node_cnn_sequences "https://github.com/anish-dev09/ics-networks/tree/main/data/processed/cnn_sequences"
click node_feature_engineering "https://github.com/anish-dev09/ics-networks/blob/main/src/features/feature_engineering.py"
click node_ics_feature_extractor "https://github.com/anish-dev09/ics-networks/blob/main/src/features/ics_feature_extractor.py"
click node_baseline_detector "https://github.com/anish-dev09/ics-networks/blob/main/src/models/baseline_detector.py"
click node_ml_models "https://github.com/anish-dev09/ics-networks/blob/main/src/models/ml_models.py"
click node_cnn_models "https://github.com/anish-dev09/ics-networks/blob/main/src/models/cnn_models.py"
click node_ensemble_detector "https://github.com/anish-dev09/ics-networks/blob/main/src/models/ensemble_detector.py"
click node_deep_anomaly_detector "https://github.com/anish-dev09/ics-networks/blob/main/src/models/deep_anomaly_detector.py"
click node_protocol_detector "https://github.com/anish-dev09/ics-networks/blob/main/src/protocols/protocol_detector.py"
click node_modbus_parser "https://github.com/anish-dev09/ics-networks/blob/main/src/protocols/modbus_parser.py"
click node_dnp3_parser "https://github.com/anish-dev09/ics-networks/blob/main/src/protocols/dnp3_parser.py"
click node_s7comm_parser "https://github.com/anish-dev09/ics-networks/blob/main/src/protocols/s7comm_parser.py"
click node_integrated_detector "https://github.com/anish-dev09/ics-networks/blob/main/src/detection/integrated_detector.py"
click node_advanced_integrated_detector "https://github.com/anish-dev09/ics-networks/blob/main/src/detection/advanced_integrated_detector.py"
click node_temporal_analyzer "https://github.com/anish-dev09/ics-networks/blob/main/src/utils/temporal_analyzer.py"
click node_adaptive_threshold "https://github.com/anish-dev09/ics-networks/blob/main/src/utils/adaptive_threshold.py"
click node_online_learning_system "https://github.com/anish-dev09/ics-networks/blob/main/src/utils/online_learning_system.py"
click node_train_ml_models "https://github.com/anish-dev09/ics-networks/blob/main/train_ml_models.py"
click node_train_cnn_model "https://github.com/anish-dev09/ics-networks/blob/main/train_cnn_model.py"
click node_prepare_cnn_data "https://github.com/anish-dev09/ics-networks/blob/main/prepare_cnn_data.py"
click node_compare_models "https://github.com/anish-dev09/ics-networks/blob/main/compare_models.py"
click node_optimize_thresholds "https://github.com/anish-dev09/ics-networks/blob/main/optimize_thresholds.py"
click node_train_zero_day_detector "https://github.com/anish-dev09/ics-networks/blob/main/train_zero_day_detector.py"
click node_results_models "https://github.com/anish-dev09/ics-networks/tree/main/results/models"
click node_results_metrics "https://github.com/anish-dev09/ics-networks/tree/main/results/metrics"
click node_streamlit_app "https://github.com/anish-dev09/ics-networks/blob/main/streamlit_app.py"
click node_demo_app "https://github.com/anish-dev09/ics-networks/blob/main/demo/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_data_raw_hai,node_data_loader,node_hai_loader,node_sequence_generator,node_cnn_sequences,node_streamlit_app,node_demo_app toneBlue
class node_feature_engineering,node_ics_feature_extractor,node_results_models,node_results_metrics toneAmber
class node_baseline_detector,node_ml_models,node_cnn_models,node_ensemble_detector,node_deep_anomaly_detector toneMint
class node_protocol_detector,node_modbus_parser,node_dnp3_parser,node_s7comm_parser toneRose
class node_integrated_detector,node_advanced_integrated_detector,node_temporal_analyzer,node_adaptive_threshold,node_online_learning_system toneIndigo
class node_train_ml_models,node_train_cnn_model,node_prepare_cnn_data,node_compare_models,node_optimize_thresholds,node_train_zero_day_detector toneTeal
```

## Components

### Data pipeline

Raw HAI ingestion and CNN-ready sequence preparation.

- [HAI raw data](https://github.com/anish-dev09/ics-networks/tree/main/data/raw/hai): dataset · Source HAI dataset files. · `data/raw/hai`
- [Data loader](https://github.com/anish-dev09/ics-networks/blob/main/src/data/data_loader.py): ingestion · General data loading utilities. · `src/data/data_loader.py`
- [HAI loader](https://github.com/anish-dev09/ics-networks/blob/main/src/data/hai_loader.py): ingestion · HAI-specific loading and slicing. · `src/data/hai_loader.py`
- [Sequence gen](https://github.com/anish-dev09/ics-networks/blob/main/src/data/sequence_generator.py): sequence prep · Sliding-window generation for CNN inputs. · `src/data/sequence_generator.py`
- [CNN sequences](https://github.com/anish-dev09/ics-networks/tree/main/data/processed/cnn_sequences): processed data · Prepared NumPy tensors and metadata for temporal models. · `data/processed/cnn_sequences`

### Feature engineering

Classical ML feature construction from sensor streams.

- [Feature engineering](https://github.com/anish-dev09/ics-networks/blob/main/src/features/feature_engineering.py): feature builder · Statistical, temporal, lag, and correlation features. · `src/features/feature_engineering.py`
- [ICS extractor](https://github.com/anish-dev09/ics-networks/blob/main/src/features/ics_feature_extractor.py): feature builder · ICS-specific feature extraction from sensor streams. · `src/features/ics_feature_extractor.py`

### Detectors

Baseline, ML, CNN, and ensemble detector implementations.

- [Baseline detector](https://github.com/anish-dev09/ics-networks/blob/main/src/models/baseline_detector.py): anomaly rules · Isolation Forest, Z-score, and IQR baselines. · `src/models/baseline_detector.py`
- [ML models](https://github.com/anish-dev09/ics-networks/blob/main/src/models/ml_models.py): classical ML · Random Forest and XGBoost detector wrappers. · `src/models/ml_models.py`
- [CNN models](https://github.com/anish-dev09/ics-networks/blob/main/src/models/cnn_models.py): deep learning · 1D-CNN architecture for sequence-based detection. · `src/models/cnn_models.py`
- [Ensemble detector](https://github.com/anish-dev09/ics-networks/blob/main/src/models/ensemble_detector.py): fusion · Combines multiple detector outputs. · `src/models/ensemble_detector.py`
- [Deep anomaly](https://github.com/anish-dev09/ics-networks/blob/main/src/models/deep_anomaly_detector.py): deep anomaly · Higher-level anomaly detection variant. · `src/models/deep_anomaly_detector.py`

### Protocol layer

Industrial protocol parsing and protocol-aware validation.

- [Protocol detector](https://github.com/anish-dev09/ics-networks/blob/main/src/protocols/protocol_detector.py): protocol routing · Identifies protocol-specific traffic handling. · `src/protocols/protocol_detector.py`
- [Modbus parser](https://github.com/anish-dev09/ics-networks/blob/main/src/protocols/modbus_parser.py): protocol parser · Parses Modbus traffic. · `src/protocols/modbus_parser.py`
- [DNP3 parser](https://github.com/anish-dev09/ics-networks/blob/main/src/protocols/dnp3_parser.py): protocol parser · Parses DNP3 traffic. · `src/protocols/dnp3_parser.py`
- [S7comm parser](https://github.com/anish-dev09/ics-networks/blob/main/src/protocols/s7comm_parser.py): protocol parser · Parses S7comm traffic. · `src/protocols/s7comm_parser.py`

### Orchestration

Integrated detection and runtime adaptation logic.

- [Integrated detector](https://github.com/anish-dev09/ics-networks/blob/main/src/detection/integrated_detector.py): orchestrator · Combines parsing, features, models, and scoring. · `src/detection/integrated_detector.py`
- [Advanced detector](https://github.com/anish-dev09/ics-networks/blob/main/src/detection/advanced_integrated_detector.py): orchestrator · Expanded coordinated detection pipeline. · `src/detection/advanced_integrated_detector.py`
- [Temporal analyzer](https://github.com/anish-dev09/ics-networks/blob/main/src/utils/temporal_analyzer.py): runtime context · Temporal context extraction and analysis. · `src/utils/temporal_analyzer.py`
- [Adaptive threshold](https://github.com/anish-dev09/ics-networks/blob/main/src/utils/adaptive_threshold.py): runtime tuning · Dynamic threshold adjustment. · `src/utils/adaptive_threshold.py`
- [Online learning](https://github.com/anish-dev09/ics-networks/blob/main/src/utils/online_learning_system.py): adaptation · Incremental adaptation and drift handling. · `src/utils/online_learning_system.py`

### Training and eval

Offline scripts that build models and compare results.

- [Train ML](https://github.com/anish-dev09/ics-networks/blob/main/train_ml_models.py): script · Offline training entry point for classical models. · `train_ml_models.py`
- [Train CNN](https://github.com/anish-dev09/ics-networks/blob/main/train_cnn_model.py): script · Offline training entry point for CNN models. · `train_cnn_model.py`
- [Prepare CNN](https://github.com/anish-dev09/ics-networks/blob/main/prepare_cnn_data.py): script · Builds CNN-ready processed arrays. · `prepare_cnn_data.py`
- [Compare models](https://github.com/anish-dev09/ics-networks/blob/main/compare_models.py): evaluation · Aggregates model comparison metrics. · `compare_models.py`
- [Optimize thresholds](https://github.com/anish-dev09/ics-networks/blob/main/optimize_thresholds.py): evaluation · Searches decision thresholds. · `optimize_thresholds.py`
- [Zero-day train](https://github.com/anish-dev09/ics-networks/blob/main/train_zero_day_detector.py): script · Trains specialized zero-day detector flow. · `train_zero_day_detector.py`

### Demo UI

Streamlit presentation layer and sample data.

- [Streamlit app](https://github.com/anish-dev09/ics-networks/blob/main/streamlit_app.py): ui · Primary demo interface. · `streamlit_app.py`
- [Demo app](https://github.com/anish-dev09/ics-networks/blob/main/demo/app.py): ui module · Secondary demo entry point. · `demo/app.py`

### Persisted artifacts

Saved models, metadata, and evaluation outputs.

- [Model artifacts](https://github.com/anish-dev09/ics-networks/tree/main/results/models): artifact store · Persisted trained models and metadata. · `results/models`
- [Metrics store](https://github.com/anish-dev09/ics-networks/tree/main/results/metrics): artifact store · Persisted evaluation summaries and comparisons. · `results/metrics`

## Connections

- HAI raw data → Data loader: load · Raw HAI data is read by generic loaders.
- Data loader → HAI loader: specialize · HAI-specific slicing builds on the general loader.
- HAI loader → Feature engineering: tabular path · Loaded sensor data feeds engineered features.
- HAI loader → Sequence gen: sequence path · Loaded sensor data feeds sliding windows.
- Sequence gen → CNN sequences: persist · Generated arrays are stored for CNN training.
- Feature engineering → ICS extractor: extend · ICS-specific extraction enriches engineered features.
- ICS extractor → ML models: features · Classical ML models consume engineered features.
- Feature engineering → ML models: features · Feature matrices drive classical detectors.
- Sequence gen → CNN models: sequences · Temporal windows feed the CNN architecture.
- Baseline detector → Ensemble detector: baseline input · Baseline scores can be fused into ensembles.
- ML models → Ensemble detector: model input · Classical model outputs contribute to fusion.
- CNN models → Deep anomaly: deep input · CNN outputs can be wrapped by deeper anomaly logic.
- Protocol detector → Modbus parser: route · Protocol-aware routing can select Modbus parsing.
- Protocol detector → DNP3 parser: route · Protocol-aware routing can select DNP3 parsing.
- Protocol detector → S7comm parser: route · Protocol-aware routing can select S7comm parsing.
- Modbus parser → Integrated detector: parse · Parsed protocol context informs integrated detection.
- DNP3 parser → Integrated detector: parse · Parsed protocol context informs integrated detection.
- S7comm parser → Integrated detector: parse · Parsed protocol context informs integrated detection.
- Temporal analyzer → Integrated detector: context · Temporal context supports scoring decisions.
- Adaptive threshold → Integrated detector: thresholds · Adaptive thresholds tune integrated scoring.
- Online learning → Advanced detector: adapt · Online learning supports advanced orchestration.
- Integrated detector → Advanced detector: compose · Advanced detector builds on the integrated detector.
- Prepare CNN → CNN sequences: generate · Preparation script creates the processed CNN arrays.
- Train ML → Model artifacts: save · Classical training writes model artifacts.
- Train CNN → Model artifacts: save · CNN training writes model artifacts.
- Compare models → Metrics store: write · Model comparison outputs metrics summaries.
- Optimize thresholds → Metrics store: write · Threshold search writes evaluation outputs.
- Zero-day train → Model artifacts: save · Zero-day training produces persisted detectors.
- Model artifacts → Streamlit app: load · The demo consumes saved models for visualization.
- Model artifacts → Demo app: load · The demo consumes saved models for visualization.
- Streamlit app → Demo app: share UI · Both entry points represent the presentation layer.