# zaffar0561/ids architecture

> Architecture diagram of the GitHub repository zaffar0561/ids, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zaffar0561/ids
- Repository: https://github.com/zaffar0561/ids

## Overview

1. This repository is a small, pipeline-style intrusion detection system for a home network. It is split into two major paths: an offline model-building path and an online monitoring path. The offline side prepares data and trains the classifier; the online side captures live packets, scores them, enriches suspicious activity, and surfaces results in a dashboard.

2. The offline ML subsystem lives in `src/dataset_prep.py` and `src/train_model.py`. `dataset_prep.py` prepares the NSL-KDD dataset, which is the training source mentioned in the README. `train_model.py` then trains the detection model and saves artifacts into `models/`. This is the only part of the repo that depends on historical labeled data rather than live traffic.

3. The inference artifacts are stored separately from code in `models/`. The presence of `rf_model.joblib` and `proto_encoder.joblib` indicates a serialized Random Forest classifier plus a preprocessing/encoding artifact used at runtime. That separation is important architecturally: the live detector depends on these saved artifacts rather than retraining in place.

4. Live packet collection is handled by `src/sniffer.py`. Its role is to observe network traffic on the wire and persist packet observations for downstream processing. In the architecture diagram, this is the ingress point for real-time data and sits before any ML scoring or threat lookup.

5. Real-time detection is implemented in `src/realtime_detect.py`. This module is the main online processing stage: it consumes the captured traffic, extracts features, loads the saved model artifacts, and classifies packets or flows as normal versus suspicious. It is the bridge between raw packet capture and security actions.

6. `src/threat_intel.py` is the enrichment and risk-assessment boundary. The README says it integrates with AbuseIPDB, so suspicious IPs can be checked against an external reputation service using an API key from the environment. This module likely adds context to alerts and helps decide whether an IP should be treated as high risk.

7. The system’s response layer is split between logging, alerting, and optional blocking. The README describes auto-blocking on Windows using firewall rules, so the runtime likely branches from “suspicious” detections into either passive alert generation or active remediation. This is the main control boundary in the design because it turns inference results into host-level actions.

8. `dashboard/app.py` is the user-facing observability layer. It is a Streamlit application that presents live traffic, alerts, and threat intelligence. Architecturally, it does not appear to be the detector itself; it reads the outputs generated by the capture/detection pipeline and visualizes them.

9. Persistence is file-based rather than service-based. The README references captured packets, alerts, and logs stored on disk, and the tree shows prebuilt model artifacts in `models/`. That means the main data flow is local filesystem-backed: capture writes data, detection reads it and writes alerts, and the dashboard reads those outputs.

10. The repo has a clear runtime boundary between offline preparation and online monitoring. Offline steps build the model and encoder once; online steps require network capture permissions, an optional AbuseIPDB API key, and dashboard access on port 8501. The architecture is therefore a single-machine, locally deployed IDS with one external service dependency and optional OS-specific firewall integration.

Main entry points: [`src/realtime_detect.py`](https://github.com/zaffar0561/ids/blob/main/src/realtime_detect.py) (Live Detect), [`src/threat_intel.py`](https://github.com/zaffar0561/ids/blob/main/src/threat_intel.py) (Threat Intel), [`models`](https://github.com/zaffar0561/ids/tree/main/models) (Model Store), [`dashboard/app.py`](https://github.com/zaffar0561/ids/blob/main/dashboard/app.py) (Dashboard)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_offline["Offline ML"]
  node_nslkdd[("NSL-KDD<br/>training data")]
  node_dataset_prep["Prep Data<br/>dataset prep<br/>[dataset_prep.py]"]
  node_train_model["Train Model<br/>model training<br/>[train_model.py]"]
end

subgraph group_online["Online IDS"]
  node_sniffer["Sniffer<br/>packet capture<br/>[sniffer.py]"]
  node_realtime_detect["Live Detect<br/>online inference<br/>[realtime_detect.py]"]
  node_threat_intel["Threat Intel<br/>enrichment<br/>[threat_intel.py]"]
  node_response{{"Response<br/>alert/remediation"}}
  node_firewall["Firewall Rule<br/>host blocking"]
end

subgraph group_storage["Local Storage"]
  node_model_store[("Model Store<br/>artifact store")]
  node_rf_model["RF Model<br/>classifier artifact<br/>[rf_model.joblib]"]
  node_proto_encoder["Proto Encoder<br/>preprocess artifact"]
  node_packet_log["Packet Log<br/>capture output"]
  node_alert_log["Alert Log<br/>detection output"]
end

subgraph group_ui["Dashboard"]
  node_dashboard_app["Dashboard<br/>streamlit ui<br/>[app.py]"]
end

subgraph group_external["External Service"]
  node_abuseipdb(("AbuseIPDB<br/>reputation api"))
end

node_nslkdd -->|"input"| node_dataset_prep
node_dataset_prep -->|"prepared data"| node_train_model
node_train_model -->|"exports"| node_model_store
node_model_store -->|"contains"| node_rf_model
node_model_store -->|"contains"| node_proto_encoder
node_sniffer -->|"writes"| node_packet_log
node_packet_log -->|"feeds"| node_realtime_detect
node_sniffer -->|"live stream"| node_realtime_detect
node_rf_model -->|"scores with"| node_realtime_detect
node_proto_encoder -->|"preprocesses"| node_realtime_detect
node_realtime_detect -->|"suspicious ips"| node_threat_intel
node_threat_intel -->|"lookup"| node_abuseipdb
node_realtime_detect -->|"detections"| node_response
node_threat_intel -->|"risk context"| node_response
node_response -->|"alerts"| node_alert_log
node_response -->|"block"| node_firewall
node_alert_log -->|"visualizes"| node_dashboard_app
node_packet_log -->|"visualizes"| node_dashboard_app
node_threat_intel -->|"enriches view"| node_dashboard_app

click node_dataset_prep "https://github.com/zaffar0561/ids/blob/main/src/dataset_prep.py"
click node_train_model "https://github.com/zaffar0561/ids/blob/main/src/train_model.py"
click node_model_store "https://github.com/zaffar0561/ids/tree/main/models"
click node_rf_model "https://github.com/zaffar0561/ids/blob/main/models/rf_model.joblib"
click node_proto_encoder "https://github.com/zaffar0561/ids/blob/main/models/proto_encoder.joblib"
click node_sniffer "https://github.com/zaffar0561/ids/blob/main/src/sniffer.py"
click node_realtime_detect "https://github.com/zaffar0561/ids/blob/main/src/realtime_detect.py"
click node_threat_intel "https://github.com/zaffar0561/ids/blob/main/src/threat_intel.py"
click node_dashboard_app "https://github.com/zaffar0561/ids/blob/main/dashboard/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_nslkdd,node_dataset_prep,node_train_model toneBlue
class node_sniffer,node_realtime_detect,node_threat_intel,node_response,node_firewall toneAmber
class node_model_store,node_rf_model,node_proto_encoder,node_packet_log,node_alert_log toneMint
class node_dashboard_app toneRose
class node_abuseipdb toneIndigo
```

## Components

### Offline ML

Historical dataset preparation and model training.

- **NSL-KDD**: training data · Historical labeled dataset used to build the detector.
- [Prep Data](https://github.com/zaffar0561/ids/blob/main/src/dataset_prep.py): dataset prep · Cleans and prepares the training dataset for model building. · `src/dataset_prep.py`
- [Train Model](https://github.com/zaffar0561/ids/blob/main/src/train_model.py): model training · Trains the classifier and exports runtime artifacts. · `src/train_model.py`

### Online IDS

Live packet capture, scoring, enrichment, and response.

- [Sniffer](https://github.com/zaffar0561/ids/blob/main/src/sniffer.py): packet capture · Captures live network traffic from the wire. · `src/sniffer.py`
- [Live Detect](https://github.com/zaffar0561/ids/blob/main/src/realtime_detect.py): online inference · Extracts features, loads artifacts, and scores traffic in real time. · `src/realtime_detect.py`
- [Threat Intel](https://github.com/zaffar0561/ids/blob/main/src/threat_intel.py): enrichment · Checks suspicious IPs against a reputation service and adds context. · `src/threat_intel.py`
- **Response**: alert/remediation · Branches detections into alerts, logging, and optional blocking.
- **Firewall Rule**: host blocking · Windows firewall enforcement for active blocking.

### Local Storage

Filesystem-backed artifacts and runtime outputs.

- [Model Store](https://github.com/zaffar0561/ids/tree/main/models): artifact store · Serialized inference artifacts consumed by the live detector. · `models`
- [RF Model](https://github.com/zaffar0561/ids/blob/main/models/rf_model.joblib): classifier artifact · Saved Random Forest classifier used for scoring. · `models/rf_model.joblib`
- [Proto Encoder](https://github.com/zaffar0561/ids/blob/main/models/proto_encoder.joblib): preprocess artifact · Saved encoding/preprocessing artifact used at runtime. · `models/proto_encoder.joblib`
- **Packet Log**: capture output · Persisted packet observations written for downstream processing.
- **Alert Log**: detection output · Persisted suspicious-event and alert records.

### Dashboard

Human-facing monitoring interface.

- [Dashboard](https://github.com/zaffar0561/ids/blob/main/dashboard/app.py): streamlit ui · Visualizes live traffic, alerts, and threat intel. · `dashboard/app.py`

### External Service

Third-party reputation lookup used during enrichment.

- **AbuseIPDB**: reputation api · External IP reputation source used during enrichment.

## Connections

- NSL-KDD → Prep Data: input · Training data is prepared for feature extraction and modeling.
- Prep Data → Train Model: prepared data · Prepared dataset feeds model training.
- Train Model → Model Store: exports · Training saves artifacts for live inference.
- Model Store → RF Model: contains · Serialized classifier artifact stored on disk.
- Model Store → Proto Encoder: contains · Serialized encoder/preprocessor stored on disk.
- Sniffer → Packet Log: writes · Captured traffic is persisted locally.
- Packet Log → Live Detect: feeds · Recorded observations are consumed by the detector.
- Sniffer → Live Detect: live stream · Raw packets flow into real-time scoring.
- RF Model → Live Detect: scores with · Classifier artifact is loaded during inference.
- Proto Encoder → Live Detect: preprocesses · Encoding artifact is loaded during inference.
- Live Detect → Threat Intel: suspicious ips · Suspicious detections are enriched with reputation data.
- Threat Intel → AbuseIPDB: lookup · Reputation checks are sent to the external service.
- Live Detect → Response: detections · Inference results flow into the response boundary.
- Threat Intel → Response: risk context · Enrichment informs alerting and blocking decisions.
- Response → Alert Log: alerts · Detected incidents are written for later review.
- Response → Firewall Rule: block · Optional active remediation applies host firewall rules.
- Alert Log → Dashboard: visualizes · Dashboard reads alerts and incident history.
- Packet Log → Dashboard: visualizes · Dashboard displays captured traffic summaries.
- Threat Intel → Dashboard: enriches view · Threat intelligence is surfaced in the UI.