# rajal-ui/froststream-nids architecture

> Architecture diagram of the GitHub repository rajal-ui/froststream-nids, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rajal-ui/froststream-nids
- Repository: https://github.com/rajal-ui/froststream-nids

## Overview

FrostStream NIDS ingests packets or existing flow records, aggregates them into NSL-KDD-compatible features, and routes flows through distinct LOCAL or CLOUD paths. Local scoring uses trained artifacts and stores alerts in SQLite; cloud flows pass through Firehose and Snowflake ingestion before Snowpark inference, drift monitoring, and alerting. Qualifying cloud alerts may trigger optional AWS NACL mitigation. The dashboard reports cloud or local operational data. Cloud stages and major capabilities follow the README; some implementation wiring is documented rather than visible in the sampled source.

Main entry points: [`src/common/execution_mode.py`](https://github.com/rajal-ui/froststream-nids/blob/master/src/common/execution_mode.py) (Mode and sinks), [`snowflake_setup/01_tables.sql`](https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/01_tables.sql) (Flow features), [`src/models/inference.py`](https://github.com/rajal-ui/froststream-nids/blob/master/src/models/inference.py) (Inference and scoring), [`ingestion/flow_processor.py`](https://github.com/rajal-ui/froststream-nids/blob/master/ingestion/flow_processor.py) (Flow aggregation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Flow ingestion"]
  node_extractor["Flow aggregation<br/>[flow_processor.py]"]
  node_mode["Mode and sinks<br/>[execution_mode.py]"]
  node_firehose["Firehose"]
end

subgraph group_local["Local detection"]
  node_training["Ensemble training<br/>[train.py]"]
  node_dataset["NSL-KDD data<br/>[dataset_loader.py]"]
  node_artifacts["Model artifacts<br/>[constants.py]"]
  node_localalerts[("Local alert store<br/>[alert_logger.py]")]
  node_localmodel["Local inference<br/>[inference.py]"]
end

subgraph group_cloud["Cloud detection"]
  node_s3["S3 flow landing"]
  node_snowpipe["Snowpipe loading<br/>[02_snowpipe.sql]"]
  node_raw[("Raw flow landing<br/>[01_tables.sql]")]
  node_features["Feature flattening"]
  node_featuretable[("Flow features<br/>[01_tables.sql]")]
  node_inference["Inference and scoring<br/>[inference.py]"]
  node_drift["Drift monitoring<br/>[drift_monitor.py]"]
  node_alerts[("NIDS alerts<br/>[01_tables.sql]")]
end

subgraph group_response["Operations and response"]
  node_dashboard["SOC dashboard<br/>[sis_dashboard.py]"]
  node_dispatch["SOAR dispatch"]
  node_mitigator["NACL mitigation<br/>[mitigator.py]"]
  node_cleanup["Rule cleanup<br/>[cleanup.py]"]
end

node_traffic(("Packets or flows"))
node_awsnacl["AWS Network ACL"]

node_traffic -->|"provides packets"| node_extractor
node_extractor -->|"routes flows"| node_mode
node_mode -->|"sends cloud flows"| node_firehose
node_mode -->|"scores local flows"| node_localmodel
node_mode -->|"writes local alerts"| node_localalerts
node_firehose -->|"delivers flows"| node_s3
node_s3 -->|"loads files"| node_snowpipe
node_snowpipe -->|"writes rows"| node_raw
node_raw -->|"reads rows"| node_features
node_features -->|"writes features"| node_featuretable
node_dataset -->|"supplies training data"| node_training
node_training -->|"saves models"| node_artifacts
node_featuretable -->|"scores pending flows"| node_inference
node_artifacts -->|"loads models"| node_inference
node_inference -->|"writes alerts"| node_alerts
node_inference -->|"marks processed"| node_featuretable
node_featuretable -->|"monitors features"| node_drift
node_alerts -->|"queries status"| node_dashboard
node_localalerts -.->|"local fallback"| node_dashboard
node_alerts -->|"selects qualifying alerts"| node_dispatch
node_dispatch -.->|"invokes mitigation"| node_mitigator
node_mitigator -->|"creates deny rule"| node_awsnacl
node_awsnacl -.->|"expires rules"| node_cleanup

click node_extractor "https://github.com/rajal-ui/froststream-nids/blob/master/ingestion/flow_processor.py"
click node_mode "https://github.com/rajal-ui/froststream-nids/blob/master/src/common/execution_mode.py"
click node_snowpipe "https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/02_snowpipe.sql"
click node_raw "https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/01_tables.sql"
click node_features "https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/03_snowpark_integration.sql"
click node_featuretable "https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/01_tables.sql"
click node_training "https://github.com/rajal-ui/froststream-nids/blob/master/src/models/train.py"
click node_dataset "https://github.com/rajal-ui/froststream-nids/blob/master/data/dataset_loader.py"
click node_artifacts "https://github.com/rajal-ui/froststream-nids/blob/master/src/models/constants.py"
click node_inference "https://github.com/rajal-ui/froststream-nids/blob/master/src/models/inference.py"
click node_drift "https://github.com/rajal-ui/froststream-nids/blob/master/src/models/drift_monitor.py"
click node_alerts "https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/01_tables.sql"
click node_localalerts "https://github.com/rajal-ui/froststream-nids/blob/master/storage/alert_logger.py"
click node_localmodel "https://github.com/rajal-ui/froststream-nids/blob/master/src/models/inference.py"
click node_dashboard "https://github.com/rajal-ui/froststream-nids/blob/master/streamlit_app/sis_dashboard.py"
click node_dispatch "https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/03_snowpark_integration.sql"
click node_mitigator "https://github.com/rajal-ui/froststream-nids/blob/master/src/soar/mitigator.py"
click node_cleanup "https://github.com/rajal-ui/froststream-nids/blob/master/src/soar/cleanup.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_extractor,node_mode,node_firehose toneBlue
class node_training,node_dataset,node_artifacts,node_localalerts,node_localmodel toneAmber
class node_s3,node_snowpipe,node_raw,node_features,node_featuretable,node_inference,node_drift,node_alerts toneMint
class node_dashboard,node_dispatch,node_mitigator,node_cleanup toneRose
class node_traffic,node_awsnacl toneIndigo
```

## Components

### Flow ingestion

- [Flow aggregation](https://github.com/rajal-ui/froststream-nids/blob/master/ingestion/flow_processor.py): `ingestion/flow_processor.py`
- [Mode and sinks](https://github.com/rajal-ui/froststream-nids/blob/master/src/common/execution_mode.py): `src/common/execution_mode.py`
- **Firehose**

### Local detection

- [Ensemble training](https://github.com/rajal-ui/froststream-nids/blob/master/src/models/train.py): `src/models/train.py`
- [NSL-KDD data](https://github.com/rajal-ui/froststream-nids/blob/master/data/dataset_loader.py): `data/dataset_loader.py`
- [Model artifacts](https://github.com/rajal-ui/froststream-nids/blob/master/src/models/constants.py): `src/models/constants.py`
- [Local alert store](https://github.com/rajal-ui/froststream-nids/blob/master/storage/alert_logger.py): `storage/alert_logger.py`
- [Local inference](https://github.com/rajal-ui/froststream-nids/blob/master/src/models/inference.py): `src/models/inference.py`

### Cloud detection

- **S3 flow landing**
- [Snowpipe loading](https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/02_snowpipe.sql): `snowflake_setup/02_snowpipe.sql`
- [Raw flow landing](https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/01_tables.sql): `snowflake_setup/01_tables.sql`
- [Feature flattening](https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/03_snowpark_integration.sql): `snowflake_setup/03_snowpark_integration.sql`
- [Flow features](https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/01_tables.sql): `snowflake_setup/01_tables.sql`
- [Inference and scoring](https://github.com/rajal-ui/froststream-nids/blob/master/src/models/inference.py): `src/models/inference.py`
- [Drift monitoring](https://github.com/rajal-ui/froststream-nids/blob/master/src/models/drift_monitor.py): `src/models/drift_monitor.py`
- [NIDS alerts](https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/01_tables.sql): `snowflake_setup/01_tables.sql`

### Operations and response

- [SOC dashboard](https://github.com/rajal-ui/froststream-nids/blob/master/streamlit_app/sis_dashboard.py): `streamlit_app/sis_dashboard.py`
- [SOAR dispatch](https://github.com/rajal-ui/froststream-nids/blob/master/snowflake_setup/03_snowpark_integration.sql): `snowflake_setup/03_snowpark_integration.sql`
- [NACL mitigation](https://github.com/rajal-ui/froststream-nids/blob/master/src/soar/mitigator.py): `src/soar/mitigator.py`
- [Rule cleanup](https://github.com/rajal-ui/froststream-nids/blob/master/src/soar/cleanup.py): `src/soar/cleanup.py`

### Other components

- **Packets or flows**
- **AWS Network ACL**

## Connections

- Packets or flows → Flow aggregation: provides packets
- Flow aggregation → Mode and sinks: routes flows
- Mode and sinks → Firehose: sends cloud flows
- Mode and sinks → Local inference: scores local flows
- Mode and sinks → Local alert store: writes local alerts
- Firehose → S3 flow landing: delivers flows
- S3 flow landing → Snowpipe loading: loads files
- Snowpipe loading → Raw flow landing: writes rows
- Raw flow landing → Feature flattening: reads rows
- Feature flattening → Flow features: writes features
- NSL-KDD data → Ensemble training: supplies training data
- Ensemble training → Model artifacts: saves models
- Flow features → Inference and scoring: scores pending flows
- Model artifacts → Inference and scoring: loads models
- Inference and scoring → NIDS alerts: writes alerts
- Inference and scoring → Flow features: marks processed
- Flow features → Drift monitoring: monitors features
- NIDS alerts → SOC dashboard: queries status
- Local alert store → SOC dashboard: local fallback
- NIDS alerts → SOAR dispatch: selects qualifying alerts
- SOAR dispatch → NACL mitigation: invokes mitigation
- NACL mitigation → AWS Network ACL: creates deny rule
- AWS Network ACL → Rule cleanup: expires rules