How r-v-abhishek/network-intrusion-detection-nosql works
What this project is (type + purpose)
This is a containerized, real-time Network Intrusion Detection System (NIDS) implemented in Python, using:
• Redis for real-time event streaming/caching
• Apache Cassandra for persistent storage (network events + alerts)
• Docker / Docker Compose for deployment orchestration
• A lightweight Dashboard API layer (the dashboard/ package) to expose alerts/analytics
Top-level “system” building blocks
• docker-compose.yml / Dockerfile: defines deployable services/containers (at minimum: Python app container, Redis, Cassandra).
• src/main.py: the primary application entrypoint—likely starts the pipeline and/or API server.
• streaming/: the core streaming/inference runtime:
• pipeline_runner.py — orchestrates the end-to-end pipeline
• binary_inference.py — binary classification (intrusion vs benign)
• multiclass_inference.py — attack-type classification
• dashboard/: API surfaces for consumers:
• api_alerts.py — alert querying/serving
• api_analytics.py — aggregate analytics endpoints
• config/: centralized configuration (config.py)—important because it defines how services connect (Redis/Cassandra hosts, ports, keyspaces, stream names, etc.).
• data/UNSW-NB15/sample.csv, unsw_feature_names.json, unsw_model_training.ipynb: offline dataset/training artifacts; these should appear as “offline training” or “reference/training” components (separate from realtime path).
Tests indicate intended seams
• tests/test_pipeline.py implies the pipeline is a distinct unit (good component boundary).
• tests/test_dashboard_api.py implies dashboard APIs are separable components.
• tests/test_config.py implies config is a foundational dependency across components.
Component / Module (Python packages and runtime flow inside the app container)
Optionally add:
This repo strongly benefits from separating runtime architecture (Redis/Cassandra/streaming) from delivery architecture (Jenkins/CI compose).
Important to capture explicitly
• Redis is not just a cache here; per README it’s real-time event streaming. Show it as a message broker / stream with directional flow.
• Cassandra is system of record. Show it as durable persistence and the source for analytics queries.
• src/main.py (Entrypoint / Service bootstrap)
• config/config.py (loads settings)
• streaming/pipeline_runner.py (starts pipeline loop)
• dashboard API modules (register routes / run server) if applicable
• streaming/pipeline_runner.py
• dashboard/api_alerts.py, dashboard/api_analytics.py
• nids-app (Python, built by Dockerfile)
• redis
• cassandra
• Network links between containers (same Docker network)
• Environment/config injection (env vars / config file) into nids-app
• Exposed ports:
• Dashboard API port (if it runs a web server)
• Redis/Cassandra internal ports (usually not exposed publicly in prod—show as internal unless the compose exposes them)
If compose defines volumes (common for Cassandra), show Cassandra data volume for persistence.
Given this repo, it’s better to split more than less. At minimum split into:
• Pipeline runner
• Binary inference
• Multiclass inference
• Redis (stream)
• Cassandra (persistence)
• Dashboard Alerts API
• Dashboard Analytics API
• Config module
• Offline training assets
• CI pipeline components
1. System Context: Users/SOC → Dashboard API; Telemetry Source → Detection System; external dependencies Redis/Cassandra.