How anandprafull/fedsecureai works
1. Overall shape
- FedSecureAI is a federated intrusion-detection system with three training clients—Army, Navy, Air Force—and a central Flower-based aggregator.
- The repo is split into three main boundaries: a Python FL backend, a training dashboard (dashboard/), and a SOC/security console (soc-dashboard/).
2. Data ingestion and preprocessing
- The raw input is the CIC-IDS2017 CSV dataset placed under data/raw/.
- src/data_processor.py turns those raw files into partitioned, non-IID client datasets stored in data/partitions/, which is the handoff point from offline preprocessing into federated training.
3. Federated training runtime
- src/server.py runs the central federated server.
- src/client.py runs the local training nodes for each compartmented domain.
- The clients train locally on their partition and exchange model updates with the server; the server aggregates those updates into a global model.
4. Model and training logic
- src/model.py defines the PyTorch intrusion-detection network used by both server-side evaluation and client-side training.
- src/strategy.py contains the aggregation policy, including the repo’s main defense mechanism: coordinate-wise median aggregation.
- The README also indicates FedProx is used to stabilize training under strong non-IID splits.
5. Adversarial and security simulation
- src/attack_sim.py models a gradient-flipping / red-teaming attack so the FL loop can be tested under malicious client behavior.
- This is part of the security boundary around training rather than a separate product feature: it exists to stress the aggregation logic and produce security telemetry.
6. Persistence and audit artifacts
- Training outputs are persisted to logs/ and saved_models/.
- Key artifacts include the latest global model, per-node local checkpoints, and training logs; these are the main data source for both dashboards and offline evaluation.
- This persistence layer is important because the UI components appear to consume files rather than live model objects.
7. Explainability pipeline
- src/explainability.py generates SHAP-based feature attribution for the trained global model.
- The README says the generated outputs are written back into logs/ as images and metadata, making explainability an offline artifact-generation step that the dashboard can trigger and display.
8. Training dashboard
- dashboard/ is a separate Next.js app for monitoring the FL training lifecycle.
- Its API routes suggest three functions: serving dashboard data, exposing artifacts by name, and triggering explainability generation.
- Conceptually, it is a read/trigger layer over the persisted training and explainability artifacts, not the training engine itself.
9. SOC dashboard
- soc-dashboard/ is a second Next.js app dedicated to security operations.
- It has routes for SOC event data and report generation, and a lib/soc-data.ts helper that likely centralizes event parsing/analysis.
- This console is the incident-response boundary: it focuses on attack events, drift, node risk, and report output rather than training metrics.
10. Evaluation and validation
- src/evaluate_saved_models.py compares the persisted global checkpoint against the three local checkpoints using held-out partition data and reports classification metrics.
- Tests in tests/ cover the data processor and aggregation strategy, which are the two most architecture-sensitive Python subsystems.
11. Configuration and deployment
- config.yaml centralizes runtime settings for the server, training, FedProx, aggregation, saved-model paths, and attack parameters.
- Deployment is containerized with separate Dockerfiles for client, server, and the two dashboards, orchestrated by docker-compose.yml.
- This reinforces the repo’s main architectural separation: backend FL runtime plus two independent frontend consoles.
12. Primary end-to-end flow
- Raw dataset → preprocessing/partitioning → local client training → server aggregation → persisted checkpoints/logs → dashboard and SOC visualization/reporting.
- The only major cross-cutting dependencies are the shared filesystem artifacts in logs/ and saved_models/, which act as the integration contract between training, explainability, evaluation, and the UI layers.
Main entry points: server.py Server · client.py Client · logs Logs · api Dashboard API
Components 17
Python FL backend
Federated learning runtime, model logic, security simulation, and offline evaluation.
- Data prep · Transforms raw CSV input into partitioned non-IID client datasets.
src/data_processor.py - Server · Central federated server that coordinates rounds and aggregates client updates.
src/server.py - Client · Local training node used by each compartmented client domain.
src/client.py - Model · Intrusion-detection network shared by client training and server-side evaluation.
src/model.py - Aggregation · Aggregation policy centered on coordinate-wise median with FedProx stabilization.
src/strategy.py - Attack sim · Red-team simulation of malicious client updates to stress the federation.
src/attack_sim.py - Explainability · Generates feature attribution artifacts for the trained global model.
src/explainability.py - Evaluation · Compares persisted global and local checkpoints against held-out partition data.
src/evaluate_saved_models.py
Training dashboard
Next.js app for monitoring training and explainability artifacts.
- Dashboard API · Serves dashboard data, artifact access, and explainability triggers.
dashboard/app/api - Training UI · Operator console for monitoring federated training and artifacts.
dashboard/app/page.tsx
SOC dashboard
Next.js app for security operations, events, and reports.
Shared artifacts
Filesystem boundary for datasets, checkpoints, logs, and generated outputs.
- Raw data · Source intrusion dataset placed under data/raw/ for offline preprocessing.
data/raw - Partitions · Partitioned training datasets for the federated clients.
data/partitions - Logs · Training metrics, explainability outputs, and audit artifacts consumed by UIs and evaluation.
logs - Saved models · Persisted global and local model checkpoints used by dashboards and evaluation.
saved_models
Connections 23
- Raw data → Data prep: ingest · Offline preprocessing reads raw CIC-IDS2017 CSV files.
- Data prep → Partitions: write · Produces partitioned non-IID client datasets.
- Partitions → Client: train on · Each client trains locally on its assigned partition.
- Client → Server: update · Clients send local model updates to the federated server.
- Server → Aggregation: aggregate with · Server applies the aggregation policy each round.
- Aggregation → Model: combine · Aggregation operates over model updates and weights.
- Client → Model: fit · Clients train the shared PyTorch intrusion-detection network.
- Server → Model: evaluate · Server-side evaluation uses the shared model definition.
- Attack sim → Client: poison · Simulated malicious behavior targets client updates during training.
- Attack sim → Aggregation: stress · Attack simulation is used to test robustness of aggregation.
- Server → Logs: emit · Training metrics and security telemetry are persisted as artifacts.
- Server → Saved models: save · Global checkpoints and selected local checkpoints are persisted.
- Explainability → Logs: write · SHAP outputs and metadata are stored for later display.
- Saved models → Evaluation: load · Evaluation consumes persisted checkpoints.
- Partitions → Evaluation: test on · Held-out partition data is used for checkpoint comparison.
- Logs → Dashboard API: serve · Dashboard reads metrics, artifacts, and explainability outputs from files.
- Saved models → Dashboard API: serve · Dashboard exposes model artifacts by name.
- Dashboard API → Training UI: feed · API routes back the training dashboard interface.
- Explainability → Dashboard API: trigger · Dashboard can initiate explainability generation.
- Logs → SOC API: serve · SOC console reads events, reports, and security artifacts from persisted outputs.
- SOC data → SOC API: support · SOC helper centralizes parsing and analysis for API responses.
- SOC API → SOC UI: feed · API routes back the SOC console interface.
- Server → SOC API: emit telemetry · Training and attack telemetry can surface in the SOC boundary.