thakur-sahab-01/air-gapped-noc-copilot

1. System boundary

This repo is an offline predictive NOC copilot for MPLS links. The architecture is split into an offline build/train phase and an air-gapped inference phase. At runtime, the system is designed to operate with zero outbound network calls.

2. Top-level orchestration

setup_pipeline.py is the bootstrap entry point. It checks whether trained artifacts already exist, otherwise it runs the generation/training pipeline and then launches the dashboard. The repo also supports running each stage manually.

3. Synthetic telemetry source

src/synthetic_data.py generates the operational dataset in data/telemetry.csv. It simulates multiple links over time with realistic daily patterns and failure signatures, including latency, jitter, loss, utilisation, interface errors, BGP/LSP flaps, CPU, and temperature.

4. Feature engineering layer

src/features.py turns raw telemetry into rolling-window features used by the models. This is the main transformation boundary between time-series telemetry and supervised learning inputs. The README indicates the features are built per link over historical windows.

5. Model training and artifacts

src/train_model.py trains two XGBoost classifiers for different prediction horizons: 30 minutes and 240 minutes ahead. The resulting artifacts are saved under models/ as model_target_30min.joblib, model_target_240min.joblib, plus feature_columns.json and metrics.json.

6. Generalisation strategy

The models are evaluated on unseen links, not just later timestamps, using a group-based split by link_id. That means the architecture is intentionally validating whether the predictors generalize across infrastructure instances, not only across time.

7. Live simulation / replay

src/simulate.py and the dashboard together provide the runtime telemetry stream. The dashboard can advance a simulated clock and inject failures, which feeds the prediction pipeline with fresh link state as if it were a live NOC feed.

8. Prediction + explainability

src/explain.py provides SHAP-based attribution for each prediction. It maps raw feature contributions into one of five root-cause categories such as congestion, interface degradation, BGP instability, LSP flapping, or hardware/control-plane fault, and produces a confidence score for that diagnosis.

9. Operator narrative generation

src/copilot.py converts the structured diagnosis into a human-readable NOC briefing. It has two backends: a local template-based narrator and an optional Ollama LLM backend. The template fallback is important because it preserves offline operation if no local model is available.

10. Offline knowledge base

src/knowledge_base.py implements retrieval over the runbook corpus in data/runbooks/ using TF-IDF. This gives the copilot grounded operational guidance without any external search or model download dependency.

11. Dashboard as the integration point

dashboard.py is the user-facing integration layer. It ties together the simulated telemetry, risk ranking, SHAP visualization, copilot explanation, runbook retrieval, and operator feedback controls into a single Streamlit NOC console.

12. Data and model flow

The end-to-end flow is: synthetic telemetry → feature windows → trained classifiers → risk scores → SHAP explanation → root-cause mapping → narrative/runbook guidance → operator action. The main architectural boundary is that all of these steps run locally from checked-in code plus generated artifacts in data/ and models/.

13. External dependency boundary

The only optional external runtime dependency is Ollama, used locally for richer narrative generation. If it is unavailable, the system falls back automatically, so the core architecture remains fully functional in an isolated environment.