# thakur-sahab-01/air-gapped-noc-copilot architecture
> Architecture diagram of the GitHub repository thakur-sahab-01/air-gapped-noc-copilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-23). It is an AI-made overview: check details against the source.
- Interactive diagram: https://gitdiagram.com/thakur-sahab-01/air-gapped-noc-copilot
- Repository: https://github.com/thakur-sahab-01/air-gapped-noc-copilot
## Overview
1. **System boundary**
This repo is an offline predictive NOC copilot for MPLS links. The architecture is split into an **offline build/train phase** and an **air-gapped inference phase**. At runtime, the system is designed to operate with **zero outbound network calls**.
2. **Top-level orchestration**
`setup_pipeline.py` is the bootstrap entry point. It checks whether trained artifacts already exist, otherwise it runs the generation/training pipeline and then launches the dashboard. The repo also supports running each stage manually.
3. **Synthetic telemetry source**
`src/synthetic_data.py` generates the operational dataset in `data/telemetry.csv`. It simulates multiple links over time with realistic daily patterns and failure signatures, including latency, jitter, loss, utilisation, interface errors, BGP/LSP flaps, CPU, and temperature.
4. **Feature engineering layer**
`src/features.py` turns raw telemetry into rolling-window features used by the models. This is the main transformation boundary between time-series telemetry and supervised learning inputs. The README indicates the features are built per link over historical windows.
5. **Model training and artifacts**
`src/train_model.py` trains two XGBoost classifiers for different prediction horizons: **30 minutes** and **240 minutes** ahead. The resulting artifacts are saved under `models/` as `model_target_30min.joblib`, `model_target_240min.joblib`, plus `feature_columns.json` and `metrics.json`.
6. **Generalisation strategy**
The models are evaluated on **unseen links**, not just later timestamps, using a group-based split by `link_id`. That means the architecture is intentionally validating whether the predictors generalize across infrastructure instances, not only across time.
7. **Live simulation / replay**
`src/simulate.py` and the dashboard together provide the runtime telemetry stream. The dashboard can advance a simulated clock and inject failures, which feeds the prediction pipeline with fresh link state as if it were a live NOC feed.
8. **Prediction + explainability**
`src/explain.py` provides SHAP-based attribution for each prediction. It maps raw feature contributions into one of five root-cause categories such as congestion, interface degradation, BGP instability, LSP flapping, or hardware/control-plane fault, and produces a confidence score for that diagnosis.
9. **Operator narrative generation**
`src/copilot.py` converts the structured diagnosis into a human-readable NOC briefing. It has two backends: a local template-based narrator and an optional Ollama LLM backend. The template fallback is important because it preserves offline operation if no local model is available.
10. **Offline knowledge base**
`src/knowledge_base.py` implements retrieval over the runbook corpus in `data/runbooks/` using TF-IDF. This gives the copilot grounded operational guidance without any external search or model download dependency.
11. **Dashboard as the integration point**
`dashboard.py` is the user-facing integration layer. It ties together the simulated telemetry, risk ranking, SHAP visualization, copilot explanation, runbook retrieval, and operator feedback controls into a single Streamlit NOC console.
12. **Data and model flow**
The end-to-end flow is: **synthetic telemetry → feature windows → trained classifiers → risk scores → SHAP explanation → root-cause mapping → narrative/runbook guidance → operator action**. The main architectural boundary is that all of these steps run locally from checked-in code plus generated artifacts in `data/` and `models/`.
13. **External dependency boundary**
The only optional external runtime dependency is **Ollama**, used locally for richer narrative generation. If it is unavailable, the system falls back automatically, so the core architecture remains fully functional in an isolated environment.
Main entry points: [`dashboard.py`](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/dashboard.py) (NOC dashboard), [`src/knowledge_base.py`](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/knowledge_base.py) (Runbook retrieval), [`src/train_model.py`](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/train_model.py) (Train model), [`src/features.py`](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/features.py) (Feature windows)
## Diagram (Mermaid)
```mermaid
flowchart TD
subgraph group_group_build["Offline build/train"]
node_setup_pipeline["Setup pipeline
bootstrap
[setup_pipeline.py]"]
node_synthetic_data["Synthetic telemetry
data generator
[synthetic_data.py]"]
node_telemetry_csv["Telemetry CSV
dataset artifact"]
node_features["Feature windows
feature engineering
[features.py]"]
node_train_model["Train model
model training
[train_model.py]"]
node_model_30[("30 min model
xgboost artifact")]
node_model_240[("240 min model
xgboost artifact")]
node_feature_columns["Feature schema
artifact"]
node_metrics["Training metrics
artifact
[metrics.json]"]
end
subgraph group_group_runtime["Air-gapped runtime"]
node_simulate(("Telemetry replay
runtime simulator
[simulate.py]"))
node_dashboard["NOC dashboard
streamlit console
[dashboard.py]"]
node_explain{{"Explain risk
shap attribution
[explain.py]"}}
node_copilot["Copilot briefing
narration
[copilot.py]"]
end
subgraph group_group_knowledge["Local knowledge"]
node_knowledge_base["Runbook retrieval
tf-idf search
[knowledge_base.py]"]
node_runbook_bgp["BGP runbook"]
node_runbook_congestion["Congestion runbook
[congestion.txt]"]
node_runbook_hw["Hardware runbook
[hardware_fault.txt]"]
node_runbook_if["Interface runbook"]
node_runbook_lsp["LSP runbook
[lsp_flapping.txt]"]
end
node_setup_pipeline -->|"generate data"| node_synthetic_data
node_synthetic_data -->|"write dataset"| node_telemetry_csv
node_telemetry_csv -->|"window features"| node_features
node_features -->|"train on"| node_train_model
node_train_model -->|"save"| node_model_30
node_train_model -->|"save"| node_model_240
node_train_model -->|"save"| node_feature_columns
node_train_model -->|"save"| node_metrics
node_setup_pipeline -->|"launch"| node_dashboard
node_simulate -->|"feed telemetry"| node_dashboard
node_dashboard -->|"infer features"| node_features
node_dashboard -->|"score 30m"| node_model_30
node_dashboard -->|"score 240m"| node_model_240
node_dashboard -->|"explain prediction"| node_explain
node_explain -->|"diagnosis"| node_copilot
node_copilot -->|"ground guidance"| node_knowledge_base
node_knowledge_base -->|"index"| node_runbook_bgp
node_knowledge_base -->|"index"| node_runbook_congestion
node_knowledge_base -->|"index"| node_runbook_hw
node_knowledge_base -->|"index"| node_runbook_if
node_knowledge_base -->|"index"| node_runbook_lsp
node_dashboard -->|"brief operator"| node_copilot
click node_setup_pipeline "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/setup_pipeline.py"
click node_synthetic_data "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/synthetic_data.py"
click node_features "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/features.py"
click node_train_model "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/train_model.py"
click node_model_30 "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/models/model_target_30min.joblib"
click node_model_240 "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/models/model_target_240min.joblib"
click node_feature_columns "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/models/feature_columns.json"
click node_metrics "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/models/metrics.json"
click node_simulate "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/simulate.py"
click node_dashboard "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/dashboard.py"
click node_explain "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/explain.py"
click node_copilot "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/copilot.py"
click node_knowledge_base "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/knowledge_base.py"
click node_runbook_bgp "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/bgp_instability.txt"
click node_runbook_congestion "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/congestion.txt"
click node_runbook_hw "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/hardware_fault.txt"
click node_runbook_if "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/interface_degradation.txt"
click node_runbook_lsp "https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/lsp_flapping.txt"
classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_setup_pipeline,node_synthetic_data,node_telemetry_csv,node_features,node_train_model,node_model_30,node_model_240,node_feature_columns,node_metrics toneBlue
class node_simulate,node_dashboard,node_explain,node_copilot toneAmber
class node_knowledge_base,node_runbook_bgp,node_runbook_congestion,node_runbook_hw,node_runbook_if,node_runbook_lsp toneMint
```
## Components
### Offline build/train
Synthetic data generation, feature engineering, and model training that produce local artifacts.
- [Setup pipeline](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/setup_pipeline.py): bootstrap · Checks for existing artifacts, runs training if needed, then launches the dashboard. · `setup_pipeline.py`
- [Synthetic telemetry](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/synthetic_data.py): data generator · Generates multi-link telemetry with daily patterns and fault signatures. · `src/synthetic_data.py`
- **Telemetry CSV**: dataset artifact · Generated operational telemetry used for feature building and training.
- [Feature windows](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/features.py): feature engineering · Builds rolling per-link features from historical telemetry. · `src/features.py`
- [Train model](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/train_model.py): model training · Trains horizon-specific XGBoost classifiers and writes model artifacts. · `src/train_model.py`
- [30 min model](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/models/model_target_30min.joblib): xgboost artifact · Classifier for near-term prediction horizon. · `models/model_target_30min.joblib`
- [240 min model](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/models/model_target_240min.joblib): xgboost artifact · Classifier for longer prediction horizon. · `models/model_target_240min.joblib`
- [Feature schema](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/models/feature_columns.json): artifact · Saved feature column list used to align inference inputs. · `models/feature_columns.json`
- [Training metrics](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/models/metrics.json): artifact · Saved evaluation results for the trained models. · `models/metrics.json`
### Air-gapped runtime
Simulation, prediction, explanation, and operator-facing copilot flow with no outbound calls.
- [Telemetry replay](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/simulate.py): runtime simulator · Advances simulated time and injects failures for live-like operation. · `src/simulate.py`
- [NOC dashboard](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/dashboard.py): streamlit console · Main operator interface combining telemetry, risk ranking, explanations, and controls. · `dashboard.py`
- [Explain risk](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/explain.py): shap attribution · Maps model outputs to root-cause categories with confidence scoring. · `src/explain.py`
- [Copilot briefing](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/copilot.py): narration · Turns structured diagnosis into a human-readable NOC briefing. · `src/copilot.py`
### Local knowledge
Runbook corpus and retrieval used to ground operator guidance offline.
- [Runbook retrieval](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/src/knowledge_base.py): tf-idf search · Retrieves grounded guidance from local runbooks using offline text search. · `src/knowledge_base.py`
- [BGP runbook](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/bgp_instability.txt): runbook · Guidance for instability and control-plane issues. · `data/runbooks/bgp_instability.txt`
- [Congestion runbook](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/congestion.txt): runbook · Guidance for sustained utilization and latency issues. · `data/runbooks/congestion.txt`
- [Hardware runbook](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/hardware_fault.txt): runbook · Guidance for device faults and environmental problems. · `data/runbooks/hardware_fault.txt`
- [Interface runbook](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/interface_degradation.txt): runbook · Guidance for interface degradation and errors. · `data/runbooks/interface_degradation.txt`
- [LSP runbook](https://github.com/thakur-sahab-01/air-gapped-noc-copilot/blob/master/data/runbooks/lsp_flapping.txt): runbook · Guidance for path flapping and MPLS instability. · `data/runbooks/lsp_flapping.txt`
## Connections
- Setup pipeline → Synthetic telemetry: generate data · Creates telemetry when artifacts are missing.
- Synthetic telemetry → Telemetry CSV: write dataset · Persists the simulated telemetry dataset.
- Telemetry CSV → Feature windows: window features · Transforms raw telemetry into supervised learning inputs.
- Feature windows → Train model: train on · Feeds engineered windows into the training pipeline.
- Train model → 30 min model: save · Writes the 30-minute horizon classifier.
- Train model → 240 min model: save · Writes the 240-minute horizon classifier.
- Train model → Feature schema: save · Stores the feature schema used at inference time.
- Train model → Training metrics: save · Stores evaluation results and model metrics.
- Setup pipeline → NOC dashboard: launch · Starts the operator console after artifacts are ready.
- Telemetry replay → NOC dashboard: feed telemetry · Provides live-like link state to the console.
- NOC dashboard → Feature windows: infer features · Builds the same rolling inputs for scoring.
- NOC dashboard → 30 min model: score 30m · Runs the near-term risk classifier.
- NOC dashboard → 240 min model: score 240m · Runs the longer-horizon risk classifier.
- NOC dashboard → Explain risk: explain prediction · Generates SHAP-based root-cause attribution.
- Explain risk → Copilot briefing: diagnosis · Passes category and confidence into narration.
- Copilot briefing → Runbook retrieval: ground guidance · Retrieves relevant runbook content for the briefing.
- Runbook retrieval → BGP runbook: index · Uses BGP instability guidance.
- Runbook retrieval → Congestion runbook: index · Uses congestion guidance.
- Runbook retrieval → Hardware runbook: index · Uses hardware fault guidance.
- Runbook retrieval → Interface runbook: index · Uses interface degradation guidance.
- Runbook retrieval → LSP runbook: index · Uses LSP flapping guidance.
- NOC dashboard → Copilot briefing: brief operator · Renders the human-readable recommendation.