# abdull66-hash/detection-under-budgets architecture

> Architecture diagram of the GitHub repository abdull66-hash/detection-under-budgets, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abdull66-hash/detection-under-budgets
- Repository: https://github.com/abdull66-hash/detection-under-budgets

## Overview

This repository is a research project evaluating flow-sampling policies for network-attack monitoring. Its principal workflow loads and cleans a dataset, splits it into training, scorer, and test sets, trains attack models, applies sampling policies, and evaluates flow- and campaign-level detection; it also compares a second regime that samples training data. Baselines, diagnostic checks, regime analyses, summary tables, and report figures are supporting research branches. Source excerpts show the experiment orchestration and key helpers; edges for other modules rely only on the SOURCE INDEX, so unsampled behavior is not inferred.

Main entry points: [`p1_monitoring_budget/src/run_regime2.py`](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/run_regime2.py) (Sampled-train regime), [`p1_monitoring_budget/src/run_experiment.py`](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/run_experiment.py) (Full-train regime), [`p1_monitoring_budget/src/preprocess.py`](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/preprocess.py) (Clean splits), [`p1_monitoring_budget/src/baseline.py`](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/baseline.py) (Baseline runs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data_prep["Data preparation"]
  node_prepared_data["Clean splits<br/>[preprocess.py]"]
end

subgraph group_experiments["Experiments"]
  node_sampler["Sampling policies<br/>[sampling.py]"]
  node_campaigns["Campaign metrics<br/>[campaigns.py]"]
  node_models["Attack models<br/>[models.py]"]
  node_baseline["Baseline runs<br/>[baseline.py]"]
  node_regime_one["Full-train regime<br/>[run_experiment.py]"]
  node_regime_two["Sampled-train regime<br/>[run_regime2.py]"]
end

subgraph group_analysis["Analysis and reporting"]
  node_column_check["Column checks<br/>[column_check.py]"]
  node_host_spread["Host-keyed spread"]
  node_regime_analysis["Regime analysis<br/>[analyze_regimes.py]"]
  node_analysis["Result analysis<br/>[analyze.py]"]
  node_summary["Summary tables<br/>[summary_table.py]"]
  node_figures["Report figures<br/>[figures.py]"]
  node_result_files["Experiment results"]
end

subgraph group_shared["Shared helpers"]
  node_io["File helpers<br/>[io.py]"]
  node_metrics["Metric helpers<br/>[metrics.py]"]
  node_seed["Seed control<br/>[seed.py]"]
  node_plotting["Plot helpers<br/>[plotting.py]"]
end

node_researcher(("Researcher"))
node_dataset[("Flow dataset")]

node_researcher -->|"runs baseline"| node_baseline
node_researcher -->|"runs experiment"| node_regime_one
node_researcher -->|"runs regime"| node_regime_two
node_dataset -->|"loads and cleans"| node_prepared_data
node_prepared_data -->|"loads parquet"| node_io
node_baseline -->|"loads splits"| node_prepared_data
node_baseline -->|"builds models"| node_models
node_baseline -->|"scores predictions"| node_metrics
node_baseline -->|"sets seeds"| node_seed
node_regime_one -->|"loads splits"| node_prepared_data
node_regime_one -->|"builds masks"| node_sampler
node_regime_one -->|"measures campaigns"| node_campaigns
node_regime_one -->|"trains models"| node_models
node_regime_one -->|"scores predictions"| node_metrics
node_regime_one -->|"sets seeds"| node_seed
node_regime_one -->|"saves results"| node_io
node_regime_two -->|"loads splits"| node_prepared_data
node_regime_two -->|"builds masks"| node_sampler
node_regime_two -->|"measures campaigns"| node_campaigns
node_regime_two -->|"trains models"| node_models
node_regime_two -->|"reuses masks"| node_regime_one
node_regime_two -->|"scores predictions"| node_metrics
node_regime_two -->|"sets seeds"| node_seed
node_regime_two -->|"saves results"| node_io
node_baseline -->|"writes results"| node_result_files
node_regime_one -->|"writes results"| node_result_files
node_regime_two -->|"writes results"| node_result_files
node_column_check -->|"uses baseline"| node_baseline
node_host_spread -->|"uses sampling"| node_sampler
node_regime_analysis -->|"uses campaign logic"| node_campaigns
node_regime_analysis -->|"uses preprocessing"| node_prepared_data
node_analysis -->|"uses metrics"| node_metrics
node_summary -->|"uses preprocessing"| node_prepared_data
node_figures -->|"loads split data"| node_prepared_data
node_figures -->|"styles and saves"| node_plotting
node_figures -->|"reads and writes figures"| node_result_files
node_analysis -->|"uses file helpers"| node_io
node_regime_analysis -->|"uses file helpers"| node_io
node_summary -->|"uses file helpers"| node_io
node_column_check -->|"uses file helpers"| node_io
node_host_spread -->|"uses file helpers"| node_io

click node_prepared_data "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/preprocess.py"
click node_sampler "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/sampling.py"
click node_campaigns "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/campaigns.py"
click node_models "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/models.py"
click node_baseline "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/baseline.py"
click node_regime_one "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/run_experiment.py"
click node_regime_two "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/run_regime2.py"
click node_column_check "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/column_check.py"
click node_host_spread "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/host_keyed_spread.py"
click node_regime_analysis "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/analyze_regimes.py"
click node_analysis "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/analyze.py"
click node_summary "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/summary_table.py"
click node_figures "https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/figures.py"
click node_result_files "https://github.com/abdull66-hash/detection-under-budgets/tree/main/p1_monitoring_budget/results"
click node_io "https://github.com/abdull66-hash/detection-under-budgets/blob/main/common/io.py"
click node_metrics "https://github.com/abdull66-hash/detection-under-budgets/blob/main/common/metrics.py"
click node_seed "https://github.com/abdull66-hash/detection-under-budgets/blob/main/common/seed.py"
click node_plotting "https://github.com/abdull66-hash/detection-under-budgets/blob/main/common/plotting.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prepared_data toneBlue
class node_sampler,node_campaigns,node_models,node_baseline,node_regime_one,node_regime_two,node_dataset toneAmber
class node_column_check,node_host_spread,node_regime_analysis,node_analysis,node_summary,node_figures,node_result_files toneMint
class node_io,node_metrics,node_seed,node_plotting toneRose
class node_researcher toneIndigo
```

## Components

### Data preparation

- [Clean splits](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/preprocess.py): `p1_monitoring_budget/src/preprocess.py`

### Experiments

- [Sampling policies](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/sampling.py): `p1_monitoring_budget/src/sampling.py`
- [Campaign metrics](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/campaigns.py): `p1_monitoring_budget/src/campaigns.py`
- [Attack models](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/models.py): `p1_monitoring_budget/src/models.py`
- [Baseline runs](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/baseline.py): `p1_monitoring_budget/src/baseline.py`
- [Full-train regime](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/run_experiment.py): `p1_monitoring_budget/src/run_experiment.py`
- [Sampled-train regime](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/run_regime2.py): `p1_monitoring_budget/src/run_regime2.py`

### Analysis and reporting

- [Column checks](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/column_check.py): `p1_monitoring_budget/src/column_check.py`
- [Host-keyed spread](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/host_keyed_spread.py): `p1_monitoring_budget/src/host_keyed_spread.py`
- [Regime analysis](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/analyze_regimes.py): `p1_monitoring_budget/src/analyze_regimes.py`
- [Result analysis](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/analyze.py): `p1_monitoring_budget/src/analyze.py`
- [Summary tables](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/summary_table.py): `p1_monitoring_budget/src/summary_table.py`
- [Report figures](https://github.com/abdull66-hash/detection-under-budgets/blob/main/p1_monitoring_budget/src/figures.py): `p1_monitoring_budget/src/figures.py`
- [Experiment results](https://github.com/abdull66-hash/detection-under-budgets/tree/main/p1_monitoring_budget/results): `p1_monitoring_budget/results`

### Shared helpers

- [File helpers](https://github.com/abdull66-hash/detection-under-budgets/blob/main/common/io.py): `common/io.py`
- [Metric helpers](https://github.com/abdull66-hash/detection-under-budgets/blob/main/common/metrics.py): `common/metrics.py`
- [Seed control](https://github.com/abdull66-hash/detection-under-budgets/blob/main/common/seed.py): `common/seed.py`
- [Plot helpers](https://github.com/abdull66-hash/detection-under-budgets/blob/main/common/plotting.py): `common/plotting.py`

### Other components

- **Researcher**
- **Flow dataset**

## Connections

- Researcher → Baseline runs: runs baseline (evidence: `p1_monitoring_budget/src/baseline.py`)
- Researcher → Full-train regime: runs experiment (evidence: `p1_monitoring_budget/src/run_experiment.py`)
- Researcher → Sampled-train regime: runs regime (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Flow dataset → Clean splits: loads and cleans (evidence: `p1_monitoring_budget/src/preprocess.py`)
- Clean splits → File helpers: loads parquet (evidence: `p1_monitoring_budget/src/preprocess.py`)
- Baseline runs → Clean splits: loads splits (evidence: `p1_monitoring_budget/src/baseline.py`)
- Baseline runs → Attack models: builds models (evidence: `p1_monitoring_budget/src/baseline.py`)
- Baseline runs → Metric helpers: scores predictions (evidence: `p1_monitoring_budget/src/baseline.py`)
- Baseline runs → Seed control: sets seeds (evidence: `p1_monitoring_budget/src/baseline.py`)
- Full-train regime → Clean splits: loads splits (evidence: `p1_monitoring_budget/src/run_experiment.py`)
- Full-train regime → Sampling policies: builds masks (evidence: `p1_monitoring_budget/src/run_experiment.py`)
- Full-train regime → Campaign metrics: measures campaigns (evidence: `p1_monitoring_budget/src/run_experiment.py`)
- Full-train regime → Attack models: trains models (evidence: `p1_monitoring_budget/src/run_experiment.py`)
- Full-train regime → Metric helpers: scores predictions (evidence: `p1_monitoring_budget/src/run_experiment.py`)
- Full-train regime → Seed control: sets seeds (evidence: `p1_monitoring_budget/src/run_experiment.py`)
- Full-train regime → File helpers: saves results (evidence: `p1_monitoring_budget/src/run_experiment.py`)
- Sampled-train regime → Clean splits: loads splits (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Sampled-train regime → Sampling policies: builds masks (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Sampled-train regime → Campaign metrics: measures campaigns (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Sampled-train regime → Attack models: trains models (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Sampled-train regime → Full-train regime: reuses masks (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Sampled-train regime → Metric helpers: scores predictions (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Sampled-train regime → Seed control: sets seeds (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Sampled-train regime → File helpers: saves results (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Baseline runs → Experiment results: writes results (evidence: `p1_monitoring_budget/src/baseline.py`)
- Full-train regime → Experiment results: writes results (evidence: `p1_monitoring_budget/src/run_experiment.py`)
- Sampled-train regime → Experiment results: writes results (evidence: `p1_monitoring_budget/src/run_regime2.py`)
- Column checks → Baseline runs: uses baseline (evidence: `p1_monitoring_budget/src/column_check.py`)
- Host-keyed spread → Sampling policies: uses sampling (evidence: `p1_monitoring_budget/src/host_keyed_spread.py`)
- Regime analysis → Campaign metrics: uses campaign logic (evidence: `p1_monitoring_budget/src/analyze_regimes.py`)
- Regime analysis → Clean splits: uses preprocessing (evidence: `p1_monitoring_budget/src/analyze_regimes.py`)
- Result analysis → Metric helpers: uses metrics (evidence: `p1_monitoring_budget/src/analyze.py`)
- Summary tables → Clean splits: uses preprocessing (evidence: `p1_monitoring_budget/src/summary_table.py`)
- Report figures → Clean splits: loads split data (evidence: `p1_monitoring_budget/src/figures.py`)
- Report figures → Plot helpers: styles and saves (evidence: `p1_monitoring_budget/src/figures.py`)
- Report figures → Experiment results: reads and writes figures (evidence: `p1_monitoring_budget/src/figures.py`)
- Result analysis → File helpers: uses file helpers (evidence: `p1_monitoring_budget/src/analyze.py`)
- Regime analysis → File helpers: uses file helpers (evidence: `p1_monitoring_budget/src/analyze_regimes.py`)
- Summary tables → File helpers: uses file helpers (evidence: `p1_monitoring_budget/src/summary_table.py`)
- Column checks → File helpers: uses file helpers (evidence: `p1_monitoring_budget/src/column_check.py`)
- Host-keyed spread → File helpers: uses file helpers (evidence: `p1_monitoring_budget/src/host_keyed_spread.py`)