# salaudeenn-egov/dst-automation architecture

> Architecture diagram of the GitHub repository salaudeenn-egov/dst-automation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/salaudeenn-egov/dst-automation
- Repository: https://github.com/salaudeenn-egov/dst-automation

## Overview

DST Campaign Automation turns scheduled or manually requested campaign runs into performance and CDD-sync workbooks, internal and partner Word reports, and shared Drive/Slack outputs. The scheduler reloads campaign settings; configuration resolves dates and Elasticsearch indices; the selected SPAQ/AZM or ITN/LLIN path analyzes campaign data and tracks CDD sync, with optional stock reporting, before notification. The README documents these major stages. Sampled excerpts show the scheduler’s dispatch and some service calls; unsampled implementations and several orchestration details are represented only where documented.

Main entry points: [`scheduler.py`](https://github.com/salaudeenn-egov/dst-automation/blob/main/scheduler.py) (Campaign scheduler), [`run.py`](https://github.com/salaudeenn-egov/dst-automation/blob/main/run.py) (One-off runner), [`pipeline/report.py`](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/report.py) (SPAQ/AZM reports), [`pipeline/report_itn.py`](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/report_itn.py) (ITN/LLIN reports)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Run orchestration"]
  node_scheduler["Campaign scheduler<br/>[scheduler.py]"]
  node_runner["One-off runner<br/>[run.py]"]
  node_config["Campaign config<br/>[config.py]"]
end

subgraph group_campaign_data["Campaign data"]
  node_spaq_analysis["SPAQ/AZM analysis<br/>[analyze.py]"]
  node_itn_analysis["ITN/LLIN analysis<br/>[analyze_itn.py]"]
  node_spaq_sync["SPAQ/AZM CDD sync<br/>[cdd_sync.py]"]
  node_itn_sync["ITN/LLIN CDD sync<br/>[cdd_sync_itn.py]"]
  node_stock["Stock reporting<br/>[stock.py]"]
end

subgraph group_reporting["Reporting"]
  node_spaq_report["SPAQ/AZM reports<br/>[report.py]"]
  node_itn_report["ITN/LLIN reports<br/>[report_itn.py]"]
  node_error_tracer["Error tracer<br/>[error_tracer.py]"]
end

subgraph group_delivery["Delivery"]
  node_delivery["Drive and Slack"]
end

node_operator(("Campaign operator"))
node_sheets["Google Sheets"]
node_elasticsearch[("Elasticsearch")]
node_groq["Groq API"]

node_operator -->|"starts scheduled runs"| node_scheduler
node_operator -->|"starts one-off runs"| node_runner
node_scheduler -->|"reloads campaign rows"| node_sheets
node_scheduler -->|"builds run config"| node_config
node_runner -->|"builds run config"| node_config
node_config -->|"reads campaign settings"| node_sheets
node_scheduler -->|"dispatches SPAQ/AZM"| node_spaq_analysis
node_scheduler -->|"dispatches ITN/LLIN"| node_itn_analysis
node_scheduler -->|"runs sync stage"| node_spaq_sync
node_scheduler -->|"runs sync stage"| node_itn_sync
node_scheduler -->|"runs report stage"| node_spaq_report
node_scheduler -->|"runs report stage"| node_itn_report
node_runner -->|"dispatches SPAQ/AZM"| node_spaq_analysis
node_runner -->|"dispatches ITN/LLIN"| node_itn_analysis
node_runner -->|"runs sync stage"| node_spaq_sync
node_runner -->|"runs sync stage"| node_itn_sync
node_runner -->|"runs report stage"| node_spaq_report
node_runner -->|"runs report stage"| node_itn_report
node_spaq_analysis -->|"queries campaign data"| node_elasticsearch
node_itn_analysis -->|"queries campaign data"| node_elasticsearch
node_spaq_sync -->|"queries sync data"| node_elasticsearch
node_itn_sync -->|"queries sync data"| node_elasticsearch
node_spaq_report -->|"requests narrative"| node_groq
node_itn_report -->|"requests narrative"| node_groq
node_spaq_report -->|"uses error metrics"| node_error_tracer
node_itn_report -->|"uses error metrics"| node_error_tracer
node_spaq_report -->|"uploads and posts"| node_delivery
node_itn_report -->|"uploads and posts"| node_delivery
node_scheduler -->|"optionally runs stock"| node_stock
node_runner -->|"optionally runs stock"| node_stock
node_stock -->|"queries stock data"| node_elasticsearch

click node_scheduler "https://github.com/salaudeenn-egov/dst-automation/blob/main/scheduler.py"
click node_runner "https://github.com/salaudeenn-egov/dst-automation/blob/main/run.py"
click node_config "https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/config.py"
click node_spaq_analysis "https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/analyze.py"
click node_itn_analysis "https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/analyze_itn.py"
click node_spaq_sync "https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/cdd_sync.py"
click node_itn_sync "https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/cdd_sync_itn.py"
click node_stock "https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/stock.py"
click node_spaq_report "https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/report.py"
click node_itn_report "https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/report_itn.py"
click node_error_tracer "https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/error_tracer.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scheduler,node_runner,node_config toneBlue
class node_spaq_analysis,node_itn_analysis,node_spaq_sync,node_itn_sync,node_stock,node_elasticsearch toneAmber
class node_spaq_report,node_itn_report,node_error_tracer,node_groq toneMint
class node_delivery toneRose
class node_operator,node_sheets toneIndigo
```

## Components

### Run orchestration

- [Campaign scheduler](https://github.com/salaudeenn-egov/dst-automation/blob/main/scheduler.py): `scheduler.py`
- [One-off runner](https://github.com/salaudeenn-egov/dst-automation/blob/main/run.py): `run.py`
- [Campaign config](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/config.py): `pipeline/config.py`

### Campaign data

- [SPAQ/AZM analysis](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/analyze.py): `pipeline/analyze.py`
- [ITN/LLIN analysis](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/analyze_itn.py): `pipeline/analyze_itn.py`
- [SPAQ/AZM CDD sync](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/cdd_sync.py): `pipeline/cdd_sync.py`
- [ITN/LLIN CDD sync](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/cdd_sync_itn.py): `pipeline/cdd_sync_itn.py`
- [Stock reporting](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/stock.py): `pipeline/stock.py`

### Reporting

- [SPAQ/AZM reports](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/report.py): `pipeline/report.py`
- [ITN/LLIN reports](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/report_itn.py): `pipeline/report_itn.py`
- [Error tracer](https://github.com/salaudeenn-egov/dst-automation/blob/main/pipeline/error_tracer.py): `pipeline/error_tracer.py`

### Delivery

- **Drive and Slack**

### Other components

- **Campaign operator**
- **Google Sheets**
- **Elasticsearch**
- **Groq API**

## Connections

- Campaign operator → Campaign scheduler: starts scheduled runs (evidence: `README.md`)
- Campaign operator → One-off runner: starts one-off runs (evidence: `README.md`)
- Campaign scheduler → Google Sheets: reloads campaign rows (evidence: `README.md`)
- Campaign scheduler → Campaign config: builds run config (evidence: `scheduler.py`)
- One-off runner → Campaign config: builds run config (evidence: `run.py`)
- Campaign config → Google Sheets: reads campaign settings (evidence: `README.md`)
- Campaign scheduler → SPAQ/AZM analysis: dispatches SPAQ/AZM (evidence: `scheduler.py`)
- Campaign scheduler → ITN/LLIN analysis: dispatches ITN/LLIN (evidence: `scheduler.py`)
- Campaign scheduler → SPAQ/AZM CDD sync: runs sync stage (evidence: `README.md`)
- Campaign scheduler → ITN/LLIN CDD sync: runs sync stage (evidence: `README.md`)
- Campaign scheduler → SPAQ/AZM reports: runs report stage (evidence: `README.md`)
- Campaign scheduler → ITN/LLIN reports: runs report stage (evidence: `README.md`)
- One-off runner → SPAQ/AZM analysis: dispatches SPAQ/AZM (evidence: `run.py`)
- One-off runner → ITN/LLIN analysis: dispatches ITN/LLIN (evidence: `run.py`)
- One-off runner → SPAQ/AZM CDD sync: runs sync stage (evidence: `run.py`)
- One-off runner → ITN/LLIN CDD sync: runs sync stage (evidence: `run.py`)
- One-off runner → SPAQ/AZM reports: runs report stage (evidence: `run.py`)
- One-off runner → ITN/LLIN reports: runs report stage (evidence: `run.py`)
- SPAQ/AZM analysis → Elasticsearch: queries campaign data (evidence: `pipeline/analyze.py`)
- ITN/LLIN analysis → Elasticsearch: queries campaign data (evidence: `pipeline/analyze_itn.py`)
- SPAQ/AZM CDD sync → Elasticsearch: queries sync data (evidence: `pipeline/cdd_sync.py`)
- ITN/LLIN CDD sync → Elasticsearch: queries sync data (evidence: `pipeline/cdd_sync_itn.py`)
- SPAQ/AZM reports → Groq API: requests narrative (evidence: `pipeline/report.py`)
- ITN/LLIN reports → Groq API: requests narrative (evidence: `README.md`)
- SPAQ/AZM reports → Error tracer: uses error metrics (evidence: `pipeline/report.py`)
- ITN/LLIN reports → Error tracer: uses error metrics (evidence: `pipeline/report_itn.py`)
- SPAQ/AZM reports → Drive and Slack: uploads and posts (evidence: `README.md`)
- ITN/LLIN reports → Drive and Slack: uploads and posts (evidence: `README.md`)
- Campaign scheduler → Stock reporting: optionally runs stock (evidence: `scheduler.py`)
- One-off runner → Stock reporting: optionally runs stock (evidence: `run.py`)
- Stock reporting → Elasticsearch: queries stock data (evidence: `pipeline/stock.py`)
