# katherinewyers/tool-dq-workbench architecture

> Architecture diagram of the GitHub repository katherinewyers/tool-dq-workbench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/katherinewyers/tool-dq-workbench
- Repository: https://github.com/katherinewyers/tool-dq-workbench

## Overview

#### Purpose and operating model
Data Quality Workbench monitors DHIS2 data quality over time: validation-rule violations, outliers, and metadata-integrity issues. It writes resulting metrics back to DHIS2 as ordinary data values, enabling DHIS2-native charts, maps, and pivots. Configuration is created interactively, then executed separately on a schedule.

#### Entry points and deployment
- CLI monitoring entry point: `dq_monitor.py` and `app/cli.py`; intended for scheduled `dq-monitor --config ...` runs.
- Local configuration UI: `app/web/app.py`, with routes registered from `app/web/routes/ui_blueprint.py` and `app/web/routes/api_blueprint.py`.
- Containerized execution is defined by `Dockerfile` and `docker-compose.yml`.
- Windows packaged startup is represented by `installer/main_win.py` and installer specifications under `installer/`.

#### Configuration and connection boundary
Configuration loading and normalization are centralized in `app/core/config_loader.py`; the initial/default structure is exemplified by `config/sample_config.yml`. The web UI edits server credentials and analysis stages, including `app/web/routes/edit_server.py` and the per-stage edit routes. DHIS2 base URL and API token may come from environment variables or persisted configuration. The application has no built-in UI authentication, so deployment must remain local or trusted.

#### Analysis pipeline
The CLI reads configured stages, obtains DHIS2 data via shared API handling in `app/core/api_utils.py`, then dispatches the applicable analyzer:
- Validation-rule tracking: `app/analyzers/rule_analyzer.py`
- Outlier tracking: `app/analyzers/outlier_analyzer.py`
- Metadata integrity tracking: `app/analyzers/integrity_analyzer.py`
- Stage orchestration/representation: `app/analyzers/stage_analyzer.py`

Each stage converts DHIS2 quality findings into snapshot values suitable for posting to configured DHIS2 destination data elements.

#### DHIS2 writeback and temporal model
DHIS2 is the material external service and both source of quality findings and destination for generated metrics. Period and numeric semantics are isolated in `app/core/period_type.py`, `app/core/period_utils.py`, and `app/core/numeric_value_types.py`; these form the boundary that maps collected snapshots to DHIS2-compatible reporting periods and values. UID utilities in `app/core/uid_utils.py` support DHIS2 identifier handling.

#### Min/max generation subsystem
A distinct min/max workflow calculates thresholds/statistics rather than directly running a monitoring stage. Its domain model and aggregation logic are in `app/minmax/min_max_factory.py`, `app/minmax/min_max_statistics.py`, and `app/minmax/min_max_results_tracker.py`. The UI exposes separate min/max pages and execution endpoints through `app/web/routes/index_minmax.py`, `app/web/routes/minmax_analysis.py`, and `app/web/routes/run_minmax_stage.py`.

#### Web interaction boundary
The Flask-style web layer renders server and stage forms, invokes configuration mutations, and can execute stages for interactive feedback. Core run paths are `app/web/routes/run.py` and `app/web/routes/run_stage.py`; stage deletion/editing is route-specific. Templates under `app/web/templates/` and static Bootstrap/JavaScript assets provide a server-rendered local UI rather than a separate frontend runtime.

Main entry points: [`app/core/api_utils.py`](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/core/api_utils.py) (DHIS2 API handling), [`app/analyzers/stage_analyzer.py`](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/analyzers/stage_analyzer.py) (Stage orchestration), [`app/core/config_loader.py`](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/core/config_loader.py) (Configuration loader), [`app/web/app.py`](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/app.py) (Web application)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["Entry points"]
  node_cli["Monitor CLI<br/>Python CLI<br/>[dq_monitor.py]"]
  node_container{{"Container deployment<br/>deployment definition"}}
end

subgraph group_web["Local web UI"]
  node_web_app["Web application<br/>Flask app<br/>[app.py]"]
  node_ui_routes["UI routes<br/>route blueprint<br/>[ui_blueprint.py]"]
  node_api_routes["API routes<br/>route blueprint<br/>[api_blueprint.py]"]
  node_stage_run_routes["Interactive stage runs<br/>web routes<br/>[run_stage.py]"]
end

subgraph group_core["Core execution"]
  node_config["Configuration loader<br/>configuration boundary<br/>[config_loader.py]"]
  node_sample_config["Sample configuration<br/>YAML template<br/>[sample_config.yml]"]
  node_api_client["DHIS2 API handling<br/>API client<br/>[api_utils.py]"]
  node_period_values["Period and value mapping<br/>DHIS2 value semantics<br/>[period_type.py]"]
end

subgraph group_analysis["Quality analysis"]
  node_stage_orchestrator["Stage orchestration<br/>analysis dispatcher<br/>[stage_analyzer.py]"]
  node_rule_analyzer["Validation-rule tracker<br/>analyzer<br/>[rule_analyzer.py]"]
  node_outlier_analyzer["Outlier tracker<br/>analyzer"]
  node_integrity_analyzer["Metadata-integrity tracker<br/>analyzer"]
end

subgraph group_minmax["Min/max workflow"]
  node_minmax_routes["Min/max UI and execution<br/>web routes"]
  node_minmax_engine["Min/max statistics<br/>threshold engine<br/>[min_max_factory.py]"]
end

node_dhis2[("DHIS2<br/>external platform")]

node_cli -->|"loads config"| node_config
node_cli -->|"runs stages"| node_stage_orchestrator
node_web_app -->|"registers"| node_ui_routes
node_web_app -->|"registers"| node_api_routes
node_web_app -->|"serves"| node_stage_run_routes
node_ui_routes -->|"edits settings"| node_config
node_api_routes -->|"mutates config"| node_config
node_stage_run_routes -->|"executes"| node_stage_orchestrator
node_sample_config -.->|"initial structure"| node_config
node_config -->|"provides credentials"| node_api_client
node_stage_orchestrator -->|"dispatches"| node_rule_analyzer
node_stage_orchestrator -->|"dispatches"| node_outlier_analyzer
node_stage_orchestrator -->|"dispatches"| node_integrity_analyzer
node_rule_analyzer -->|"queries findings"| node_api_client
node_outlier_analyzer -->|"queries findings"| node_api_client
node_integrity_analyzer -->|"queries findings"| node_api_client
node_stage_orchestrator -->|"maps snapshots"| node_period_values
node_period_values -->|"formats writeback values"| node_api_client
node_api_client -->|"reads findings / posts metrics"| node_dhis2
node_minmax_routes -->|"runs"| node_minmax_engine
node_minmax_engine -->|"retrieves data"| node_api_client
node_container -.->|"runs scheduled command"| node_cli
node_container -.->|"hosts locally"| node_web_app

click node_cli "https://github.com/katherinewyers/tool-dq-workbench/blob/main/dq_monitor.py"
click node_web_app "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/app.py"
click node_ui_routes "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/routes/ui_blueprint.py"
click node_api_routes "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/routes/api_blueprint.py"
click node_stage_run_routes "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/routes/run_stage.py"
click node_config "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/core/config_loader.py"
click node_sample_config "https://github.com/katherinewyers/tool-dq-workbench/blob/main/config/sample_config.yml"
click node_api_client "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/core/api_utils.py"
click node_period_values "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/core/period_type.py"
click node_stage_orchestrator "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/analyzers/stage_analyzer.py"
click node_rule_analyzer "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/analyzers/rule_analyzer.py"
click node_outlier_analyzer "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/analyzers/outlier_analyzer.py"
click node_integrity_analyzer "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/analyzers/integrity_analyzer.py"
click node_minmax_routes "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/routes/run_minmax_stage.py"
click node_minmax_engine "https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/minmax/min_max_factory.py"
click node_container "https://github.com/katherinewyers/tool-dq-workbench/blob/main/Dockerfile"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_container toneBlue
class node_web_app,node_ui_routes,node_api_routes,node_stage_run_routes toneAmber
class node_config,node_sample_config,node_api_client,node_period_values toneMint
class node_stage_orchestrator,node_rule_analyzer,node_outlier_analyzer,node_integrity_analyzer toneRose
class node_minmax_routes,node_minmax_engine toneIndigo
class node_dhis2 toneNeutral
```

## Components

### Entry points

- [Monitor CLI](https://github.com/katherinewyers/tool-dq-workbench/blob/main/dq_monitor.py): Python CLI · Scheduled dq-monitor execution. · `dq_monitor.py`
- [Container deployment](https://github.com/katherinewyers/tool-dq-workbench/blob/main/Dockerfile): deployment definition · `Dockerfile`

### Local web UI

Server-rendered configuration and interactive execution UI.

- [Web application](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/app.py): Flask app · Local trusted configuration UI. · `app/web/app.py`
- [UI routes](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/routes/ui_blueprint.py): route blueprint · `app/web/routes/ui_blueprint.py`
- [API routes](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/routes/api_blueprint.py): route blueprint · `app/web/routes/api_blueprint.py`
- [Interactive stage runs](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/routes/run_stage.py): web routes · `app/web/routes/run_stage.py`

### Core execution

- [Configuration loader](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/core/config_loader.py): configuration boundary · Loads and normalizes persisted and environment-supplied settings. · `app/core/config_loader.py`
- [Sample configuration](https://github.com/katherinewyers/tool-dq-workbench/blob/main/config/sample_config.yml): YAML template · `config/sample_config.yml`
- [DHIS2 API handling](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/core/api_utils.py): API client · `app/core/api_utils.py`
- [Period and value mapping](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/core/period_type.py): DHIS2 value semantics · Maps snapshots to DHIS2 reporting periods and numeric values. · `app/core/period_type.py`

### Quality analysis

- [Stage orchestration](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/analyzers/stage_analyzer.py): analysis dispatcher · Selects and represents configured monitoring stages. · `app/analyzers/stage_analyzer.py`
- [Validation-rule tracker](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/analyzers/rule_analyzer.py): analyzer · `app/analyzers/rule_analyzer.py`
- [Outlier tracker](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/analyzers/outlier_analyzer.py): analyzer · `app/analyzers/outlier_analyzer.py`
- [Metadata-integrity tracker](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/analyzers/integrity_analyzer.py): analyzer · `app/analyzers/integrity_analyzer.py`

### Min/max workflow

- [Min/max UI and execution](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/web/routes/run_minmax_stage.py): web routes · Separate threshold-generation workflow. · `app/web/routes/run_minmax_stage.py`
- [Min/max statistics](https://github.com/katherinewyers/tool-dq-workbench/blob/main/app/minmax/min_max_factory.py): threshold engine · Calculates threshold statistics rather than monitoring snapshots. · `app/minmax/min_max_factory.py`

### Other components

- **DHIS2**: external platform · Supplies quality findings and stores generated metric data values.

## Connections

- Monitor CLI → Configuration loader: loads config
- Monitor CLI → Stage orchestration: runs stages
- Web application → UI routes: registers
- Web application → API routes: registers
- Web application → Interactive stage runs: serves
- UI routes → Configuration loader: edits settings
- API routes → Configuration loader: mutates config
- Interactive stage runs → Stage orchestration: executes
- Sample configuration → Configuration loader: initial structure
- Configuration loader → DHIS2 API handling: provides credentials
- Stage orchestration → Validation-rule tracker: dispatches
- Stage orchestration → Outlier tracker: dispatches
- Stage orchestration → Metadata-integrity tracker: dispatches
- Validation-rule tracker → DHIS2 API handling: queries findings
- Outlier tracker → DHIS2 API handling: queries findings
- Metadata-integrity tracker → DHIS2 API handling: queries findings
- Stage orchestration → Period and value mapping: maps snapshots
- Period and value mapping → DHIS2 API handling: formats writeback values
- DHIS2 API handling → DHIS2: reads findings / posts metrics
- Min/max UI and execution → Min/max statistics: runs
- Min/max statistics → DHIS2 API handling: retrieves data
- Container deployment → Monitor CLI: runs scheduled command
- Container deployment → Web application: hosts locally