# nirvishagarara/ai-data-quality-copilot architecture

> Architecture diagram of the GitHub repository nirvishagarara/ai-data-quality-copilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nirvishagarara/ai-data-quality-copilot
- Repository: https://github.com/nirvishagarara/ai-data-quality-copilot

## Overview

Data Quality Copilot monitors a DuckDB warehouse for schema drift and statistical anomalies, generates dbt-compatible tests, and can explain detected issues with Claude, alert Slack, and render lineage. Users enter through the CLI or Streamlit dashboard; the CLI dispatches commands to the monitoring and analysis modules, while the dashboard can also access the FastAPI service. Results and historical baselines are persisted for review. The README documents the principal stages and optional capabilities; some implementation details, including test generation and portions of the API and dashboard, were not sampled, so their connections are limited to documented or indexed evidence.

Main entry points: [`dq_copilot.py`](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/dq_copilot.py) (CLI Commands), [`src/monitoring/schema_monitor.py`](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/monitoring/schema_monitor.py) (Schema Drift Monitor), [`src/monitoring/anomaly_detector.py`](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/monitoring/anomaly_detector.py) (Anomaly Detector), [`src/api/main.py`](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/api/main.py) (REST API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User Interfaces"]
  node_cli["CLI Commands<br/>[dq_copilot.py]"]
  node_dashboard["Streamlit Dashboard<br/>[app.py]"]
  node_api["REST API<br/>[main.py]"]
end

subgraph group_observe["Warehouse Observability"]
  node_schema["Schema Drift Monitor<br/>[schema_monitor.py]"]
  node_anomaly["Anomaly Detector"]
  node_testgen["Test Generator<br/>[test_generator.py]"]
end

subgraph group_insights["Analysis and Delivery"]
  node_analyzer["Root-Cause Analyzer"]
  node_claude{{"Claude API"}}
  node_slack{{"Slack Webhook"}}
  node_alerts["Slack Alerts<br/>[slack_alerts.py]"]
  node_lineage["Lineage Graph<br/>[lineage_graph.py]"]
end

subgraph group_state["Configuration and State"]
  node_warehouse[("DuckDB Warehouse<br/>[warehouse.duckdb]")]
  node_snapshots[("Schema and Metric History")]
  node_config["Runtime Configuration<br/>[config.py]"]
  node_reports["Reports and Test Outputs"]
end

node_user(("Engineer"))

node_user -->|"runs commands"| node_cli
node_user -->|"explores results"| node_dashboard
node_cli -->|"runs scan"| node_schema
node_cli -->|"runs scan"| node_anomaly
node_cli -->|"dispatches explain"| node_analyzer
node_cli -->|"dispatches tests"| node_testgen
node_cli -->|"dispatches alerts"| node_alerts
node_cli -->|"dispatches lineage"| node_lineage
node_dashboard -->|"requests data"| node_api
node_api -->|"runs detector"| node_anomaly
node_api -->|"serves schema"| node_schema
node_schema -->|"reads schema"| node_warehouse
node_anomaly -->|"measures tables"| node_warehouse
node_schema -->|"stores drift history"| node_snapshots
node_anomaly -->|"reads baselines"| node_snapshots
node_testgen -->|"profiles tables"| node_warehouse
node_schema -->|"loads settings"| node_config
node_anomaly -->|"loads settings"| node_config
node_analyzer -->|"requests analysis"| node_claude
node_analyzer -->|"saves reports"| node_reports
node_alerts -->|"posts alerts"| node_slack
node_lineage -->|"uses issue status"| node_reports
node_lineage -->|"loads graph edges"| node_config
node_testgen -->|"writes generated tests"| node_reports

click node_cli "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/dq_copilot.py"
click node_dashboard "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/frontend/app.py"
click node_api "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/api/main.py"
click node_schema "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/monitoring/schema_monitor.py"
click node_anomaly "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/monitoring/anomaly_detector.py"
click node_testgen "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/monitoring/test_generator.py"
click node_warehouse "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/data/warehouse.duckdb"
click node_snapshots "https://github.com/nirvishagarara/ai-data-quality-copilot/tree/main/data/snapshots"
click node_config "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/config.py"
click node_analyzer "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/llm/root_cause_analyzer.py"
click node_alerts "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/alerts/slack_alerts.py"
click node_lineage "https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/lineage/lineage_graph.py"
click node_reports "https://github.com/nirvishagarara/ai-data-quality-copilot/tree/main/data"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_dashboard,node_api toneBlue
class node_schema,node_anomaly,node_testgen toneAmber
class node_analyzer,node_claude,node_slack,node_alerts,node_lineage toneMint
class node_warehouse,node_snapshots,node_config,node_reports toneRose
class node_user toneIndigo
```

## Components

### User Interfaces

- [CLI Commands](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/dq_copilot.py): `dq_copilot.py`
- [Streamlit Dashboard](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/frontend/app.py): `frontend/app.py`
- [REST API](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/api/main.py): `src/api/main.py`

### Warehouse Observability

- [Schema Drift Monitor](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/monitoring/schema_monitor.py): `src/monitoring/schema_monitor.py`
- [Anomaly Detector](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/monitoring/anomaly_detector.py): `src/monitoring/anomaly_detector.py`
- [Test Generator](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/monitoring/test_generator.py): `src/monitoring/test_generator.py`

### Analysis and Delivery

- [Root-Cause Analyzer](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/llm/root_cause_analyzer.py): `src/llm/root_cause_analyzer.py`
- **Claude API**
- **Slack Webhook**
- [Slack Alerts](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/alerts/slack_alerts.py): `src/alerts/slack_alerts.py`
- [Lineage Graph](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/lineage/lineage_graph.py): `src/lineage/lineage_graph.py`

### Configuration and State

- [DuckDB Warehouse](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/data/warehouse.duckdb): `data/warehouse.duckdb`
- [Schema and Metric History](https://github.com/nirvishagarara/ai-data-quality-copilot/tree/main/data/snapshots): `data/snapshots`
- [Runtime Configuration](https://github.com/nirvishagarara/ai-data-quality-copilot/blob/main/src/config.py): `src/config.py`
- [Reports and Test Outputs](https://github.com/nirvishagarara/ai-data-quality-copilot/tree/main/data): `data`

### Other components

- **Engineer**

## Connections

- Engineer → CLI Commands: runs commands (evidence: `README.md`)
- Engineer → Streamlit Dashboard: explores results (evidence: `README.md`)
- CLI Commands → Schema Drift Monitor: runs scan (evidence: `dq_copilot.py`)
- CLI Commands → Anomaly Detector: runs scan (evidence: `dq_copilot.py`)
- CLI Commands → Root-Cause Analyzer: dispatches explain (evidence: `dq_copilot.py`)
- CLI Commands → Test Generator: dispatches tests (evidence: `dq_copilot.py`)
- CLI Commands → Slack Alerts: dispatches alerts (evidence: `dq_copilot.py`)
- CLI Commands → Lineage Graph: dispatches lineage (evidence: `dq_copilot.py`)
- Streamlit Dashboard → REST API: requests data (evidence: `src/api/main.py`)
- REST API → Anomaly Detector: runs detector (evidence: `src/api/main.py`)
- REST API → Schema Drift Monitor: serves schema (evidence: `src/api/main.py`)
- Schema Drift Monitor → DuckDB Warehouse: reads schema (evidence: `src/monitoring/schema_monitor.py`)
- Anomaly Detector → DuckDB Warehouse: measures tables (evidence: `src/monitoring/anomaly_detector.py`)
- Schema Drift Monitor → Schema and Metric History: stores drift history (evidence: `src/monitoring/schema_monitor.py`)
- Anomaly Detector → Schema and Metric History: reads baselines (evidence: `README.md`)
- Test Generator → DuckDB Warehouse: profiles tables (evidence: `README.md`)
- Schema Drift Monitor → Runtime Configuration: loads settings (evidence: `src/monitoring/schema_monitor.py`)
- Anomaly Detector → Runtime Configuration: loads settings (evidence: `src/monitoring/anomaly_detector.py`)
- Root-Cause Analyzer → Claude API: requests analysis (evidence: `src/llm/root_cause_analyzer.py`)
- Root-Cause Analyzer → Reports and Test Outputs: saves reports (evidence: `src/llm/root_cause_analyzer.py`)
- Slack Alerts → Slack Webhook: posts alerts (evidence: `src/alerts/slack_alerts.py`)
- Lineage Graph → Reports and Test Outputs: uses issue status (evidence: `src/lineage/lineage_graph.py`)
- Lineage Graph → Runtime Configuration: loads graph edges (evidence: `src/lineage/lineage_graph.py`)
- Test Generator → Reports and Test Outputs: writes generated tests (evidence: `README.md`)
