# elidehaene01/data-observability-agent architecture

> Architecture diagram of the GitHub repository elidehaene01/data-observability-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/elidehaene01/data-observability-agent
- Repository: https://github.com/elidehaene01/data-observability-agent

## Overview

Data-Observability-Agent compares source ERP data with dbt warehouse models, then handles scheduled data-load checks and PR-triggered discrepancy classification as distinct workflows. Documented PR capabilities also include merge gating, per-model documentation refresh, and Confluence reconciliation reports. The graph preserves these README-described capabilities; their wiring is not fully visible in sampled source, so uncertain implementation links are omitted. Sampled source supports the reconciliation connectors, LangGraph classification branches, results history, and dashboard read path.

Main entry points: [`reconciliation/aggregate_checks.py`](https://github.com/elidehaene01/data-observability-agent/blob/main/reconciliation/aggregate_checks.py) (Aggregate checks), [`agent/graph.py`](https://github.com/elidehaene01/data-observability-agent/blob/main/agent/graph.py) (Agent routing graph), [`agent/nodes/fetch_reconciliation_results.py`](https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/fetch_reconciliation_results.py) (Fetch results), [`agent/nodes/action_nodes.py`](https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/action_nodes.py) (Branch actions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_checks["Reconciliation"]
  node_reconcile["Aggregate checks"]
  node_sample_checks["Sample checks<br/>[sample_checks.py]"]
  node_result_models["Reconciliation results<br/>[models.py]"]
end

subgraph group_reasoning["Agent workflow"]
  node_agent_graph["Agent routing graph<br/>[graph.py]"]
  node_fetch_results["Fetch results"]
  node_analyze_diff["Analyze SQL diff<br/>[analyze_diff.py]"]
  node_classify["Classify discrepancy"]
  node_actions["Branch actions<br/>[action_nodes.py]"]
  node_agent_state["Agent state<br/>[state.py]"]
  node_merge_gate["Merge gate"]
end

subgraph group_docs["Model docs"]
  node_doc_manifest["Read dbt artifacts<br/>[manifest.py]"]
  node_doc_models["Model structures<br/>[models.py]"]
  node_doc_render["Render model docs<br/>[render.py]"]
end

subgraph group_connectors["External connectors"]
  node_confluence["Confluence"]
  node_ticketing["Jira and Slack"]
  node_postgres["Postgres connector<br/>[postgres_source.py]"]
  node_duckdb["DuckDB connector<br/>[duckdb_target.py]"]
end

subgraph group_history["Results and reporting"]
  node_results_store[("Run history<br/>[writer.py]")]
  node_results_reader["Results reader<br/>[reader.py]"]
  node_dashboard["HTML dashboard<br/>[html_dashboard.py]"]
end

node_scheduler(("Scheduled trigger"))
node_pull_request(("dbt PR"))
node_source_erp[("ERP source")]
node_warehouse[("dbt warehouse")]

node_scheduler -->|"starts checks"| node_reconcile
node_pull_request -->|"starts checks"| node_reconcile
node_reconcile -->|"queries source"| node_postgres
node_reconcile -->|"queries target"| node_duckdb
node_postgres -->|"reads data"| node_source_erp
node_duckdb -->|"reads models"| node_warehouse
node_reconcile -->|"runs checks"| node_sample_checks
node_reconcile -->|"creates results"| node_result_models
node_pull_request -->|"invokes agent"| node_agent_graph
node_agent_graph -->|"dispatches"| node_fetch_results
node_fetch_results -->|"reads history"| node_results_store
node_fetch_results -->|"passes results"| node_analyze_diff
node_analyze_diff -->|"passes evidence"| node_classify
node_classify -->|"updates state"| node_agent_state
node_agent_graph -->|"routes outcome"| node_actions
node_actions -->|"publishes docs"| node_confluence
node_actions -->|"creates and notifies"| node_ticketing
node_pull_request -->|"checks classification"| node_merge_gate
node_pull_request -->|"refreshes model docs"| node_doc_manifest
node_doc_manifest -->|"builds structures"| node_doc_models
node_doc_models -->|"renders docs"| node_doc_render
node_doc_render -->|"publishes pages"| node_confluence
node_reconcile -->|"writes results"| node_results_store
node_results_reader -->|"reads rows"| node_results_store
node_dashboard -->|"queries history"| node_results_reader

click node_reconcile "https://github.com/elidehaene01/data-observability-agent/blob/main/reconciliation/aggregate_checks.py"
click node_sample_checks "https://github.com/elidehaene01/data-observability-agent/blob/main/reconciliation/sample_checks.py"
click node_result_models "https://github.com/elidehaene01/data-observability-agent/blob/main/reconciliation/models.py"
click node_agent_graph "https://github.com/elidehaene01/data-observability-agent/blob/main/agent/graph.py"
click node_fetch_results "https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/fetch_reconciliation_results.py"
click node_analyze_diff "https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/analyze_diff.py"
click node_classify "https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/classify_discrepancy.py"
click node_actions "https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/action_nodes.py"
click node_agent_state "https://github.com/elidehaene01/data-observability-agent/blob/main/agent/state.py"
click node_doc_manifest "https://github.com/elidehaene01/data-observability-agent/blob/main/model_docs/manifest.py"
click node_doc_models "https://github.com/elidehaene01/data-observability-agent/blob/main/model_docs/models.py"
click node_doc_render "https://github.com/elidehaene01/data-observability-agent/blob/main/model_docs/render.py"
click node_ticketing "https://github.com/elidehaene01/data-observability-agent/tree/main/connectors/ticketing"
click node_postgres "https://github.com/elidehaene01/data-observability-agent/blob/main/connectors/source/postgres_source.py"
click node_duckdb "https://github.com/elidehaene01/data-observability-agent/blob/main/connectors/target/duckdb_target.py"
click node_results_store "https://github.com/elidehaene01/data-observability-agent/blob/main/results_store/writer.py"
click node_results_reader "https://github.com/elidehaene01/data-observability-agent/blob/main/results_store/reader.py"
click node_dashboard "https://github.com/elidehaene01/data-observability-agent/blob/main/connectors/reporting/html_dashboard.py"
click node_merge_gate "https://github.com/elidehaene01/data-observability-agent/blob/main/scripts/run_code_change_check.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_reconcile,node_sample_checks,node_result_models toneBlue
class node_agent_graph,node_fetch_results,node_analyze_diff,node_classify,node_actions,node_agent_state,node_merge_gate,node_source_erp,node_warehouse toneAmber
class node_doc_manifest,node_doc_models,node_doc_render toneMint
class node_confluence,node_ticketing,node_postgres,node_duckdb toneRose
class node_results_store,node_results_reader,node_dashboard,node_scheduler,node_pull_request toneIndigo
```

## Components

### Reconciliation

- [Aggregate checks](https://github.com/elidehaene01/data-observability-agent/blob/main/reconciliation/aggregate_checks.py): `reconciliation/aggregate_checks.py`
- [Sample checks](https://github.com/elidehaene01/data-observability-agent/blob/main/reconciliation/sample_checks.py): `reconciliation/sample_checks.py`
- [Reconciliation results](https://github.com/elidehaene01/data-observability-agent/blob/main/reconciliation/models.py): `reconciliation/models.py`

### Agent workflow

- [Agent routing graph](https://github.com/elidehaene01/data-observability-agent/blob/main/agent/graph.py): `agent/graph.py`
- [Fetch results](https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/fetch_reconciliation_results.py): `agent/nodes/fetch_reconciliation_results.py`
- [Analyze SQL diff](https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/analyze_diff.py): `agent/nodes/analyze_diff.py`
- [Classify discrepancy](https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/classify_discrepancy.py): `agent/nodes/classify_discrepancy.py`
- [Branch actions](https://github.com/elidehaene01/data-observability-agent/blob/main/agent/nodes/action_nodes.py): `agent/nodes/action_nodes.py`
- [Agent state](https://github.com/elidehaene01/data-observability-agent/blob/main/agent/state.py): `agent/state.py`
- [Merge gate](https://github.com/elidehaene01/data-observability-agent/blob/main/scripts/run_code_change_check.py): `scripts/run_code_change_check.py`

### Model docs

- [Read dbt artifacts](https://github.com/elidehaene01/data-observability-agent/blob/main/model_docs/manifest.py): `model_docs/manifest.py`
- [Model structures](https://github.com/elidehaene01/data-observability-agent/blob/main/model_docs/models.py): `model_docs/models.py`
- [Render model docs](https://github.com/elidehaene01/data-observability-agent/blob/main/model_docs/render.py): `model_docs/render.py`

### External connectors

- **Confluence**
- [Jira and Slack](https://github.com/elidehaene01/data-observability-agent/tree/main/connectors/ticketing): `connectors/ticketing`
- [Postgres connector](https://github.com/elidehaene01/data-observability-agent/blob/main/connectors/source/postgres_source.py): `connectors/source/postgres_source.py`
- [DuckDB connector](https://github.com/elidehaene01/data-observability-agent/blob/main/connectors/target/duckdb_target.py): `connectors/target/duckdb_target.py`

### Results and reporting

- [Run history](https://github.com/elidehaene01/data-observability-agent/blob/main/results_store/writer.py): `results_store/writer.py`
- [Results reader](https://github.com/elidehaene01/data-observability-agent/blob/main/results_store/reader.py): `results_store/reader.py`
- [HTML dashboard](https://github.com/elidehaene01/data-observability-agent/blob/main/connectors/reporting/html_dashboard.py): `connectors/reporting/html_dashboard.py`

### Other components

- **Scheduled trigger**
- **dbt PR**
- **ERP source**
- **dbt warehouse**

## Connections

- Scheduled trigger → Aggregate checks: starts checks
- dbt PR → Aggregate checks: starts checks
- Aggregate checks → Postgres connector: queries source
- Aggregate checks → DuckDB connector: queries target
- Postgres connector → ERP source: reads data
- DuckDB connector → dbt warehouse: reads models
- Aggregate checks → Sample checks: runs checks
- Aggregate checks → Reconciliation results: creates results
- dbt PR → Agent routing graph: invokes agent
- Agent routing graph → Fetch results: dispatches
- Fetch results → Run history: reads history
- Fetch results → Analyze SQL diff: passes results
- Analyze SQL diff → Classify discrepancy: passes evidence
- Classify discrepancy → Agent state: updates state
- Agent routing graph → Branch actions: routes outcome
- Branch actions → Confluence: publishes docs
- Branch actions → Jira and Slack: creates and notifies
- dbt PR → Merge gate: checks classification
- dbt PR → Read dbt artifacts: refreshes model docs
- Read dbt artifacts → Model structures: builds structures
- Model structures → Render model docs: renders docs
- Render model docs → Confluence: publishes pages
- Aggregate checks → Run history: writes results
- Results reader → Run history: reads rows
- HTML dashboard → Results reader: queries history