# openjiuwen-ai/dataswarm architecture

> Architecture diagram of the GitHub repository openjiuwen-ai/dataswarm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openjiuwen-ai/dataswarm
- Repository: https://github.com/openjiuwen-ai/dataswarm

## Overview

DataSwarm is an enterprise data-agent application for onboarding data sources and answering natural-language data questions with auditable results. The principal workflow starts with a user request, proceeds through the documented session/context/planning/execution analysis chain, and returns conclusions and artifacts. Distinct supporting capabilities include data-source onboarding, metadata and semantic/metric lookup, warehouse design/build, and report/dashboard output. The README and source index establish several links; unsampled runtime wiring is omitted rather than inferred.

Main entry points: [`dataswarm/skills/dw-analysis/scripts/analysis_flow.py`](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/analysis_flow.py) (Analysis chain), [`dataswarm/skills/dw-warehouse-design/scripts/plan.py`](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/plan.py) (Warehouse plan), [`dataswarm/skills/dw-analysis/scripts/flow/commands.py`](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/flow/commands.py) (Flow commands), [`dataswarm/skills/_dw_common/scripts/dw_client.py`](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/_dw_common/scripts/dw_client.py) (Data access client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_analysis["Analysis workflow"]
  node_analysis_chain["Analysis chain<br/>[analysis_flow.py]"]
  node_analysis_commands["Flow commands<br/>[commands.py]"]
  node_analysis_execution["Execution core<br/>[execute_core.py]"]
  node_analysis_ledger["Trace observability<br/>[observability.py]"]
  node_data_insight["Data insight<br/>[analyzer.py]"]
end

subgraph group_data["Data access"]
  node_onboarding["Source onboarding<br/>[onboard.py]"]
  node_nl2sql["Natural-language SQL<br/>[execute_sql.py]"]
  node_db_client["SQL connection layer<br/>[_db_client.py]"]
  node_metric_query["Metric lifecycle/query<br/>[metric_query.py]"]
  node_metadata_semantics["Glossary and recall<br/>[recall_tables.py]"]
end

subgraph group_warehouse["Warehouse engineering"]
  node_design_plan["Warehouse plan<br/>[plan.py]"]
  node_discovery["Source discovery<br/>[discovery.py]"]
  node_schema_inference["Schema inference<br/>[infer.py]"]
  node_design_intent["Intent parsing<br/>[intent.py]"]
  node_semantic_schema["Semantic specification<br/>[schema.py]"]
  node_plan_render["Plan rendering<br/>[render.py]"]
  node_task_bridge["Plan to tasks<br/>[plan_to_tasks.py]"]
  node_pipeline_execution["Pipeline execution"]
end

subgraph group_outputs["Analytical outputs"]
  node_report_resolver["Report resolver<br/>[resolver.py]"]
  node_report_renderer["Report rendering<br/>[page.py]"]
  node_dashboard["BI dashboard<br/>[dashboard_tool.py]"]
end

subgraph group_shared["Shared runtime"]
  node_data_client["Data access client<br/>[dw_client.py]"]
  node_workspace_paths["Runtime paths<br/>[dw_paths.py]"]
end

node_user(("Analyst"))
node_dwcli_service{{"dwcli web API"}}

node_user -->|"asks analysis"| node_analysis_chain
node_analysis_chain -.->|"routes flow"| node_analysis_commands
node_analysis_chain -.->|"executes analysis"| node_analysis_execution
node_analysis_chain -->|"traces steps"| node_analysis_ledger
node_analysis_chain -.->|"produces analysis"| node_data_insight
node_analysis_chain -.->|"delivers artifacts"| node_report_resolver
node_user -->|"onboards sources"| node_onboarding
node_onboarding -->|"accesses data"| node_data_client
node_analysis_commands -->|"uses web API"| node_dwcli_service
node_metric_query -->|"calls metric-v2"| node_dwcli_service
node_nl2sql -.->|"uses connection layer"| node_db_client
node_db_client -->|"delegates data access"| node_data_client
node_db_client -->|"queries service"| node_dwcli_service
node_analysis_commands -.->|"queries metrics"| node_metric_query
node_analysis_commands -.->|"recalls context"| node_metadata_semantics
node_design_plan -->|"discovers sources"| node_discovery
node_design_plan -->|"infers schema"| node_schema_inference
node_design_plan -->|"uses intent"| node_design_intent
node_design_plan -->|"validates specification"| node_semantic_schema
node_design_plan -->|"renders plan"| node_plan_render
node_task_bridge -.->|"dispatches pipeline"| node_pipeline_execution
node_report_resolver -->|"reads data"| node_data_client
node_report_resolver -.->|"prepares report"| node_report_renderer
node_user -->|"explores dashboards"| node_dashboard
node_data_client -->|"resolves runtime paths"| node_workspace_paths

click node_analysis_chain "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/analysis_flow.py"
click node_analysis_commands "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/flow/commands.py"
click node_analysis_execution "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/flow/execute_core.py"
click node_analysis_ledger "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/flow/observability.py"
click node_data_insight "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/data-insight/scripts/analyzer.py"
click node_onboarding "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-datasource-onboarding/scripts/onboard.py"
click node_data_client "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/_dw_common/scripts/dw_client.py"
click node_nl2sql "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/nl2sql/scripts/execute_sql.py"
click node_db_client "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/nl2sql/scripts/_db_client.py"
click node_metric_query "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-metrics/scripts/metric_query.py"
click node_metadata_semantics "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/nl2sql/scripts/recall_tables.py"
click node_design_plan "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/plan.py"
click node_discovery "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/discovery.py"
click node_schema_inference "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/infer.py"
click node_design_intent "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/intent.py"
click node_semantic_schema "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/schema.py"
click node_plan_render "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/render.py"
click node_task_bridge "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-build/scripts/plan_to_tasks.py"
click node_pipeline_execution "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-build/scripts/execute_pipeline.py"
click node_report_resolver "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-report/scripts/resolver.py"
click node_report_renderer "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-report/scripts/render/page.py"
click node_dashboard "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-bi-dashboard/scripts/dashboard_tool.py"
click node_workspace_paths "https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/_dw_common/scripts/dw_paths.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_analysis_chain,node_analysis_commands,node_analysis_execution,node_analysis_ledger,node_data_insight toneBlue
class node_onboarding,node_nl2sql,node_db_client,node_metric_query,node_metadata_semantics toneAmber
class node_design_plan,node_discovery,node_schema_inference,node_design_intent,node_semantic_schema,node_plan_render,node_task_bridge,node_pipeline_execution,node_dwcli_service toneMint
class node_report_resolver,node_report_renderer,node_dashboard toneRose
class node_data_client,node_workspace_paths,node_user toneIndigo
```

## Components

### Analysis workflow

- [Analysis chain](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/analysis_flow.py): `dataswarm/skills/dw-analysis/scripts/analysis_flow.py`
- [Flow commands](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/flow/commands.py): `dataswarm/skills/dw-analysis/scripts/flow/commands.py`
- [Execution core](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/flow/execute_core.py): `dataswarm/skills/dw-analysis/scripts/flow/execute_core.py`
- [Trace observability](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-analysis/scripts/flow/observability.py): `dataswarm/skills/dw-analysis/scripts/flow/observability.py`
- [Data insight](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/data-insight/scripts/analyzer.py): `dataswarm/skills/data-insight/scripts/analyzer.py`

### Data access

- [Source onboarding](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-datasource-onboarding/scripts/onboard.py): `dataswarm/skills/dw-datasource-onboarding/scripts/onboard.py`
- [Natural-language SQL](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/nl2sql/scripts/execute_sql.py): `dataswarm/skills/nl2sql/scripts/execute_sql.py`
- [SQL connection layer](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/nl2sql/scripts/_db_client.py): `dataswarm/skills/nl2sql/scripts/_db_client.py`
- [Metric lifecycle/query](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-metrics/scripts/metric_query.py): `dataswarm/skills/dw-metrics/scripts/metric_query.py`
- [Glossary and recall](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/nl2sql/scripts/recall_tables.py): `dataswarm/skills/nl2sql/scripts/recall_tables.py`

### Warehouse engineering

- [Warehouse plan](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/plan.py): `dataswarm/skills/dw-warehouse-design/scripts/plan.py`
- [Source discovery](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/discovery.py): `dataswarm/skills/dw-warehouse-design/scripts/discovery.py`
- [Schema inference](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/infer.py): `dataswarm/skills/dw-warehouse-design/scripts/infer.py`
- [Intent parsing](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/intent.py): `dataswarm/skills/dw-warehouse-design/scripts/intent.py`
- [Semantic specification](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/schema.py): `dataswarm/skills/dw-warehouse-design/scripts/schema.py`
- [Plan rendering](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-design/scripts/render.py): `dataswarm/skills/dw-warehouse-design/scripts/render.py`
- [Plan to tasks](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-build/scripts/plan_to_tasks.py): `dataswarm/skills/dw-warehouse-build/scripts/plan_to_tasks.py`
- [Pipeline execution](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-warehouse-build/scripts/execute_pipeline.py): `dataswarm/skills/dw-warehouse-build/scripts/execute_pipeline.py`

### Analytical outputs

- [Report resolver](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-report/scripts/resolver.py): `dataswarm/skills/dw-report/scripts/resolver.py`
- [Report rendering](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-report/scripts/render/page.py): `dataswarm/skills/dw-report/scripts/render/page.py`
- [BI dashboard](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/dw-bi-dashboard/scripts/dashboard_tool.py): `dataswarm/skills/dw-bi-dashboard/scripts/dashboard_tool.py`

### Shared runtime

- [Data access client](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/_dw_common/scripts/dw_client.py): `dataswarm/skills/_dw_common/scripts/dw_client.py`
- [Runtime paths](https://github.com/openjiuwen-ai/dataswarm/blob/main/dataswarm/skills/_dw_common/scripts/dw_paths.py): `dataswarm/skills/_dw_common/scripts/dw_paths.py`

### Other components

- **Analyst**
- **dwcli web API**

## Connections

- Analyst → Analysis chain: asks analysis (evidence: `README.md`)
- Analysis chain → Flow commands: routes flow
- Analysis chain → Execution core: executes analysis
- Analysis chain → Trace observability: traces steps (evidence: `README.md`)
- Analysis chain → Data insight: produces analysis
- Analysis chain → Report resolver: delivers artifacts
- Analyst → Source onboarding: onboards sources (evidence: `README.md`)
- Source onboarding → Data access client: accesses data (evidence: `dataswarm/skills/dw-datasource-onboarding/scripts/onboard.py`)
- Flow commands → dwcli web API: uses web API (evidence: `README.md`)
- Metric lifecycle/query → dwcli web API: calls metric-v2 (evidence: `dataswarm/skills/dw-metrics/scripts/metric_query.py`)
- Natural-language SQL → SQL connection layer: uses connection layer
- SQL connection layer → Data access client: delegates data access (evidence: `dataswarm/skills/nl2sql/scripts/_db_client.py`)
- SQL connection layer → dwcli web API: queries service (evidence: `dataswarm/skills/nl2sql/scripts/_db_client.py`)
- Flow commands → Metric lifecycle/query: queries metrics
- Flow commands → Glossary and recall: recalls context
- Warehouse plan → Source discovery: discovers sources (evidence: `dataswarm/skills/dw-warehouse-design/scripts/plan.py`)
- Warehouse plan → Schema inference: infers schema (evidence: `dataswarm/skills/dw-warehouse-design/scripts/plan.py`)
- Warehouse plan → Intent parsing: uses intent (evidence: `dataswarm/skills/dw-warehouse-design/scripts/plan.py`)
- Warehouse plan → Semantic specification: validates specification (evidence: `dataswarm/skills/dw-warehouse-design/scripts/plan.py`)
- Warehouse plan → Plan rendering: renders plan (evidence: `dataswarm/skills/dw-warehouse-design/scripts/plan.py`)
- Plan to tasks → Pipeline execution: dispatches pipeline
- Report resolver → Data access client: reads data (evidence: `dataswarm/skills/dw-report/scripts/resolver.py`)
- Report resolver → Report rendering: prepares report
- Analyst → BI dashboard: explores dashboards (evidence: `README.md`)
- Data access client → Runtime paths: resolves runtime paths (evidence: `dataswarm/skills/_dw_common/scripts/dw_client.py`)
