# databendlabs/databend architecture

> Architecture diagram of the GitHub repository databendlabs/databend, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/databendlabs/databend
- Repository: https://github.com/databendlabs/databend

## Overview

Databend is a Rust enterprise data warehouse. Its principal workflow is a client submitting SQL through the HTTP query API, followed by query planning and pipeline execution; results are returned through the query interface. The README also describes analytics, vector and full-text search, schema evolution, transactions, branching, and sandboxed Python UDFs. The graph preserves these product areas, but several are represented by their existing source directories because their implementations were not sampled. Links without direct evidence are omitted or marked unverified.

Main entry points: [`src/query/service/src/servers/http/v1/http_query_handlers.rs`](https://github.com/databendlabs/databend/blob/main/src/query/service/src/servers/http/v1/http_query_handlers.rs) (HTTP query API), [`src/query/service/src`](https://github.com/databendlabs/databend/tree/main/src/query/service/src) (Query planning), [`src/query/service/src/pipelines/executor/query_pipeline_executor.rs`](https://github.com/databendlabs/databend/blob/main/src/query/service/src/pipelines/executor/query_pipeline_executor.rs) (Pipeline executor), [`src/meta/api/src/api_impl`](https://github.com/databendlabs/databend/tree/main/src/meta/api/src/api_impl) (Metadata API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_query["SQL query engine"]
  node_http["HTTP query API"]
  node_planning["Query planning"]
  node_pipeline["Pipeline construction"]
  node_pipeline_core["Pipeline graph<br/>[pipeline.rs]"]
  node_executor["Pipeline executor"]
  node_executor_graph["Running graph<br/>[executor_graph.rs]"]
  node_executor_tasks["Worker task queue"]
  node_query_context["Query context"]
end

subgraph group_data["Data and search"]
  node_storage[("Table storage")]
  node_search["Vector and text search"]
end

subgraph group_metadata["Metadata services"]
  node_transactions["Transactions"]
  node_branching["Data branching<br/>[storage.rs]"]
  node_meta_api["Metadata API"]
  node_meta_storage[("Metadata storage<br/>[storage_params.rs]")]
  node_tenant_keys["Tenant keys<br/>[ident.rs]"]
end

subgraph group_agent["Agent execution"]
  node_sandbox_udf["Sandbox UDFs"]
  node_agent_workers["Sandbox workers"]
end

subgraph group_runtime["Runtime support"]
  node_memory["Memory management<br/>[memory_manager.rs]"]
  node_stack_symbols["Stack symbolization<br/>[library_manager.rs]"]
end

node_client(("SQL client"))

node_client -->|"submits SQL"| node_http
node_http -.->|"starts query"| node_planning
node_planning -.->|"builds pipeline"| node_pipeline
node_pipeline -->|"constructs graph"| node_pipeline_core
node_pipeline_core -->|"executes pipeline"| node_executor
node_executor -->|"runs graph"| node_executor_graph
node_executor_graph -->|"schedules tasks"| node_executor_tasks
node_http -->|"uses sessions"| node_query_context
node_planning -.->|"reads data"| node_storage
node_search -.->|"searches data"| node_storage
node_meta_api -.->|"persists metadata"| node_meta_storage
node_meta_api -->|"uses tenant keys"| node_tenant_keys
node_sandbox_udf -.->|"dispatches logic"| node_agent_workers
node_http -.->|"orchestrates with SQL"| node_sandbox_udf
node_memory -.->|"limits resources"| node_executor

click node_http "https://github.com/databendlabs/databend/blob/main/src/query/service/src/servers/http/v1/http_query_handlers.rs"
click node_planning "https://github.com/databendlabs/databend/tree/main/src/query/service/src"
click node_pipeline "https://github.com/databendlabs/databend/blob/main/src/query/service/src/pipelines/pipeline_builder.rs"
click node_pipeline_core "https://github.com/databendlabs/databend/blob/main/src/query/pipeline/src/core/pipeline.rs"
click node_executor "https://github.com/databendlabs/databend/blob/main/src/query/service/src/pipelines/executor/query_pipeline_executor.rs"
click node_executor_graph "https://github.com/databendlabs/databend/blob/main/src/query/service/src/pipelines/executor/executor_graph.rs"
click node_executor_tasks "https://github.com/databendlabs/databend/blob/main/src/query/service/src/pipelines/executor/query_executor_tasks.rs"
click node_query_context "https://github.com/databendlabs/databend/tree/main/src/query/service/src/sessions"
click node_storage "https://github.com/databendlabs/databend/tree/main/src/query/storages"
click node_search "https://github.com/databendlabs/databend/tree/main/src/query"
click node_transactions "https://github.com/databendlabs/databend/tree/main/src/meta/api/src/txn"
click node_branching "https://github.com/databendlabs/databend/blob/main/src/query/ee/src/data_sharing/storage.rs"
click node_meta_api "https://github.com/databendlabs/databend/tree/main/src/meta/api/src/api_impl"
click node_meta_storage "https://github.com/databendlabs/databend/blob/main/src/meta/app-storage/src/storage_params.rs"
click node_tenant_keys "https://github.com/databendlabs/databend/blob/main/src/meta/app/src/tenant_key/ident.rs"
click node_sandbox_udf "https://github.com/databendlabs/databend/tree/main/src/query/script"
click node_memory "https://github.com/databendlabs/databend/blob/main/src/common/base/src/runtime/memory/memory_manager.rs"
click node_stack_symbols "https://github.com/databendlabs/databend/blob/main/src/common/exception/src/elf/library_manager.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_http,node_planning,node_pipeline,node_pipeline_core,node_executor,node_executor_graph,node_executor_tasks,node_query_context,node_client toneBlue
class node_storage,node_search toneAmber
class node_transactions,node_branching,node_meta_api,node_meta_storage,node_tenant_keys toneMint
class node_sandbox_udf,node_agent_workers toneRose
class node_memory,node_stack_symbols toneIndigo
```

## Components

### SQL query engine

- [HTTP query API](https://github.com/databendlabs/databend/blob/main/src/query/service/src/servers/http/v1/http_query_handlers.rs): `src/query/service/src/servers/http/v1/http_query_handlers.rs`
- [Query planning](https://github.com/databendlabs/databend/tree/main/src/query/service/src): `src/query/service/src`
- [Pipeline construction](https://github.com/databendlabs/databend/blob/main/src/query/service/src/pipelines/pipeline_builder.rs): `src/query/service/src/pipelines/pipeline_builder.rs`
- [Pipeline graph](https://github.com/databendlabs/databend/blob/main/src/query/pipeline/src/core/pipeline.rs): `src/query/pipeline/src/core/pipeline.rs`
- [Pipeline executor](https://github.com/databendlabs/databend/blob/main/src/query/service/src/pipelines/executor/query_pipeline_executor.rs): `src/query/service/src/pipelines/executor/query_pipeline_executor.rs`
- [Running graph](https://github.com/databendlabs/databend/blob/main/src/query/service/src/pipelines/executor/executor_graph.rs): `src/query/service/src/pipelines/executor/executor_graph.rs`
- [Worker task queue](https://github.com/databendlabs/databend/blob/main/src/query/service/src/pipelines/executor/query_executor_tasks.rs): `src/query/service/src/pipelines/executor/query_executor_tasks.rs`
- [Query context](https://github.com/databendlabs/databend/tree/main/src/query/service/src/sessions): `src/query/service/src/sessions`

### Data and search

- [Table storage](https://github.com/databendlabs/databend/tree/main/src/query/storages): `src/query/storages`
- [Vector and text search](https://github.com/databendlabs/databend/tree/main/src/query): `src/query`

### Metadata services

- [Transactions](https://github.com/databendlabs/databend/tree/main/src/meta/api/src/txn): `src/meta/api/src/txn`
- [Data branching](https://github.com/databendlabs/databend/blob/main/src/query/ee/src/data_sharing/storage.rs): `src/query/ee/src/data_sharing/storage.rs`
- [Metadata API](https://github.com/databendlabs/databend/tree/main/src/meta/api/src/api_impl): `src/meta/api/src/api_impl`
- [Metadata storage](https://github.com/databendlabs/databend/blob/main/src/meta/app-storage/src/storage_params.rs): `src/meta/app-storage/src/storage_params.rs`
- [Tenant keys](https://github.com/databendlabs/databend/blob/main/src/meta/app/src/tenant_key/ident.rs): `src/meta/app/src/tenant_key/ident.rs`

### Agent execution

- [Sandbox UDFs](https://github.com/databendlabs/databend/tree/main/src/query/script): `src/query/script`
- **Sandbox workers**

### Runtime support

- [Memory management](https://github.com/databendlabs/databend/blob/main/src/common/base/src/runtime/memory/memory_manager.rs): `src/common/base/src/runtime/memory/memory_manager.rs`
- [Stack symbolization](https://github.com/databendlabs/databend/blob/main/src/common/exception/src/elf/library_manager.rs): `src/common/exception/src/elf/library_manager.rs`

### Other components

- **SQL client**

## Connections

- SQL client → HTTP query API: submits SQL (evidence: `README.md`)
- HTTP query API → Query planning: starts query (evidence: `src/query/service/src/servers/http/v1/http_query_handlers.rs`)
- Query planning → Pipeline construction: builds pipeline (evidence: `src/query/service/src/pipelines/pipeline_builder.rs`)
- Pipeline construction → Pipeline graph: constructs graph (evidence: `src/query/service/src/pipelines/pipeline_builder.rs`)
- Pipeline graph → Pipeline executor: executes pipeline (evidence: `src/query/service/src/pipelines/executor/query_pipeline_executor.rs`)
- Pipeline executor → Running graph: runs graph (evidence: `src/query/service/src/pipelines/executor/query_pipeline_executor.rs`)
- Running graph → Worker task queue: schedules tasks (evidence: `src/query/service/src/pipelines/executor/executor_graph.rs`)
- HTTP query API → Query context: uses sessions (evidence: `src/query/service/src/servers/http/v1/http_query_handlers.rs`)
- Query planning → Table storage: reads data
- Vector and text search → Table storage: searches data (evidence: `src/query/storages/orc/src/read_pipeline.rs`)
- Metadata API → Metadata storage: persists metadata
- Metadata API → Tenant keys: uses tenant keys
- Sandbox UDFs → Sandbox workers: dispatches logic
- HTTP query API → Sandbox UDFs: orchestrates with SQL
- Memory management → Pipeline executor: limits resources
