# slothflowlabs/duckle architecture

> Architecture diagram of the GitHub repository slothflowlabs/duckle, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/slothflowlabs/duckle
- Repository: https://github.com/slothflowlabs/duckle

## Overview

Duckle is a self-hosted ETL platform: users author pipelines visually or in SQL/Python, run them locally, then deploy the same pipeline file for scheduled headless execution. The main path runs from workspace editing and persistence through the runner into DuckDB-backed execution and pipeline outputs. Separate capabilities include server operation and scheduling, dbt, AI assistance, Git workflows, and MCP access. README documentation supports these product branches; sampled source confirms selected engine provisioning and runner behavior, but does not establish every internal call path.

Main entry points: [`crates/duckle-runner/src/main.rs`](https://github.com/slothflowlabs/duckle/blob/main/crates/duckle-runner/src/main.rs) (Headless runner), [`frontend/src/App.tsx`](https://github.com/slothflowlabs/duckle/blob/main/frontend/src/App.tsx) (Workflow editor), [`crates/duckdb-engine/src/lib.rs`](https://github.com/slothflowlabs/duckle/blob/main/crates/duckdb-engine/src/lib.rs) (DuckDB engine), [`apps/desktop/src/main.rs`](https://github.com/slothflowlabs/duckle/blob/main/apps/desktop/src/main.rs) (Desktop app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_authoring["Pipeline authoring"]
  node_desktop["Desktop app<br/>[main.rs]"]
  node_editor["Workflow editor<br/>[App.tsx]"]
  node_canvas["Pipeline canvas<br/>[Canvas.tsx]"]
  node_pipeline_types["Pipeline model<br/>[pipeline-types.ts]"]
  node_workspace["Workspace files<br/>[workspace.ts]"]
end

subgraph group_execution["Pipeline execution"]
  node_runner["Headless runner<br/>[main.rs]"]
  node_workflow_engine["Workflow engine<br/>[lib.rs]"]
  node_duckdb_engine["DuckDB engine<br/>[lib.rs]"]
  node_connectors["Data connectors<br/>[lib.rs]"]
end

subgraph group_operations["Deployment and operations"]
  node_scheduler["Scheduling<br/>[lib.rs]"]
  node_server["Server console<br/>[serve.rs]"]
  node_audit["Audit and roles<br/>[audit.rs]"]
  node_deployment["Pipeline bundles"]
end

subgraph group_integrations["Optional integrations"]
  node_dbt["dbt integration<br/>[dbt_engine.rs]"]
  node_ai["Duckie assistant<br/>[llama_chat.rs]"]
  node_git["Git workflow<br/>[workspace_git.rs]"]
  node_mcp["MCP server<br/>[main.rs]"]
end

node_user(("Pipeline author"))
node_agent(("External agent"))
node_data_sources[("Data sources")]
node_data_outputs[("Pipeline outputs")]

node_user -->|"authors in"| node_desktop
node_desktop -->|"opens editor"| node_editor
node_editor -->|"edits graph"| node_canvas
node_canvas -->|"defines nodes"| node_pipeline_types
node_editor -->|"persists pipelines"| node_workspace
node_workspace -->|"supplies pipeline"| node_runner
node_runner -->|"runs workflow"| node_workflow_engine
node_workflow_engine -->|"executes plan"| node_duckdb_engine
node_duckdb_engine -->|"uses connectors"| node_connectors
node_connectors -->|"reads and writes"| node_data_sources
node_duckdb_engine -->|"produces results"| node_data_outputs
node_user -->|"builds artifact"| node_deployment
node_deployment -->|"embeds runner"| node_runner
node_scheduler -->|"triggers runs"| node_runner
node_user -->|"operates pipelines"| node_server
node_server -->|"records activity"| node_audit
node_runner -.->|"runs dbt nodes"| node_dbt
node_user -.->|"requests assistance"| node_ai
node_user -.->|"manages workspace"| node_git
node_agent -.->|"requests tools"| node_mcp
node_mcp -.->|"accesses pipelines"| node_runner

click node_desktop "https://github.com/slothflowlabs/duckle/blob/main/apps/desktop/src/main.rs"
click node_editor "https://github.com/slothflowlabs/duckle/blob/main/frontend/src/App.tsx"
click node_canvas "https://github.com/slothflowlabs/duckle/blob/main/frontend/src/canvas/Canvas.tsx"
click node_pipeline_types "https://github.com/slothflowlabs/duckle/blob/main/frontend/src/pipeline-types.ts"
click node_workspace "https://github.com/slothflowlabs/duckle/blob/main/frontend/src/workspace.ts"
click node_runner "https://github.com/slothflowlabs/duckle/blob/main/crates/duckle-runner/src/main.rs"
click node_workflow_engine "https://github.com/slothflowlabs/duckle/blob/main/crates/workflow-engine/src/lib.rs"
click node_duckdb_engine "https://github.com/slothflowlabs/duckle/blob/main/crates/duckdb-engine/src/lib.rs"
click node_connectors "https://github.com/slothflowlabs/duckle/blob/main/crates/connectors/src/lib.rs"
click node_scheduler "https://github.com/slothflowlabs/duckle/blob/main/crates/scheduler/src/lib.rs"
click node_server "https://github.com/slothflowlabs/duckle/blob/main/crates/duckle-runner/src/serve.rs"
click node_audit "https://github.com/slothflowlabs/duckle/blob/main/crates/duckle-runner/src/audit.rs"
click node_deployment "https://github.com/slothflowlabs/duckle/blob/main/frontend/src/workflow-ui/BuildPipelineModal.tsx"
click node_dbt "https://github.com/slothflowlabs/duckle/blob/main/apps/desktop/src/dbt_engine.rs"
click node_ai "https://github.com/slothflowlabs/duckle/blob/main/apps/desktop/src/llama_chat.rs"
click node_git "https://github.com/slothflowlabs/duckle/blob/main/apps/desktop/src/workspace_git.rs"
click node_mcp "https://github.com/slothflowlabs/duckle/blob/main/crates/duckle-mcp/src/main.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_desktop,node_editor,node_canvas,node_pipeline_types,node_workspace toneBlue
class node_runner,node_workflow_engine,node_duckdb_engine,node_connectors,node_data_sources,node_data_outputs toneAmber
class node_scheduler,node_server,node_audit,node_deployment toneMint
class node_dbt,node_ai,node_git,node_mcp toneRose
class node_user,node_agent toneIndigo
```

## Components

### Pipeline authoring

- [Desktop app](https://github.com/slothflowlabs/duckle/blob/main/apps/desktop/src/main.rs): `apps/desktop/src/main.rs`
- [Workflow editor](https://github.com/slothflowlabs/duckle/blob/main/frontend/src/App.tsx): `frontend/src/App.tsx`
- [Pipeline canvas](https://github.com/slothflowlabs/duckle/blob/main/frontend/src/canvas/Canvas.tsx): `frontend/src/canvas/Canvas.tsx`
- [Pipeline model](https://github.com/slothflowlabs/duckle/blob/main/frontend/src/pipeline-types.ts): `frontend/src/pipeline-types.ts`
- [Workspace files](https://github.com/slothflowlabs/duckle/blob/main/frontend/src/workspace.ts): `frontend/src/workspace.ts`

### Pipeline execution

- [Headless runner](https://github.com/slothflowlabs/duckle/blob/main/crates/duckle-runner/src/main.rs): `crates/duckle-runner/src/main.rs`
- [Workflow engine](https://github.com/slothflowlabs/duckle/blob/main/crates/workflow-engine/src/lib.rs): `crates/workflow-engine/src/lib.rs`
- [DuckDB engine](https://github.com/slothflowlabs/duckle/blob/main/crates/duckdb-engine/src/lib.rs): `crates/duckdb-engine/src/lib.rs`
- [Data connectors](https://github.com/slothflowlabs/duckle/blob/main/crates/connectors/src/lib.rs): `crates/connectors/src/lib.rs`

### Deployment and operations

- [Scheduling](https://github.com/slothflowlabs/duckle/blob/main/crates/scheduler/src/lib.rs): `crates/scheduler/src/lib.rs`
- [Server console](https://github.com/slothflowlabs/duckle/blob/main/crates/duckle-runner/src/serve.rs): `crates/duckle-runner/src/serve.rs`
- [Audit and roles](https://github.com/slothflowlabs/duckle/blob/main/crates/duckle-runner/src/audit.rs): `crates/duckle-runner/src/audit.rs`
- [Pipeline bundles](https://github.com/slothflowlabs/duckle/blob/main/frontend/src/workflow-ui/BuildPipelineModal.tsx): `frontend/src/workflow-ui/BuildPipelineModal.tsx`

### Optional integrations

- [dbt integration](https://github.com/slothflowlabs/duckle/blob/main/apps/desktop/src/dbt_engine.rs): `apps/desktop/src/dbt_engine.rs`
- [Duckie assistant](https://github.com/slothflowlabs/duckle/blob/main/apps/desktop/src/llama_chat.rs): `apps/desktop/src/llama_chat.rs`
- [Git workflow](https://github.com/slothflowlabs/duckle/blob/main/apps/desktop/src/workspace_git.rs): `apps/desktop/src/workspace_git.rs`
- [MCP server](https://github.com/slothflowlabs/duckle/blob/main/crates/duckle-mcp/src/main.rs): `crates/duckle-mcp/src/main.rs`

### Other components

- **Pipeline author**
- **External agent**
- **Data sources**
- **Pipeline outputs**

## Connections

- Pipeline author → Desktop app: authors in
- Desktop app → Workflow editor: opens editor
- Workflow editor → Pipeline canvas: edits graph
- Pipeline canvas → Pipeline model: defines nodes
- Workflow editor → Workspace files: persists pipelines
- Workspace files → Headless runner: supplies pipeline
- Headless runner → Workflow engine: runs workflow
- Workflow engine → DuckDB engine: executes plan
- DuckDB engine → Data connectors: uses connectors
- Data connectors → Data sources: reads and writes
- DuckDB engine → Pipeline outputs: produces results
- Pipeline author → Pipeline bundles: builds artifact
- Pipeline bundles → Headless runner: embeds runner
- Scheduling → Headless runner: triggers runs
- Pipeline author → Server console: operates pipelines
- Server console → Audit and roles: records activity
- Headless runner → dbt integration: runs dbt nodes
- Pipeline author → Duckie assistant: requests assistance
- Pipeline author → Git workflow: manages workspace
- External agent → MCP server: requests tools
- MCP server → Headless runner: accesses pipelines