# bruin-data/bruin architecture

> Architecture diagram of the GitHub repository bruin-data/bruin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bruin-data/bruin
- Repository: https://github.com/bruin-data/bruin

## Overview

Bruin is a Go CLI for building and operating data pipelines: users define pipelines and assets, then validate or execute ingestion, SQL/Python/R transformations, materializations, and quality checks. The graph highlights pipeline loading, execution, and supporting configuration, alongside documented developer features and integrations. The entry-point command registrations and selected internal links are supported by excerpts or the SOURCE INDEX; several runtime relationships are dashed because their implementing code was not sampled.

Main entry points: [`cmd/run.go`](https://github.com/bruin-data/bruin/blob/main/cmd/run.go) (Command Handlers), [`main.go`](https://github.com/bruin-data/bruin/blob/main/main.go) (CLI Entry), [`pkg/executor/sequential.go`](https://github.com/bruin-data/bruin/blob/main/pkg/executor/sequential.go) (Task Executors), [`pkg/lint/query.go`](https://github.com/bruin-data/bruin/blob/main/pkg/lint/query.go) (Query Validation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cli["CLI and Operations"]
  node_entry["CLI Entry<br/>[main.go]"]
  node_commands["Command Handlers<br/>[run.go]"]
  node_backfill["Backfill<br/>[backfill.go]"]
  node_cloud["Cloud Operations<br/>[cloud.go]"]
  node_cloud_service["Bruin Cloud API<br/>[api.go]"]
end

subgraph group_pipeline["Pipeline Lifecycle"]
  node_pipeline_model["Pipeline and Assets<br/>[pipeline.go]"]
  node_config["Config Manager<br/>[manager.go]"]
  node_scheduler["Asset Scheduler<br/>[scheduler.go]"]
end

subgraph group_execution["Execution and Quality"]
  node_executor["Task Executors<br/>[sequential.go]"]
  node_python["Python Runtime<br/>[operator.go]"]
  node_ingestr_install["Ingestr Installer"]
  node_lint["Query Validation<br/>[query.go]"]
  node_quality["Data Quality Checks<br/>[lint.go]"]
end

subgraph group_integrations["Platforms and Ingestion"]
  node_platform_ops["Warehouse Operators<br/>[operator.go]"]
  node_connections["Connection Drivers<br/>[db.go]"]
  node_ingestr["Ingestion Connectors<br/>[operator.go]"]
  node_mongo_query["Mongo Query Runtime<br/>[query.go]"]
end

subgraph group_developer["Developer Features"]
  node_sqlparser["SQL Parsing and Lineage<br/>[engine.go]"]
  node_sqlengine["SQL Dialects<br/>[api.go]"]
  node_semantic["Semantic Engine<br/>[engine.go]"]
  node_tableau["Tableau Integration<br/>[client.go]"]
end

node_user(("Pipeline Operator"))
node_warehouse[("Data Platforms")]
node_ingestion_sources[("External Data Sources")]

node_user -->|"invokes"| node_entry
node_entry -->|"registers commands"| node_commands
node_entry -->|"registers command"| node_backfill
node_entry -->|"registers command"| node_cloud
node_commands -.->|"loads pipeline"| node_pipeline_model
node_pipeline_model -->|"uses configuration"| node_config
node_pipeline_model -.->|"builds scheduled assets"| node_scheduler
node_commands -.->|"runs assets"| node_executor
node_executor -.->|"dispatches Python"| node_python
node_executor -.->|"dispatches transformations"| node_platform_ops
node_executor -.->|"dispatches ingestion"| node_ingestr
node_ingestr -.->|"uses installed runtime"| node_ingestr_install
node_ingestr -.->|"reads from"| node_ingestion_sources
node_platform_ops -.->|"executes against"| node_warehouse
node_commands -.->|"validates assets"| node_lint
node_lint -.->|"checks data quality"| node_quality
node_lint -->|"validates queries"| node_pipeline_model
node_lint -->|"selects validator"| node_connections
node_sqlparser -->|"uses dialect engine"| node_sqlengine
node_commands -.->|"offers lineage tools"| node_sqlparser
node_commands -.->|"offers semantic tools"| node_semantic
node_commands -.->|"supports Tableau import"| node_tableau
node_cloud -->|"calls API"| node_cloud_service
node_mongo_query -.->|"queries MongoDB"| node_warehouse

click node_entry "https://github.com/bruin-data/bruin/blob/main/main.go"
click node_commands "https://github.com/bruin-data/bruin/blob/main/cmd/run.go"
click node_backfill "https://github.com/bruin-data/bruin/blob/main/cmd/backfill.go"
click node_cloud "https://github.com/bruin-data/bruin/blob/main/cmd/cloud.go"
click node_pipeline_model "https://github.com/bruin-data/bruin/blob/main/pkg/pipeline/pipeline.go"
click node_config "https://github.com/bruin-data/bruin/blob/main/pkg/config/manager.go"
click node_scheduler "https://github.com/bruin-data/bruin/blob/main/pkg/scheduler/scheduler.go"
click node_executor "https://github.com/bruin-data/bruin/blob/main/pkg/executor/sequential.go"
click node_python "https://github.com/bruin-data/bruin/blob/main/pkg/python/operator.go"
click node_ingestr_install "https://github.com/bruin-data/bruin/blob/main/pkg/python/ingestr_installer.go"
click node_lint "https://github.com/bruin-data/bruin/blob/main/pkg/lint/query.go"
click node_quality "https://github.com/bruin-data/bruin/blob/main/pkg/lint/lint.go"
click node_platform_ops "https://github.com/bruin-data/bruin/blob/main/pkg/bigquery/operator.go"
click node_connections "https://github.com/bruin-data/bruin/blob/main/pkg/postgres/db.go"
click node_ingestr "https://github.com/bruin-data/bruin/blob/main/pkg/ingestr/operator.go"
click node_sqlparser "https://github.com/bruin-data/bruin/blob/main/pkg/sqlparser/engine.go"
click node_sqlengine "https://github.com/bruin-data/bruin/blob/main/pkg/sqlengine/api.go"
click node_semantic "https://github.com/bruin-data/bruin/blob/main/semantic-engine/engine.go"
click node_tableau "https://github.com/bruin-data/bruin/blob/main/pkg/tableau/client.go"
click node_mongo_query "https://github.com/bruin-data/bruin/blob/main/pkg/mongo/query.go"
click node_cloud_service "https://github.com/bruin-data/bruin/blob/main/pkg/bruincloud/api.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_commands,node_backfill,node_cloud,node_cloud_service toneBlue
class node_pipeline_model,node_config,node_scheduler,node_warehouse,node_ingestion_sources toneAmber
class node_executor,node_python,node_ingestr_install,node_lint,node_quality toneMint
class node_platform_ops,node_connections,node_ingestr,node_mongo_query toneRose
class node_sqlparser,node_sqlengine,node_semantic,node_tableau,node_user toneIndigo
```

## Components

### CLI and Operations

- [CLI Entry](https://github.com/bruin-data/bruin/blob/main/main.go): `main.go`
- [Command Handlers](https://github.com/bruin-data/bruin/blob/main/cmd/run.go): `cmd/run.go`
- [Backfill](https://github.com/bruin-data/bruin/blob/main/cmd/backfill.go): `cmd/backfill.go`
- [Cloud Operations](https://github.com/bruin-data/bruin/blob/main/cmd/cloud.go): `cmd/cloud.go`
- [Bruin Cloud API](https://github.com/bruin-data/bruin/blob/main/pkg/bruincloud/api.go): `pkg/bruincloud/api.go`

### Pipeline Lifecycle

- [Pipeline and Assets](https://github.com/bruin-data/bruin/blob/main/pkg/pipeline/pipeline.go): `pkg/pipeline/pipeline.go`
- [Config Manager](https://github.com/bruin-data/bruin/blob/main/pkg/config/manager.go): `pkg/config/manager.go`
- [Asset Scheduler](https://github.com/bruin-data/bruin/blob/main/pkg/scheduler/scheduler.go): `pkg/scheduler/scheduler.go`

### Execution and Quality

- [Task Executors](https://github.com/bruin-data/bruin/blob/main/pkg/executor/sequential.go): `pkg/executor/sequential.go`
- [Python Runtime](https://github.com/bruin-data/bruin/blob/main/pkg/python/operator.go): `pkg/python/operator.go`
- [Ingestr Installer](https://github.com/bruin-data/bruin/blob/main/pkg/python/ingestr_installer.go): `pkg/python/ingestr_installer.go`
- [Query Validation](https://github.com/bruin-data/bruin/blob/main/pkg/lint/query.go): `pkg/lint/query.go`
- [Data Quality Checks](https://github.com/bruin-data/bruin/blob/main/pkg/lint/lint.go): `pkg/lint/lint.go`

### Platforms and Ingestion

- [Warehouse Operators](https://github.com/bruin-data/bruin/blob/main/pkg/bigquery/operator.go): `pkg/bigquery/operator.go`
- [Connection Drivers](https://github.com/bruin-data/bruin/blob/main/pkg/postgres/db.go): `pkg/postgres/db.go`
- [Ingestion Connectors](https://github.com/bruin-data/bruin/blob/main/pkg/ingestr/operator.go): `pkg/ingestr/operator.go`
- [Mongo Query Runtime](https://github.com/bruin-data/bruin/blob/main/pkg/mongo/query.go): `pkg/mongo/query.go`

### Developer Features

- [SQL Parsing and Lineage](https://github.com/bruin-data/bruin/blob/main/pkg/sqlparser/engine.go): `pkg/sqlparser/engine.go`
- [SQL Dialects](https://github.com/bruin-data/bruin/blob/main/pkg/sqlengine/api.go): `pkg/sqlengine/api.go`
- [Semantic Engine](https://github.com/bruin-data/bruin/blob/main/semantic-engine/engine.go): `semantic-engine/engine.go`
- [Tableau Integration](https://github.com/bruin-data/bruin/blob/main/pkg/tableau/client.go): `pkg/tableau/client.go`

### Other components

- **Pipeline Operator**
- **Data Platforms**
- **External Data Sources**

## Connections

- Pipeline Operator → CLI Entry: invokes (evidence: `README.md`)
- CLI Entry → Command Handlers: registers commands (evidence: `main.go`)
- CLI Entry → Backfill: registers command (evidence: `main.go`)
- CLI Entry → Cloud Operations: registers command (evidence: `main.go`)
- Command Handlers → Pipeline and Assets: loads pipeline
- Pipeline and Assets → Config Manager: uses configuration (evidence: `pkg/config/manager.go`)
- Pipeline and Assets → Asset Scheduler: builds scheduled assets
- Command Handlers → Task Executors: runs assets
- Task Executors → Python Runtime: dispatches Python
- Task Executors → Warehouse Operators: dispatches transformations
- Task Executors → Ingestion Connectors: dispatches ingestion
- Ingestion Connectors → Ingestr Installer: uses installed runtime
- Ingestion Connectors → External Data Sources: reads from
- Warehouse Operators → Data Platforms: executes against
- Command Handlers → Query Validation: validates assets
- Query Validation → Data Quality Checks: checks data quality
- Query Validation → Pipeline and Assets: validates queries (evidence: `pkg/lint/query.go`)
- Query Validation → Connection Drivers: selects validator (evidence: `pkg/lint/query.go`)
- SQL Parsing and Lineage → SQL Dialects: uses dialect engine (evidence: `pkg/sqlparser/engine.go`)
- Command Handlers → SQL Parsing and Lineage: offers lineage tools
- Command Handlers → Semantic Engine: offers semantic tools
- Command Handlers → Tableau Integration: supports Tableau import
- Cloud Operations → Bruin Cloud API: calls API
- Mongo Query Runtime → Data Platforms: queries MongoDB
