How bruin-data/bruin works
Bruin is a Go CLI for building and operating data pipelines: users define pipelines and assets, then validate or execute ingestion, SQL/Python/R transformations, materializations, and quality checks. The graph highlights pipeline loading, execution, and supporting configuration, alongside documented developer features and integrations. The entry-point command registrations and selected internal links are supported by excerpts or the SOURCE INDEX; several runtime relationships are dashed because their implementing code was not sampled.
Main entry points: run.go Command Handlers · main.go CLI Entry · sequential.go Task Executors · query.go Query Validation
Components 24
CLI and Operations
- CLI Entry
main.go - Command Handlers
cmd/run.go - Backfill
cmd/backfill.go - Cloud Operations
cmd/cloud.go - Bruin Cloud API
pkg/bruincloud/api.go
Pipeline Lifecycle
- Pipeline and Assets
pkg/pipeline/pipeline.go - Config Manager
pkg/config/manager.go - Asset Scheduler
pkg/scheduler/scheduler.go
Execution and Quality
- Task Executors
pkg/executor/sequential.go - Python Runtime
pkg/python/operator.go - Ingestr Installer
pkg/python/ingestr_installer.go - Query Validation
pkg/lint/query.go - Data Quality Checks
pkg/lint/lint.go
Platforms and Ingestion
- Warehouse Operators
pkg/bigquery/operator.go - Connection Drivers
pkg/postgres/db.go - Ingestion Connectors
pkg/ingestr/operator.go - Mongo Query Runtime
pkg/mongo/query.go
Developer Features
- SQL Parsing and Lineage
pkg/sqlparser/engine.go - SQL Dialects
pkg/sqlengine/api.go - Semantic Engine
semantic-engine/engine.go - Tableau Integration
pkg/tableau/client.go
Other components
- Pipeline Operator
- Data Platforms
- External Data Sources
Connections 24
- Pipeline Operator → CLI Entry: invokes
README.md - CLI Entry → Command Handlers: registers commands
main.go - CLI Entry → Backfill: registers command
main.go - CLI Entry → Cloud Operations: registers command
main.go - Command Handlers → Pipeline and Assets: loads pipeline
- Pipeline and Assets → Config Manager: uses configuration
pkg/config/manager.go - Pipeline and Assets → Asset Scheduler: builds scheduled assets
- Command Handlers → Task Executors: runs assets
- Task Executors → Python Runtime: dispatches Python
- Task Executors → Warehouse Operators: dispatches transformations
- Task Executors → Ingestion Connectors: dispatches ingestion
- Ingestion Connectors → Ingestr Installer: uses installed runtime
- Ingestion Connectors → External Data Sources: reads from
- Warehouse Operators → Data Platforms: executes against
- Command Handlers → Query Validation: validates assets
- Query Validation → Data Quality Checks: checks data quality
- Query Validation → Pipeline and Assets: validates queries
pkg/lint/query.go - Query Validation → Connection Drivers: selects validator
pkg/lint/query.go - SQL Parsing and Lineage → SQL Dialects: uses dialect engine
pkg/sqlparser/engine.go - Command Handlers → SQL Parsing and Lineage: offers lineage tools
- Command Handlers → Semantic Engine: offers semantic tools
- Command Handlers → Tableau Integration: supports Tableau import
- Cloud Operations → Bruin Cloud API: calls API
- Mongo Query Runtime → Data Platforms: queries MongoDB