How dbt-labs/dbt-core works
dbt v2 is a Rust data-transformation product: analysts define SQL models, dbt parses and compiles projects, schedules and runs transformations against warehouses, and supports testing, lineage, and scalable project artifacts for local documentation. The graph includes the adapter execution path, warehouse metadata and relation handling, and time-machine recording/replay as an internal capability. README statements support the broad product workflow; source-index references and excerpts support only selected internal links, so unsampled subsystem connections are omitted.
Main entry points: adapter_engine.rs SQL execution engine · engine.rs Recording engine · adapter_impl.rs Adapter implementation · src Project parsing
Components 25
Project workflow
- SQL models
- Project parsing
crates/dbt-parser/src - Compilation pipeline
crates/dbt-tasks-sa/src/compilation_pipeline.rs - Model scheduling
crates/dbt-compilation/src/schedule.rs - Transformation tests
crates/dbt-tasks-core/src - State service
crates/dbt-state/src/service_client.rs - Task debugging
crates/dbt-tasks-sa/src/debug.rs
Warehouse execution
- Adapter implementation
crates/dbt-adapter/src/adapter/adapter_impl.rs - SQL execution engine
crates/dbt-adapter/src/engine/adapter_engine.rs - Data warehouse
- Warehouse metadata
crates/dbt-adapter/src/metadata
Project insights
- Project artifacts
crates/dbt-schemas/src/schemas - Local documentation
crates/dbt-docs-server/src/server.rs
Adapter internals
- Relations and configs
crates/dbt-adapter/src/relation - SQL type conversion
crates/dbt-adapter/src/sql_types.rs - Record batches
crates/dbt-adapter/src/record_batch.rs - Query cache
crates/dbt-adapter/src/cache.rs
Time machine
- Recording engine
crates/dbt-adapter/src/time_machine/engine.rs - Call events
crates/dbt-adapter/src/time_machine/event.rs - Event recorder
crates/dbt-adapter/src/time_machine/event_recorder.rs - Event replay
crates/dbt-adapter/src/time_machine/event_replay.rs - Event semantics
crates/dbt-adapter/src/time_machine/semantic.rs - Event serialization
crates/dbt-adapter/src/time_machine/serializable_impls.rs - Replay validation
crates/dbt-adapter/src/time_machine/validation.rs
Other components
- Analyst
Connections 26
- Analyst → SQL models: writes
README.md - SQL models → Project parsing: defines project
README.md - Project parsing → Compilation pipeline: feeds compilation
- Compilation pipeline → Model scheduling: schedules models
- Model scheduling → Adapter implementation: runs transformations
- Adapter implementation → SQL execution engine: executes SQL
- SQL execution engine → Data warehouse: queries
crates/dbt-adapter/src/engine/adapter_engine.rs - Warehouse metadata → SQL execution engine: executes probes
crates/dbt-adapter/src/metadata/clickhouse/mod.rs - Adapter implementation → Relations and configs: uses relations
- SQL execution engine → Record batches: processes batches
crates/dbt-adapter/src/engine/adapter_engine.rs - SQL execution engine → Query cache: uses cache
crates/dbt-adapter/src/engine/adapter_engine.rs - SQL execution engine → SQL type conversion: uses type mappings
crates/dbt-adapter/src/engine/adapter_engine.rs - SQL models → Transformation tests: assures quality
README.md - Project parsing → Project artifacts: produces artifacts
- Project artifacts → Local documentation: powers documentation
README.md - Adapter implementation → Recording engine: records or replays
- Recording engine → Event recorder: records calls
crates/dbt-adapter/src/time_machine/engine.rs - Recording engine → Event replay: replays calls
crates/dbt-adapter/src/time_machine/engine.rs - Event recorder → Call events: creates events
crates/dbt-adapter/src/time_machine/event_recorder.rs - Event replay → Call events: matches events
- Call events → Event semantics: uses semantics
crates/dbt-adapter/src/time_machine/event.rs - Event recorder → Event semantics: uses semantics
crates/dbt-adapter/src/time_machine/event_recorder.rs - Recording engine → Replay validation: validates replay
crates/dbt-adapter/src/time_machine/engine.rs - Recording engine → Event serialization: serializes arguments
crates/dbt-adapter/src/time_machine/engine.rs - Model scheduling → State service: uses prior state
- Compilation pipeline → Task debugging: uses debugging
crates/dbt-tasks-sa/src/compilation_pipeline.rs