# airbytehq/airbyte architecture

> Architecture diagram of the GitHub repository airbytehq/airbyte, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/airbytehq/airbyte
- Repository: https://github.com/airbytehq/airbyte

## Overview

Airbyte is an open-source data movement platform. The principal workflow is a configured sync: a source connector checks and discovers available streams, reads records and state, then a destination parses and transforms incoming messages, batches them, and writes them to a warehouse, lake, or database. The tree shows substantial bulk source and destination CDK runtime modules, plus connector-specific integrations. The graph focuses on that runtime path; several links between unsampled stages are marked dashed because the excerpts do not establish their dispatch or call sites.

Main entry points: [`airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/AirbyteConnectorRunner.kt`](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/AirbyteConnectorRunner.kt) (Connector runner), [`airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/pipeline/DataFlowPipeline.kt`](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/pipeline/DataFlowPipeline.kt) (Load pipeline), [`airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/read/ReadOperation.kt`](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/read/ReadOperation.kt) (Source read), [`airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/check/CheckOperation.kt`](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/check/CheckOperation.kt) (Source check)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_base["Connector runtime"]
  node_runner["Connector runner"]
  node_command["Command configuration<br/>[Configuration.kt]"]
  node_output["Protocol output"]
end

subgraph group_extract["Source extraction"]
  node_check["Source check<br/>[CheckOperation.kt]"]
  node_discover["Stream discovery"]
  node_read["Source read<br/>[ReadOperation.kt]"]
  node_state["Sync state<br/>[StateManager.kt]"]
  node_router["Message routing"]
end

subgraph group_load["Destination loading"]
  node_stream["Destination stream model"]
  node_input["Message input"]
  node_record["Raw record conversion"]
  node_pipeline["Load pipeline"]
  node_aggregate["Record aggregation<br/>[Aggregate.kt]"]
  node_schema["Destination schema"]
end

subgraph group_integrations["Destination integrations"]
  node_bigqueryformat["BigQuery file formatting"]
  node_bigquerydb["BigQuery operations"]
  node_bigquerytyping["BigQuery typing"]
end

node_operator(("Sync operator"))
node_platform["Airbyte platform"]
node_warehouse[("Destination store")]

node_operator -->|"configures sync"| node_platform
node_platform -.->|"runs connector"| node_runner
node_runner -.->|"loads configuration"| node_command
node_runner -.->|"dispatches operation"| node_check
node_runner -.->|"dispatches operation"| node_discover
node_runner -.->|"dispatches operation"| node_read
node_check -.->|"emits result"| node_output
node_discover -.->|"emits catalog"| node_output
node_read -.->|"manages state"| node_state
node_read -.->|"routes messages"| node_router
node_router -.->|"transfers records"| node_platform
node_platform -->|"sends sync messages"| node_input
node_input -.->|"provides input"| node_pipeline
node_pipeline -.->|"converts records"| node_record
node_pipeline -.->|"batches records"| node_aggregate
node_pipeline -.->|"uses schema"| node_schema
node_aggregate -.->|"formats batches"| node_bigqueryformat
node_bigqueryformat -.->|"writes files"| node_warehouse
node_bigquerytyping -.->|"executes SQL"| node_bigquerydb
node_bigquerydb -.->|"queries"| node_warehouse

click node_runner "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/AirbyteConnectorRunner.kt"
click node_command "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/command/Configuration.kt"
click node_output "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/output/StdoutOutputConsumer.kt"
click node_check "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/check/CheckOperation.kt"
click node_discover "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/discover/DiscoverOperation.kt"
click node_read "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/read/ReadOperation.kt"
click node_state "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/read/StateManager.kt"
click node_router "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/output/OutputMessageRouter.kt"
click node_stream "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/command/DestinationStream.kt"
click node_input "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/input/DataFlowPipelineInputFlow.kt"
click node_record "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/message/DestinationRecordRaw.kt"
click node_pipeline "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/pipeline/DataFlowPipeline.kt"
click node_aggregate "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/aggregate/Aggregate.kt"
click node_schema "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/schema/TableSchemaFactory.kt"
click node_bigqueryformat "https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/destination-bigquery/src/main/kotlin/io/airbyte/integrations/destination/bigquery/write/bulk_loader/BigQueryObjectStorageFormattingWriter.kt"
click node_bigquerydb "https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/java/airbyte-cdk/datastore-bigquery/src/main/kotlin/io/airbyte/cdk/db/bigquery/BigQueryDatabase.kt"
click node_bigquerytyping "https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/destination-bigquery/src/main/kotlin/io/airbyte/integrations/destination/bigquery/write/typing_deduping/BigQueryDatabaseHandler.kt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_runner,node_command,node_output toneBlue
class node_check,node_discover,node_read,node_state,node_router,node_warehouse toneAmber
class node_stream,node_input,node_record,node_pipeline,node_aggregate,node_schema toneMint
class node_bigqueryformat,node_bigquerydb,node_bigquerytyping toneRose
class node_operator,node_platform toneIndigo
```

## Components

### Connector runtime

- [Connector runner](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/AirbyteConnectorRunner.kt): `airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/AirbyteConnectorRunner.kt`
- [Command configuration](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/command/Configuration.kt): `airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/command/Configuration.kt`
- [Protocol output](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/output/StdoutOutputConsumer.kt): `airbyte-cdk/bulk/core/base/src/main/kotlin/io/airbyte/cdk/output/StdoutOutputConsumer.kt`

### Source extraction

- [Source check](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/check/CheckOperation.kt): `airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/check/CheckOperation.kt`
- [Stream discovery](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/discover/DiscoverOperation.kt): `airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/discover/DiscoverOperation.kt`
- [Source read](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/read/ReadOperation.kt): `airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/read/ReadOperation.kt`
- [Sync state](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/read/StateManager.kt): `airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/read/StateManager.kt`
- [Message routing](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/output/OutputMessageRouter.kt): `airbyte-cdk/bulk/core/extract/src/main/kotlin/io/airbyte/cdk/output/OutputMessageRouter.kt`

### Destination loading

- [Destination stream model](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/command/DestinationStream.kt): `airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/command/DestinationStream.kt`
- [Message input](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/input/DataFlowPipelineInputFlow.kt): `airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/input/DataFlowPipelineInputFlow.kt`
- [Raw record conversion](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/message/DestinationRecordRaw.kt): `airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/message/DestinationRecordRaw.kt`
- [Load pipeline](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/pipeline/DataFlowPipeline.kt): `airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/pipeline/DataFlowPipeline.kt`
- [Record aggregation](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/aggregate/Aggregate.kt): `airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/dataflow/aggregate/Aggregate.kt`
- [Destination schema](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/schema/TableSchemaFactory.kt): `airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/schema/TableSchemaFactory.kt`

### Destination integrations

- [BigQuery file formatting](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/destination-bigquery/src/main/kotlin/io/airbyte/integrations/destination/bigquery/write/bulk_loader/BigQueryObjectStorageFormattingWriter.kt): `airbyte-integrations/connectors/destination-bigquery/src/main/kotlin/io/airbyte/integrations/destination/bigquery/write/bulk_loader/BigQueryObjectStorageFormattingWriter.kt`
- [BigQuery operations](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/java/airbyte-cdk/datastore-bigquery/src/main/kotlin/io/airbyte/cdk/db/bigquery/BigQueryDatabase.kt): `airbyte-cdk/java/airbyte-cdk/datastore-bigquery/src/main/kotlin/io/airbyte/cdk/db/bigquery/BigQueryDatabase.kt`
- [BigQuery typing](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/destination-bigquery/src/main/kotlin/io/airbyte/integrations/destination/bigquery/write/typing_deduping/BigQueryDatabaseHandler.kt): `airbyte-integrations/connectors/destination-bigquery/src/main/kotlin/io/airbyte/integrations/destination/bigquery/write/typing_deduping/BigQueryDatabaseHandler.kt`

### Other components

- **Sync operator**
- **Airbyte platform**
- **Destination store**

## Connections

- Sync operator → Airbyte platform: configures sync (evidence: `README.md`)
- Airbyte platform → Connector runner: runs connector
- Connector runner → Command configuration: loads configuration
- Connector runner → Source check: dispatches operation
- Connector runner → Stream discovery: dispatches operation
- Connector runner → Source read: dispatches operation
- Source check → Protocol output: emits result
- Stream discovery → Protocol output: emits catalog
- Source read → Sync state: manages state
- Source read → Message routing: routes messages
- Message routing → Airbyte platform: transfers records
- Airbyte platform → Message input: sends sync messages (evidence: `README.md`)
- Message input → Load pipeline: provides input
- Load pipeline → Raw record conversion: converts records
- Load pipeline → Record aggregation: batches records
- Load pipeline → Destination schema: uses schema
- Record aggregation → BigQuery file formatting: formats batches
- BigQuery file formatting → Destination store: writes files
- BigQuery typing → BigQuery operations: executes SQL
- BigQuery operations → Destination store: queries
