How heade/airbyte works
Airbyte is a self-hosted data integration platform. Users configure a source, destination, and connection in the web app; the platform schedules a sync, runs source and destination connectors, and transfers records with state for incremental replication. The map highlights the UI, orchestration, connector runtime, and persistence layers, plus optional normalization and analytics. Some subsystem links are documented at product level or strongly implied by the architecture but not shown in sampled callers; these are marked dashed and uncited.
Main entry points: DefaultReplicationWorker.java Replication Worker · handlers Server Handlers · DefaultSynchronousSchedulerClient.java Scheduler Client · connectors Destination Connectors
Components 19
Configuration UI
- Web Application
airbyte-webapp/src/App.tsx - UI Routes
airbyte-webapp/src/pages/routes.tsx
Control Plane
- Server Handlers
airbyte-server/src/main/java/io/airbyte/server/handlers - Scheduler Client
airbyte-scheduler/client/src/main/java/io/airbyte/scheduler/client/DefaultSynchronousSchedulerClient.java - Temporal Jobs
airbyte-workers/src/main/java/io/airbyte/workers/temporal/TemporalClient.java
Connector Runtime
- Replication Worker
airbyte-workers/src/main/java/io/airbyte/workers/DefaultReplicationWorker.java - Worker Utilities
airbyte-workers/src/main/java/io/airbyte/workers/WorkerUtils.java - Connector Entrypoint
airbyte-cdk/python/airbyte_cdk/entrypoint.py - Source CDK
airbyte-cdk/python/airbyte_cdk/sources - Source Connectors
airbyte-integrations/connectors - Destination Connectors
airbyte-integrations/connectors
Data Handling
- Airbyte Protocol
airbyte-cdk/python/airbyte_cdk/models/airbyte_protocol.py - Schema Normalization
airbyte-integrations/bases/base-normalization/normalization
Persistence And Services
- Config Repository
airbyte-config/persistence/src/main/java/io/airbyte/config/persistence/ConfigRepository.java - Job Persistence
airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/JobPersistence.java - Usage Tracking
airbyte-analytics/src/main/java/io/airbyte/analytics/TrackingClientSingleton.java
Other components
- Airbyte User
- Warehouse Or Other Destination
- Source Application Or Database
Connections 19
- Airbyte User → Web Application: configures sync
README.md - Web Application → UI Routes: renders routes
airbyte-webapp/src/App.tsx - UI Routes → Server Handlers: requests platform
- Server Handlers → Config Repository: reads configuration
airbyte-server/src/main/java/io/airbyte/server/handlers/SchedulerHandler.java - Server Handlers → Scheduler Client: submits jobs
- Scheduler Client → Temporal Jobs: submits workflows
airbyte-scheduler/client/src/main/java/io/airbyte/scheduler/client/DefaultSynchronousSchedulerClient.java - Temporal Jobs → Replication Worker: runs sync
- Replication Worker → Worker Utilities: uses utilities
airbyte-workers/src/main/java/io/airbyte/workers/DefaultReplicationWorker.java - Replication Worker → Source CDK: reads source stream
airbyte-workers/src/main/java/io/airbyte/workers/DefaultReplicationWorker.java - Replication Worker → Destination Connectors: sends records
airbyte-workers/src/main/java/io/airbyte/workers/DefaultReplicationWorker.java - Source Connectors → Connector Entrypoint: launches through
airbyte-integrations/connectors/source-amazon-seller-partner/main.py - Connector Entrypoint → Source CDK: invokes source
airbyte-cdk/python/airbyte_cdk/entrypoint.py - Source Application Or Database → Source Connectors: provides records
- Source CDK → Source Application Or Database: reads data
- Destination Connectors → Warehouse Or Other Destination: writes records
- Replication Worker → Airbyte Protocol: transfers messages
airbyte-workers/src/main/java/io/airbyte/workers/DefaultReplicationWorker.java - Destination Connectors → Schema Normalization: may normalize
- Scheduler Client → Job Persistence: tracks jobs
- Server Handlers → Usage Tracking: may report usage