# crate/dlt-cratedb architecture

> Architecture diagram of the GitHub repository crate/dlt-cratedb, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/crate/dlt-cratedb
- Repository: https://github.com/crate/dlt-cratedb

## Overview

1. This repository is a Python package, `dlt-cratedb`, that provides a **CrateDB destination adapter for dlt**. It is explicitly described as a temporary packaging layer until the implementation can be upstreamed into dlt itself.

2. The main runtime boundary is the adapter layer under `src/dlt_cratedb/impl/cratedb`. That package contains the CrateDB-specific implementation pieces: configuration, the destination class, a factory, a SQL client, and utility helpers. This is the core path an engineer would diagram for “dlt → CrateDB”.

3. The adapter is **heavily based on dlt’s postgres adapter**. The README says the `CrateDbSqlClient` diverges from the original psycopg2-based SQL client only where CrateDB needs special handling. Architecturally, this means the repo is an adaptation/override layer rather than a full standalone destination stack.

4. `configuration.py` represents the external configuration boundary. This is where destination-specific settings are defined and validated before being consumed by the rest of the adapter, so it sits between user-provided dlt settings and the SQL execution layer.

5. `cratedb.py` and `factory.py` are the adapter construction and integration points. These files likely assemble the destination object, wire in CrateDB-specific components, and expose the entry points dlt expects when selecting a destination implementation.

6. `sql_client.py` is the operational heart of the repo. It is the component that turns dlt’s abstract loading and schema operations into concrete SQL behavior against CrateDB, with `utils.py` supporting SQL or dialect-specific transformations.

7. The `src/dlt_cratedb/patch` package indicates a second architectural boundary: **compatibility patches** applied to dlt or the inherited postgres behavior. This is a common pattern when a destination needs to override behavior without forking the entire upstream adapter.

8. The package includes `py.typed`, so the public code is intended to be type-checked and consumed as typed Python. That matters for architecture because it signals the project is designed as a reusable library, not just a private integration script.

9. The test architecture is split between unit/common test helpers and **load/integration tests**. Under `tests/load/cratedb`, the repository uses a `compose.yml`, which strongly suggests containerized CrateDB integration tests rather than pure mocks for the end-to-end path.

10. The test suite also contains shared cases and normalizers under `tests/common`, implying reusable assertion logic for schema or SQL normalization across multiple destination behaviors. That usually means the adapter is being verified against a broader dlt destination contract, not just isolated functions.

11. CI/CD is handled through GitHub Actions: `tests.yml` for validation and `release-pypi.yml` for publishing. The repository also includes `codecov.yml` and Dependabot config, so the operational boundary includes automated testing, coverage reporting, dependency maintenance, and package release.

12. In diagram terms, the cleanest architecture is: **dlt application/configuration → adapter factory → CrateDB destination object → CrateDB SQL client/patch layer → CrateDB server**, with tests exercising both internal helpers and a Docker-based live database path.

Main entry points: [`src/dlt_cratedb/impl/cratedb/__init__.py`](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/__init__.py) (cratedb impl), [`src/dlt_cratedb/impl/cratedb/sql_client.py`](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/sql_client.py) (sql client), [`tests/load/cratedb/test_cratedb_client.py`](https://github.com/crate/dlt-cratedb/blob/main/tests/load/cratedb/test_cratedb_client.py) (live tests), [`.github/workflows/tests.yml`](https://github.com/crate/dlt-cratedb/blob/main/.github/workflows/tests.yml) (tests workflow)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_grp_src["Adapter"]
  node_n_pkg["dlt-cratedb<br/>[__init__.py]"]
  node_n_implpkg["impl<br/>[__init__.py]"]
  node_n_cratedb_pkg["cratedb impl<br/>[__init__.py]"]
  node_n_config["config<br/>boundary<br/>[configuration.py]"]
  node_n_factory["factory<br/>assembly<br/>[factory.py]"]
  node_n_destination["destination<br/>integration<br/>[cratedb.py]"]
  node_n_sqlclient["sql client<br/>execution<br/>[sql_client.py]"]
  node_n_utils["utils<br/>dialect helpers<br/>[utils.py]"]
end

subgraph group_grp_patch["Compatibility"]
  node_n_patchpkg["patch<br/>compatibility<br/>[__init__.py]"]
  node_n_patch_utils["patch utils<br/>compatibility helpers<br/>[utils.py]"]
end

subgraph group_grp_tests["Tests"]
  node_n_common_cases["shared cases<br/>test support<br/>[__init__.py]"]
  node_n_norm_sql["sql upper<br/>normalizer<br/>[sql_upper.py]"]
  node_n_norm_title["title case<br/>normalizer<br/>[title_case.py]"]
  node_n_common_cfg["cfg utils<br/>test support<br/>[utils.py]"]
  node_n_live_tests["live tests<br/>integration tests"]
  node_n_compose["compose<br/>test environment<br/>[compose.yml]"]
  node_n_table_builder["table builder<br/>integration tests"]
end

subgraph group_grp_ci["Automation"]
  node_n_tests_wf["tests workflow<br/>ci<br/>[tests.yml]"]
  node_n_release_wf["release workflow<br/>ci<br/>[release-pypi.yml]"]
  node_n_dependabot["dependabot<br/>dependency automation<br/>[dependabot.yml]"]
  node_n_codecov["codecov<br/>coverage<br/>[codecov.yml]"]
end

node_n_dlt_core(("dlt core<br/>upstream dependency"))
node_n_pg_adapter(("postgres adapter<br/>upstream dependency"))
node_n_cratedb_server[("CrateDB<br/>database")]

node_n_pkg -->|"contains"| node_n_implpkg
node_n_implpkg -->|"contains"| node_n_cratedb_pkg
node_n_cratedb_pkg -->|"defines"| node_n_config
node_n_cratedb_pkg -->|"assembles"| node_n_factory
node_n_cratedb_pkg -->|"exposes"| node_n_destination
node_n_cratedb_pkg -->|"operates"| node_n_sqlclient
node_n_cratedb_pkg -->|"supports"| node_n_utils
node_n_factory -->|"builds"| node_n_destination
node_n_destination -->|"uses"| node_n_sqlclient
node_n_sqlclient -->|"calls"| node_n_utils
node_n_config -->|"adapts"| node_n_dlt_core
node_n_destination -->|"integrates"| node_n_dlt_core
node_n_sqlclient -->|"inherits from"| node_n_pg_adapter
node_n_patchpkg -->|"contains"| node_n_patch_utils
node_n_patch_utils -->|"patches"| node_n_pg_adapter
node_n_sqlclient -->|"executes against"| node_n_cratedb_server
node_n_common_cases -->|"includes"| node_n_norm_sql
node_n_common_cases -->|"includes"| node_n_norm_title
node_n_live_tests -->|"uses"| node_n_compose
node_n_live_tests -->|"verifies"| node_n_cratedb_server
node_n_live_tests -->|"covers"| node_n_sqlclient
node_n_table_builder -->|"reuses"| node_n_common_cases
node_n_live_tests -->|"reuses"| node_n_common_cfg
node_n_tests_wf -->|"runs"| node_n_live_tests
node_n_tests_wf -->|"runs"| node_n_table_builder
node_n_tests_wf -->|"reports"| node_n_codecov
node_n_release_wf -->|"publishes"| node_n_pkg
node_n_dependabot -->|"supports"| node_n_tests_wf

click node_n_pkg "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/__init__.py"
click node_n_implpkg "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/__init__.py"
click node_n_cratedb_pkg "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/__init__.py"
click node_n_config "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/configuration.py"
click node_n_factory "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/factory.py"
click node_n_destination "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/cratedb.py"
click node_n_sqlclient "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/sql_client.py"
click node_n_utils "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/utils.py"
click node_n_patchpkg "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/patch/__init__.py"
click node_n_patch_utils "https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/patch/utils.py"
click node_n_common_cases "https://github.com/crate/dlt-cratedb/blob/main/tests/common/cases/__init__.py"
click node_n_norm_sql "https://github.com/crate/dlt-cratedb/blob/main/tests/common/cases/normalizers/sql_upper.py"
click node_n_norm_title "https://github.com/crate/dlt-cratedb/blob/main/tests/common/cases/normalizers/title_case.py"
click node_n_common_cfg "https://github.com/crate/dlt-cratedb/blob/main/tests/common/configuration/utils.py"
click node_n_live_tests "https://github.com/crate/dlt-cratedb/blob/main/tests/load/cratedb/test_cratedb_client.py"
click node_n_compose "https://github.com/crate/dlt-cratedb/blob/main/tests/load/cratedb/compose.yml"
click node_n_table_builder "https://github.com/crate/dlt-cratedb/blob/main/tests/load/cratedb/test_cratedb_table_builder.py"
click node_n_tests_wf "https://github.com/crate/dlt-cratedb/blob/main/.github/workflows/tests.yml"
click node_n_release_wf "https://github.com/crate/dlt-cratedb/blob/main/.github/workflows/release-pypi.yml"
click node_n_dependabot "https://github.com/crate/dlt-cratedb/blob/main/.github/dependabot.yml"
click node_n_codecov "https://github.com/crate/dlt-cratedb/blob/main/codecov.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_pkg,node_n_implpkg,node_n_cratedb_pkg,node_n_config,node_n_factory,node_n_destination,node_n_sqlclient,node_n_utils toneBlue
class node_n_patchpkg,node_n_patch_utils toneAmber
class node_n_common_cases,node_n_norm_sql,node_n_norm_title,node_n_common_cfg,node_n_live_tests,node_n_compose,node_n_table_builder toneMint
class node_n_tests_wf,node_n_release_wf,node_n_dependabot,node_n_codecov toneRose
class node_n_dlt_core,node_n_pg_adapter,node_n_cratedb_server toneNeutral
```

## Components

### Adapter

Core CrateDB destination implementation

- [dlt-cratedb](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/__init__.py): package · Typed Python package that packages the CrateDB destination adapter · `src/dlt_cratedb/__init__.py`
- [impl](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/__init__.py): package · Internal implementation namespace for destination-specific code · `src/dlt_cratedb/impl/__init__.py`
- [cratedb impl](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/__init__.py): package · CrateDB adapter implementation boundary · `src/dlt_cratedb/impl/cratedb/__init__.py`
- [config](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/configuration.py): boundary · Destination configuration definitions and validation · `src/dlt_cratedb/impl/cratedb/configuration.py`
- [factory](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/factory.py): assembly · Constructs the destination object and wires CrateDB components · `src/dlt_cratedb/impl/cratedb/factory.py`
- [destination](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/cratedb.py): integration · CrateDB destination entry point exposed to dlt · `src/dlt_cratedb/impl/cratedb/cratedb.py`
- [sql client](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/sql_client.py): execution · Executes dlt load and schema operations as CrateDB SQL · `src/dlt_cratedb/impl/cratedb/sql_client.py`
- [utils](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/impl/cratedb/utils.py): dialect helpers · CrateDB-specific SQL and transformation helpers · `src/dlt_cratedb/impl/cratedb/utils.py`

### Compatibility

Overrides and compatibility helpers layered on upstream dlt behavior

- [patch](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/patch/__init__.py): compatibility · Compatibility boundary for patched upstream behavior · `src/dlt_cratedb/patch/__init__.py`
- [patch utils](https://github.com/crate/dlt-cratedb/blob/main/src/dlt_cratedb/patch/utils.py): compatibility helpers · Overrides or adapts inherited postgres/dlt behavior · `src/dlt_cratedb/patch/utils.py`

### Tests

Shared cases plus live CrateDB integration coverage

- [shared cases](https://github.com/crate/dlt-cratedb/blob/main/tests/common/cases/__init__.py): test support · Reusable assertions and normalization cases for destination behavior · `tests/common/cases/__init__.py`
- [sql upper](https://github.com/crate/dlt-cratedb/blob/main/tests/common/cases/normalizers/sql_upper.py): normalizer · Shared SQL normalization helper for tests · `tests/common/cases/normalizers/sql_upper.py`
- [title case](https://github.com/crate/dlt-cratedb/blob/main/tests/common/cases/normalizers/title_case.py): normalizer · Shared title-case normalization helper for tests · `tests/common/cases/normalizers/title_case.py`
- [cfg utils](https://github.com/crate/dlt-cratedb/blob/main/tests/common/configuration/utils.py): test support · Reusable configuration utilities for test setup · `tests/common/configuration/utils.py`
- [live tests](https://github.com/crate/dlt-cratedb/blob/main/tests/load/cratedb/test_cratedb_client.py): integration tests · End-to-end adapter tests against a containerized CrateDB · `tests/load/cratedb/test_cratedb_client.py`
- [compose](https://github.com/crate/dlt-cratedb/blob/main/tests/load/cratedb/compose.yml): test environment · Docker Compose service definition for live CrateDB testing · `tests/load/cratedb/compose.yml`
- [table builder](https://github.com/crate/dlt-cratedb/blob/main/tests/load/cratedb/test_cratedb_table_builder.py): integration tests · Load-path verification for table construction behavior · `tests/load/cratedb/test_cratedb_table_builder.py`

### Automation

Validation, coverage, dependency maintenance, and release workflows

- [tests workflow](https://github.com/crate/dlt-cratedb/blob/main/.github/workflows/tests.yml): ci · GitHub Actions validation pipeline · `.github/workflows/tests.yml`
- [release workflow](https://github.com/crate/dlt-cratedb/blob/main/.github/workflows/release-pypi.yml): ci · GitHub Actions publishing pipeline for PyPI releases · `.github/workflows/release-pypi.yml`
- [dependabot](https://github.com/crate/dlt-cratedb/blob/main/.github/dependabot.yml): dependency automation · Automated dependency update configuration · `.github/dependabot.yml`
- [codecov](https://github.com/crate/dlt-cratedb/blob/main/codecov.yml): coverage · Coverage reporting configuration · `codecov.yml`

### Other components

- **dlt core**: upstream dependency · Upstream dlt application and destination abstractions
- **postgres adapter**: upstream dependency · Upstream postgres-based destination behavior reused by this package
- **CrateDB**: database · Target database server for the adapter

## Connections

- dlt-cratedb → impl: contains · Top-level package exposes internal implementation namespace
- impl → cratedb impl: contains · CrateDB destination implementation lives here
- cratedb impl → config: defines · Destination configuration boundary
- cratedb impl → factory: assembles · Factory wires the destination components
- cratedb impl → destination: exposes · Destination entry point for dlt selection
- cratedb impl → sql client: operates · SQL client performs the runtime database work
- cratedb impl → utils: supports · Dialect-specific helpers shared by adapter code
- factory → destination: builds · Factory constructs the destination object
- destination → sql client: uses · Destination delegates execution to the SQL client
- sql client → utils: calls · Client relies on CrateDB-specific transformations
- config → dlt core: adapts · Configuration aligns with dlt destination settings
- destination → dlt core: integrates · Adapter plugs into dlt's destination model
- sql client → postgres adapter: inherits from · Implementation is based on the postgres adapter
- patch → patch utils: contains · Compatibility helpers are isolated under patch namespace
- patch utils → postgres adapter: patches · Overrides inherited postgres behavior where CrateDB differs
- sql client → CrateDB: executes against · SQL client talks to the CrateDB backend
- shared cases → sql upper: includes · Shared SQL normalization case
- shared cases → title case: includes · Shared title-case normalization case
- live tests → compose: uses · Integration test runtime is containerized
- live tests → CrateDB: verifies · Exercises the live database path
- live tests → sql client: covers · Primary runtime path under test
- table builder → shared cases: reuses · Table-building tests share common destination assertions
- live tests → cfg utils: reuses · Tests share configuration helpers
- tests workflow → live tests: runs · CI validates the live adapter tests
- tests workflow → table builder: runs · CI validates table-building behavior
- tests workflow → codecov: reports · Validation feeds coverage reporting
- release workflow → dlt-cratedb: publishes · Release workflow ships the typed package
- dependabot → tests workflow: supports · Dependency automation keeps validation current
