# obsessiondb/chkit architecture

> Architecture diagram of the GitHub repository obsessiondb/chkit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/obsessiondb/chkit
- Repository: https://github.com/obsessiondb/chkit

## Overview

chkit is a ClickHouse schema-management toolkit: engineers define schemas in TypeScript, generate and review SQL migrations, apply them, then check migration state and live drift. Its CLI delegates schema modeling and diff planning to the core package and communicates with ClickHouse for migration and inspection. Documented extensions import live schemas, generate types, perform checkpointed backfills, and provide ObsessionDB integration. The repository also contains a separate beta ingestion system with provider pipelines. Several subsystem links are based on README descriptions or indexed references; unsampled implementation details are not asserted.

Main entry points: [`packages/cli/src/commands/generate/command.ts`](https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/generate/command.ts) (CLI commands), [`packages/core/src/model-types.ts`](https://github.com/obsessiondb/chkit/blob/main/packages/core/src/model-types.ts) (Schema DSL), [`packages/cli/src/commands/migrate/command.ts`](https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/migrate/command.ts) (SQL migrations), [`packages/core/src/planner.ts`](https://github.com/obsessiondb/chkit/blob/main/packages/core/src/planner.ts) (Diff planner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Schema workflow"]
  node_cli["CLI commands<br/>[command.ts]"]
  node_schema["Schema DSL<br/>[model-types.ts]"]
  node_config["Config and loading<br/>[schema-loader.ts]"]
  node_planner["Diff planner<br/>[planner.ts]"]
  node_migration["SQL migrations<br/>[command.ts]"]
  node_journal[("Migration journal<br/>[migration-store.ts]")]
  node_checks["Status and checks<br/>[command.ts]"]
  node_drift["Drift comparison<br/>[command.ts]"]
end

subgraph group_clickhouse["ClickHouse runtime"]
  node_chclient["ClickHouse client<br/>[index.ts]"]
  node_introspection["Schema introspection"]
  node_target[("ClickHouse database")]
end

subgraph group_extensions["Schema extensions"]
  node_pull["Pull live schema<br/>[view-parser.ts]"]
  node_codegen["Type codegen<br/>[type-artifacts.ts]"]
  node_backfill["Checkpointed backfill<br/>[planner.ts]"]
  node_obsession["ObsessionDB integration<br/>[index.ts]"]
end

subgraph group_ingestion["Data ingestion"]
  node_ingest["Ingest runtime<br/>[plugin.ts]"]
  node_connectors["Provider pipelines<br/>[pipeline.ts]"]
  node_apis(("Business app APIs"))
end

subgraph group_python["Python support"]
  node_pycli["Python CLI<br/>[main.py]"]
  node_pycore["Python schema engine<br/>[engine.py]"]
end

node_engineer(("Engineer / CI"))

node_engineer -->|"runs commands"| node_cli
node_engineer -->|"defines schema"| node_schema
node_schema -.->|"loads definitions"| node_config
node_cli -.->|"loads configuration"| node_config
node_cli -.->|"generates plan"| node_planner
node_planner -->|"produces SQL"| node_migration
node_engineer -->|"reviews SQL"| node_migration
node_cli -->|"applies migration"| node_migration
node_cli -->|"tracks migration state"| node_journal
node_cli -->|"runs checks"| node_checks
node_checks -->|"checks live drift"| node_drift
node_cli -.->|"executes queries"| node_chclient
node_chclient -.->|"queries database"| node_target
node_introspection -.->|"reads schema"| node_target
node_pull -->|"imports live schema"| node_target
node_schema -->|"generates types"| node_codegen
node_backfill -->|"replays historical data"| node_target
node_obsession -->|"integrates connection"| node_target
node_connectors -->|"defines pipelines"| node_ingest
node_apis -.->|"provides source data"| node_connectors
node_ingest -.->|"loads records"| node_target
node_pycli -.->|"uses schema engine"| node_pycore

click node_cli "https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/generate/command.ts"
click node_schema "https://github.com/obsessiondb/chkit/blob/main/packages/core/src/model-types.ts"
click node_config "https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/runtime/schema-loader.ts"
click node_planner "https://github.com/obsessiondb/chkit/blob/main/packages/core/src/planner.ts"
click node_migration "https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/migrate/command.ts"
click node_journal "https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/runtime/migration-store.ts"
click node_checks "https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/check/command.ts"
click node_drift "https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/drift/command.ts"
click node_chclient "https://github.com/obsessiondb/chkit/blob/main/packages/clickhouse/src/index.ts"
click node_introspection "https://github.com/obsessiondb/chkit/blob/main/packages/clickhouse/src/create-table-parser.ts"
click node_pull "https://github.com/obsessiondb/chkit/blob/main/packages/plugin-pull/src/view-parser.ts"
click node_codegen "https://github.com/obsessiondb/chkit/blob/main/packages/plugin-codegen/src/generators/type-artifacts.ts"
click node_backfill "https://github.com/obsessiondb/chkit/blob/main/packages/plugin-backfill/src/planner.ts"
click node_obsession "https://github.com/obsessiondb/chkit/blob/main/packages/plugin-obsessiondb/src/index.ts"
click node_ingest "https://github.com/obsessiondb/chkit/blob/main/packages/plugin-ingest/src/plugin.ts"
click node_connectors "https://github.com/obsessiondb/chkit/blob/main/registry/attio/pipeline.ts"
click node_pycli "https://github.com/obsessiondb/chkit/blob/main/chkit_python/src/chkit/cli/main.py"
click node_pycore "https://github.com/obsessiondb/chkit/blob/main/chkit_python/src/chkit/core/engine.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_schema,node_config,node_planner,node_migration,node_journal,node_checks,node_drift toneBlue
class node_chclient,node_introspection,node_target toneAmber
class node_pull,node_codegen,node_backfill,node_obsession toneMint
class node_ingest,node_connectors,node_apis toneRose
class node_pycli,node_pycore,node_engineer toneIndigo
```

## Components

### Schema workflow

- [CLI commands](https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/generate/command.ts): `packages/cli/src/commands/generate/command.ts`
- [Schema DSL](https://github.com/obsessiondb/chkit/blob/main/packages/core/src/model-types.ts): `packages/core/src/model-types.ts`
- [Config and loading](https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/runtime/schema-loader.ts): `packages/cli/src/runtime/schema-loader.ts`
- [Diff planner](https://github.com/obsessiondb/chkit/blob/main/packages/core/src/planner.ts): `packages/core/src/planner.ts`
- [SQL migrations](https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/migrate/command.ts): `packages/cli/src/commands/migrate/command.ts`
- [Migration journal](https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/runtime/migration-store.ts): `packages/cli/src/runtime/migration-store.ts`
- [Status and checks](https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/check/command.ts): `packages/cli/src/commands/check/command.ts`
- [Drift comparison](https://github.com/obsessiondb/chkit/blob/main/packages/cli/src/commands/drift/command.ts): `packages/cli/src/commands/drift/command.ts`

### ClickHouse runtime

- [ClickHouse client](https://github.com/obsessiondb/chkit/blob/main/packages/clickhouse/src/index.ts): `packages/clickhouse/src/index.ts`
- [Schema introspection](https://github.com/obsessiondb/chkit/blob/main/packages/clickhouse/src/create-table-parser.ts): `packages/clickhouse/src/create-table-parser.ts`
- **ClickHouse database**

### Schema extensions

- [Pull live schema](https://github.com/obsessiondb/chkit/blob/main/packages/plugin-pull/src/view-parser.ts): `packages/plugin-pull/src/view-parser.ts`
- [Type codegen](https://github.com/obsessiondb/chkit/blob/main/packages/plugin-codegen/src/generators/type-artifacts.ts): `packages/plugin-codegen/src/generators/type-artifacts.ts`
- [Checkpointed backfill](https://github.com/obsessiondb/chkit/blob/main/packages/plugin-backfill/src/planner.ts): `packages/plugin-backfill/src/planner.ts`
- [ObsessionDB integration](https://github.com/obsessiondb/chkit/blob/main/packages/plugin-obsessiondb/src/index.ts): `packages/plugin-obsessiondb/src/index.ts`

### Data ingestion

- [Ingest runtime](https://github.com/obsessiondb/chkit/blob/main/packages/plugin-ingest/src/plugin.ts): `packages/plugin-ingest/src/plugin.ts`
- [Provider pipelines](https://github.com/obsessiondb/chkit/blob/main/registry/attio/pipeline.ts): `registry/attio/pipeline.ts`
- **Business app APIs**

### Python support

- [Python CLI](https://github.com/obsessiondb/chkit/blob/main/chkit_python/src/chkit/cli/main.py): `chkit_python/src/chkit/cli/main.py`
- [Python schema engine](https://github.com/obsessiondb/chkit/blob/main/chkit_python/src/chkit/core/engine.py): `chkit_python/src/chkit/core/engine.py`

### Other components

- **Engineer / CI**

## Connections

- Engineer / CI → CLI commands: runs commands (evidence: `README.md`)
- Engineer / CI → Schema DSL: defines schema (evidence: `README.md`)
- Schema DSL → Config and loading: loads definitions
- CLI commands → Config and loading: loads configuration
- CLI commands → Diff planner: generates plan
- Diff planner → SQL migrations: produces SQL (evidence: `README.md`)
- Engineer / CI → SQL migrations: reviews SQL (evidence: `README.md`)
- CLI commands → SQL migrations: applies migration (evidence: `README.md`)
- CLI commands → Migration journal: tracks migration state (evidence: `README.md`)
- CLI commands → Status and checks: runs checks (evidence: `README.md`)
- Status and checks → Drift comparison: checks live drift (evidence: `README.md`)
- CLI commands → ClickHouse client: executes queries
- ClickHouse client → ClickHouse database: queries database
- Schema introspection → ClickHouse database: reads schema
- Pull live schema → ClickHouse database: imports live schema (evidence: `README.md`)
- Schema DSL → Type codegen: generates types (evidence: `README.md`)
- Checkpointed backfill → ClickHouse database: replays historical data (evidence: `README.md`)
- ObsessionDB integration → ClickHouse database: integrates connection (evidence: `README.md`)
- Provider pipelines → Ingest runtime: defines pipelines (evidence: `registry/attio/pipeline.ts`)
- Business app APIs → Provider pipelines: provides source data
- Ingest runtime → ClickHouse database: loads records
- Python CLI → Python schema engine: uses schema engine