How clickhouse/clickhouse-go works
Project Type and Purpose
• This is an open-source Go library (“clickhouse-go”) offering two primary interfaces:
– Native TCP protocol client
– database/sql (“std”) client with optional HTTP transport
• It provides connection pooling, encoding/decoding via the ch-go engine, compression, settings, OpenTelemetry, async-insert, and clickhouse-specific features (client-info, external data, etc.).
File Structure
• Top-level API package:
– bind.go, conn_*.go, clickhouse.go, context.go, client_info.go
• Low-level modules under lib/ and lib/proto/:
– lib/binary, lib/column (code-generated column types), lib/proto for network wire formats, lib/timezone, lib/driver
• Example clients: examples/clickhouse_api and examples/std
• Benchmarks: benchmark/v1 & v2
• Tests: end-to-end in tests/, tests/std/, stress tests
• Internal tool: internal/cmd/release (CLI for packaging/release)
• CI/CD and deployment: .github/workflows, .github/cloud/service.tf, Docker compose
Extract Architecture Insights from README
• Two interface layers (Native & database/sql) share common options, settings, compression, and protocol modules.
• Uses ch-go for optimized columnar I/O.
• Supports TCP and optional HTTP transport (over native or HTTP protocols).
• Advanced query features via Context (settings, query parameters, telemetry).
Main Components and Their Relationships
A. API Layer
1. Native API (clickhouse.Open, Conn methods)
2. Std API (clickhouse.OpenDB, implements database/sql/driver)
B. Connection Manager
– Connection pool, open strategy, max idle/open, lifetime
– Context watchdog, ping, profiling
C. Protocol Layer
– TCP handlers: conn_handshake.go, conn_query.go, conn_send_query.go, conn_profile_events.go
– HTTP handlers: conn_http_*.go, HTTP proxy support
D. Encoding/Decoding & Compression
– ch-go subclient (lib/proto, lib/binary) for columnar block format
– Compression modules (LZ4, ZSTD) integrated at block level
– Code-generated column types in lib/column
E. Settings & Client Info
– Settings struct, Query parameters, ClientInfo metadata
F. AsyncInsert & Batch API
– PrepareBatch, AsyncInsert methods, batch.go, conn_batch.go
G. External Dependencies
– ClickHouse server (TCP port 9000, HTTP port 8123/8443)
– TLS stack (crypto/tls), optional HTTP proxy, environment variables
– database/sql package, net.Dialer
Architectural Patterns and Principles
• Layered Abstraction:
– Top: user-friendly API (native vs std)
– Middle: connection and protocol management
– Bottom: binary/columnar encoding + network I/O
• Pluggable Transports: TCP vs HTTP via Protocol interface
• Code-generation for performance (column types)
• Context-driven query controls (telemetry, settings)
• Separation of concerns: examples/ & benchmark/ folders isolated from core library
Detailed Capturing of Essential Elements
• Dual-interface branching (native vs std) early in flow.
• Optional paths (HTTP transport, AsyncInsert path).
• Batch vs immediate query execution.
• External utilities:
– examples/ (usage), tests/ (validation), CI/CD pipelines, Docker compose