How cjp2600/protoc-gen-structify works
1. What this repo is
protoc-gen-structify is a Go Protocol Buffers plugin. Its job is to read .proto definitions annotated with structify options and generate typed database-access code, rather than an application runtime or server.
2. Overall flow
The architecture is a one-way code-generation pipeline:
protobuf schema + custom structify options → plugin parses descriptors and model state → provider-specific code generation → Go source files for DB clients, storage/query helpers, constants, and relation handling.
3. Plugin entrypoint
The top-level main.go and plugin/plugin.go form the protoc plugin boundary. This is the executable that protoc invokes, and it delegates into the internal generator logic rather than talking to any database directly.
4. Schema/options layer
plugin/options/structify.proto and the generated structify.pb.go define the custom protobuf annotations used to describe tables, primary keys, foreign keys, relations, and provider selection. These options are the main input contract for generation.
5. Generation core
plugin/generator is the central orchestration layer. It turns protobuf descriptors and parsed metadata into output files, and its tests indicate this is where the main generation semantics are validated.
6. Provider abstraction
plugin/provider/provider.go defines the abstraction for database backends, with concrete implementations for:
- plugin/provider/postgres
- plugin/provider/sqlite
- plugin/provider/clickhouse
This is the main architectural boundary in the repo: database-specific SQL dialect and behavior are isolated behind provider packages.
7. Provider-specific templating
Each provider has its own templater and tmpl directories. The templater packages assemble generation inputs, while the tmpl packages hold the actual code templates for init logic, table code, and condition helpers. This keeps SQL-generation differences localized per database engine.
8. Import/state plumbing
plugin/import manages generated import sets, avoiding duplication and keeping output files consistent. plugin/state stores generation-time context across files/messages, likely tracking resolved tables, relations, and naming decisions while the plugin traverses the schema.
9. Utility and versioning
plugin/pkg/helper contains shared helper logic used across generation, and plugin/pkg/version exposes version metadata. These are supporting packages rather than architectural pillars, but they help stabilize output and tooling behavior.
10. Repository examples as architecture samples
The example/ tree is important because it shows the generated-code boundary in practice. The case_one, case_two, and case_click examples include generated *.db.go files alongside .proto inputs and small Go programs that exercise CRUD, pagination, and relation queries. They demonstrate the expected contract between schema, generator, and application code.
11. Database feature coverage
The README and example layout show that the generated layer supports CRUD, filtering, pagination, transactions, joins, and relation navigation. ClickHouse is treated differently from the row-oriented providers because it supports query settings and PREWHERE, which is reflected in the separate ClickHouse provider implementation.
12. Tests as architecture guardrails
The repo has focused tests around the generator, imports, helper utilities, state management, and each provider. That suggests the system is designed around deterministic code generation, with tests protecting both cross-provider behavior and backend-specific output.
13. What is not in scope
This repo does not appear to include a live database service, HTTP server, or ORM runtime. The generated code is the output artifact; the only runtime dependency is the consumer application that imports the generated package and passes in a database connection.
14. Diagram-friendly boundary summary
For an architecture diagram, the main boxes are: protobuf schema + structify options, protoc plugin entrypoint, generator/state/import orchestration, provider implementations per database, template/rendering layer, and generated Go DB-access code consumed by example apps.
Main entry points: generator.go Generator · provider.go Provider API · structify.proto Options proto · plugin.go Plugin
Components 19
Plugin entrypoint
Protoc plugin boundary and executable entry path.
Generation core
Descriptor parsing, state, imports, and orchestration.
- Generator · Coordinates descriptor analysis and file generation.
plugin/generator/generator.go - Imports · Builds and deduplicates generated import sets.
plugin/import/imports.go - State · Holds cross-file and cross-message generation context.
plugin/state/state.go - Helper · Common helper routines used across generation.
plugin/pkg/helper/helper.go - Version · Build and version metadata surfaced by the plugin.
plugin/pkg/version/version.go
Database providers
Backend-specific generation and SQL behavior.
- Provider API · Abstracts database-specific capabilities behind one contract.
plugin/provider/provider.go - Postgres · PostgreSQL-specific generation and SQL behavior.
plugin/provider/postgres/postgres.go - SQLite · SQLite-specific generation and SQL behavior.
plugin/provider/sqlite/sqlite.go - ClickHouse · ClickHouse-specific generation, including its query constraints.
plugin/provider/clickhouse/clickhouse.go - PG templater · Assembles Postgres generation inputs for templates.
plugin/provider/postgres/templater/init.go - SQLite templater · Assembles SQLite generation inputs for templates.
plugin/provider/sqlite/templater/init.go - CH templater · Assembles ClickHouse generation inputs for templates.
plugin/provider/clickhouse/templater/init.go
Schema contract
Custom protobuf options that drive generation.
- Options proto · Custom annotations for tables, keys, relations, and provider selection.
plugin/options/structify.proto - Options pb · Compiled protobuf bindings for structify options.
plugin/options/structify.pb.go
Examples
Sample generated outputs and consumer apps.
- Case one · End-to-end sample with generated DB code and consumer flows.
example/case_one - Case two · Smaller sample showing generated DB access usage.
example/case_two - Case click · ClickHouse-oriented sample output and usage.
example/case_click
Connections 21
- main.go → Plugin: invokes · Executable entry hands off to the protoc plugin boundary.
- Plugin → Generator: delegates · Plugin forwards parsed protoc input into generation orchestration.
- Options proto → Options pb: compiles to · Schema definitions are compiled into Go bindings.
- Options pb → Generator: drives · Generated annotations inform generator behavior.
- Generator → State: tracks · Generator stores schema-derived context during traversal.
- Generator → Imports: collects · Generator builds file imports as outputs are assembled.
- Generator → Provider API: selects · Generator chooses a backend through the provider abstraction.
- Provider API → Postgres: implements · Postgres is one backend implementation.
- Provider API → SQLite: implements · SQLite is one backend implementation.
- Provider API → ClickHouse: implements · ClickHouse is one backend implementation.
- Postgres → PG templater: uses · Provider-specific logic feeds templating.
- SQLite → SQLite templater: uses · Provider-specific logic feeds templating.
- ClickHouse → CH templater: uses · Provider-specific logic feeds templating.
- Generator → Helper: uses · Shared helpers support generation and naming.
- Generator → Version: reads · Version metadata is surfaced during generation/tooling.
- Generator → Case one: produces · Example package shows generated DB code as output.
- Generator → Case two: produces · Example package shows generated DB code as output.
- Generator → Case click: produces · ClickHouse example shows generated DB code as output.
- Case one → Options proto: consumes · Example schema uses the structify annotations.
- Case two → Options proto: consumes · Example schema uses the structify annotations.
- Case click → Options proto: consumes · Example schema uses the structify annotations.