How k1low/tbls works
Main Components
• CLI Interface / Entry Point:
– Represented by main.go and the cmd/ directory files (e.g., cmd.go, version.go, diff.go, doc.go, etc.).
– This component is responsible for parsing command-line arguments and dispatching the appropriate commands.
• Command Utilities:
– Modules in the cmdutil/ package that provide shared options and helper functions.
• Configuration Management:
– The config/ directory manages .tbls.yml file parsing, validation, and linting rules.
• Database Schema Connectors / Datasources:
– The datasource/ directory handles basic connection logic.
– The drivers/ directory contains multiple subdirectories (bq, clickhouse, dynamodb, mariadb, mssql, mysql, postgres, redshift, etc.) for connecting and handling different database types.
– This layer parses the DSN and executes the schema extraction.
• Schema & Document Processing:
– The schema/ directory encapsulates the extraction, modeling, and transformation of the database schema.
– The ddl/ and dict/ folders indicate additional processing for DDL parsing and dictionary replacements.
• Output Generation:
– Templates for these outputs reside within these folders (often with corresponding template files in the output/<format>/templates).
• Testing & Continuous Integration:
– The testdata/ and testutil/ directories support automated testing and CI.
– The .github folder contains workflows for CI (build, release, integration tests).
Relationships and Interactions
• The CLI (main.go and cmd/) accepts user-input (DSN, configuration options) and then dispatches operations by calling into the configuration module.
• The configuration module reads .tbls.yml (also considering environment variables) and configures settings for data connection, output paths, and linting rules.
• Using the DSN, the datasource and drivers layers establish connections to various databases (or read JSON/HTTP inputs) and extract the schema, resulting in an intermediate representation stored in the schema/ module.
• This intermediate schema is then processed by components in the output/ directory to render files using format-specific templates (Markdown, DOT, etc.). The output is generated either directly or after optional lint and diff checks.
• Testing frameworks in testdata/ and testutil/ verify the correct behavior of infrastructure modules like DDL parsing, templating, and command handling.
• All these components work together, garnering support from shared modules (like cmdutil/) and the CI/CD automation defined in .github workflows.
Architectural Patterns & Design Principles
• Modular design: Each major domain is separated (commands, config, drivers, output, schema, etc.) to allow lightweight maintenance and extensibility.
• Plugin-like extensibility: The drivers folder supports multiple databases with similar interfaces, facilitating extension for other datasources.
• Single-binary distribution: Consolidation of all functionality into one executable for CI ease-of-use.
• Template-based output generation: Customizability via personalized templates (see the personalized templates section in the README) to tailor the generated documentation.
• Command-line utility pattern: Clear separation of sub-commands (doc, diff, lint, coverage, etc.) to maintain simplicity and usability.
Relevant Technologies, Frameworks, and Libraries
• Go programming language for all core implementations.
• GitHub Actions and CI configuration files (e.g., workflows in .github) for continuous integration.
• Docker (Dockerfile and compose.yml) for containerized execution.
• External configuration (YAML via .tbls.yml) and templating for customizable outputs.
• The various Go modules like golangci-lint (via .golangci.yml) and goreleaser for building and releasing.
Elements
• Clear labels for each major component (CLI, Configuration, Datasource, Schema Processor, Output Generator, CI/CD Pipeline, etc.).
• Input flow: Command-line parameters and .tbls.yml configuration.
• Data flow: DSN parsing → Data extraction (via various drivers) → Schema conversion → Documentation output.
• Feedback flow: Diff and lint processing, which reports back to the user and can be integrated with CI pipelines.
• Annotations to indicate design patterns (e.g., plugin architecture in drivers, template-based output generation) and frameworks (e.g., GitHub Actions for CI, Docker for containerization).
– Begin at the user interface (CLI & main.go) which reads configuration (.tbls.yml) and environment variables.
– Show the interconnection with the configuration management module.
– Demonstrate how the tool leverages the datasource and drivers layers to extract and model the database schema.
– Illustrate the processing of the schema into various documentation formats via the output generation modules.
– Include supporting layers for linting, diffing, and continuous integration.
– Finally, mark supporting directories (tests, scripts, CI workflows) that provide additional context for development, testing, and deployment.