How turbot/steampipe works
Main components
• CLI Frontend: Represented by the “cmd” directory. This layer handles commands such as login, query, plugin management, documentation, and service management. It is the entry point for users (see files like root.go, login.go, query.go, etc.).
• Application Core (Business Logic & Configuration): This includes several packages (under “pkg”) for handling configuration (pkg/cmdconfig), connection management (pkg/connection and pkg/connection_sync), query execution (pkg/query), and interactive shell (pkg/interactive). These packages collectively implement the core steps of translating user input into query execution.
• Plugin Framework: The project has dedicated modules for plugins (pkg/plugin) and plugin manager services (pkg/pluginmanager and pkg/pluginmanager_service). In this area, there is integration with GRPC (see directories under pkg/pluginmanager_service/grpc/proto) which plays a role in interacting with plugins as separate logical units.
• Introspection, Export, and Snapshot: Separate components for introspection, exporting query results, and creating snapshots of the query state (pkg/introspection, pkg/export, pkg/snapshot) show that the application also supports diagnostic and data export functionalities.
• External Interfaces: Besides the CLI and core libraries, the system exposes APIs for plugins and potentially other external tools (example: Turbot Pipes integration mentioned in the README).
Relationships and interactions
• How the CLI interacts with the Plugin Framework: the command “steampipe plugin install …” is processed by the core which then interacts with the plugin management service via GRPC. Represent this bidirectional or request–response flow.
• Communication between the core query processing modules (pkg/query) and the Database layer. The query results (or even introspection data) flow back into the CLI.
• That there is additional support for “interactive” querying (pkg/interactive) which might have separate flows for autocomplete and dynamic feedback.
• A layer showing configuration management and how environment variables and config files (managed via pkg/cmdconfig and pkg/steampipeconfig) feed into the system for runtime behavior.
Architectural patterns and design principles
• Modular/Layered Architecture: The system is cleanly broken into CLI, business logic, plugin service, database access, and support services.
• Separation of Concerns: Each package is designed to handle a specific aspect (e.g., connection state management, query execution, plugin management).
• Extensibility: The plugin architecture indicates a design that supports extensibility via external modules, highlighted through GRPC service definitions.
Relevant technologies and frameworks
• Golang is used as the implementation language (evident from Go module files, .go source files, and the Makefile).
• GRPC is used for plugin manager communications.
• PostgreSQL (or an embedded database managed via “db” packages) serves as the query backend.