# turbot/steampipe architecture

> Architecture diagram of the GitHub repository turbot/steampipe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/turbot/steampipe
- Repository: https://github.com/turbot/steampipe

## Overview

##### Main components
- CLI Frontend: Represented by the “cmd” directory. This layer handles commands such as login, query, plugin management, documentation, and service management. It is the entry point for users (see files like root.go, login.go, query.go, etc.).
- Application Core (Business Logic & Configuration): This includes several packages (under “pkg”) for handling configuration (pkg/cmdconfig), connection management (pkg/connection and pkg/connection_sync), query execution (pkg/query), and interactive shell (pkg/interactive). These packages collectively implement the core steps of translating user input into query execution.
- Plugin Framework: The project has dedicated modules for plugins (pkg/plugin) and plugin manager services (pkg/pluginmanager and pkg/pluginmanager_service). In this area, there is integration with GRPC (see directories under pkg/pluginmanager_service/grpc/proto) which plays a role in interacting with plugins as separate logical units.
- Introspection, Export, and Snapshot: Separate components for introspection, exporting query results, and creating snapshots of the query state (pkg/introspection, pkg/export, pkg/snapshot) show that the application also supports diagnostic and data export functionalities.
- External Interfaces: Besides the CLI and core libraries, the system exposes APIs for plugins and potentially other external tools (example: Turbot Pipes integration mentioned in the README).

##### Relationships and interactions
- How the CLI interacts with the Plugin Framework: the command “steampipe plugin install …” is processed by the core which then interacts with the plugin management service via GRPC. Represent this bidirectional or request–response flow.
- Communication between the core query processing modules (pkg/query) and the Database layer. The query results (or even introspection data) flow back into the CLI.
- That there is additional support for “interactive” querying (pkg/interactive) which might have separate flows for autocomplete and dynamic feedback.
- A layer showing configuration management and how environment variables and config files (managed via pkg/cmdconfig and pkg/steampipeconfig) feed into the system for runtime behavior.

##### Architectural patterns and design principles
- Modular/Layered Architecture: The system is cleanly broken into CLI, business logic, plugin service, database access, and support services.
- Separation of Concerns: Each package is designed to handle a specific aspect (e.g., connection state management, query execution, plugin management).
- Extensibility: The plugin architecture indicates a design that supports extensibility via external modules, highlighted through GRPC service definitions.

##### Relevant technologies and frameworks
- Golang is used as the implementation language (evident from Go module files, .go source files, and the Makefile).
- GRPC is used for plugin manager communications.
- PostgreSQL (or an embedded database managed via “db” packages) serves as the query backend.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Layer
    subgraph "CLI Layer"
        CLI["CLI Frontend"]:::cli
    end

    %% Core Application Layer
    subgraph "Core Application"
        ConfigMgr["Configuration Manager"]:::core
        ConnMgr["Connection Manager"]:::core
        QueryExec["Query Execution Engine"]:::core
        InteractiveShell["Interactive Shell"]:::core
    end

    %% Plugin Framework Layer
    subgraph "Plugin Framework"
        PluginModules["Plugin Modules"]:::plugin
        PluginMgr["Plugin Manager"]:::plugin
        GRPC["GRPC Integration"]:::plugin
    end

    %% Database Layer
    subgraph "Database Layer"
        DBClient["Database Client"]:::db
    end

    %% Diagnostics & Export Layer
    subgraph "Diagnostics & Export"
        Introspection["Introspection Module"]:::diag
        ExportMod["Export Module"]:::diag
        SnapshotMod["Snapshot Capability"]:::diag
    end

    %% External Integration
    subgraph "External Integration"
        ExternalAPIs["External APIs & Turbot Pipes"]:::external
    end

    %% Relationships and Data Flows
    CLI -->|"input"| ConfigMgr
    CLI -->|"input-query"| QueryExec
    CLI -->|"plugin-cmd"| PluginMgr

    ConfigMgr -->|"config-settings"| ConnMgr
    ConfigMgr -->|"config-settings"| QueryExec

    QueryExec -->|"executes-query"| DBClient
    QueryExec -->|"returns-results"| CLI
    QueryExec -->|"triggers-introspection"| Introspection
    QueryExec -->|"sends-export-data"| ExportMod
    QueryExec -->|"creates-snapshot"| SnapshotMod

    InteractiveShell -->|"dynamic-input"| QueryExec

    PluginModules -->|"integrated-with"| PluginMgr
    PluginMgr -->|"communicates-via"| GRPC
    PluginMgr -->|"interfaces-with"| ExternalAPIs

    %% Styles
    classDef cli fill:#f9c,stroke:#333,stroke-width:2px;
    classDef core fill:#bbf,stroke:#333,stroke-width:2px;
    classDef plugin fill:#bfb,stroke:#333,stroke-width:2px;
    classDef db fill:#ff9,stroke:#333,stroke-width:2px;
    classDef diag fill:#fcf,stroke:#333,stroke-width:2px;
    classDef external fill:#cff,stroke:#333,stroke-width:2px;

    %% Click Events
    click CLI "https://github.com/turbot/steampipe/tree/develop/cmd/"
    click ConfigMgr "https://github.com/turbot/steampipe/tree/develop/pkg/cmdconfig"
    click ConnMgr "https://github.com/turbot/steampipe/tree/develop/pkg/connection"
    click QueryExec "https://github.com/turbot/steampipe/tree/develop/pkg/query"
    click InteractiveShell "https://github.com/turbot/steampipe/tree/develop/pkg/interactive"
    click PluginModules "https://github.com/turbot/steampipe/tree/develop/pkg/plugin"
    click PluginMgr "https://github.com/turbot/steampipe/tree/develop/pkg/pluginmanager"
    click GRPC "https://github.com/turbot/steampipe/tree/develop/pkg/pluginmanager_service/grpc/proto"
    click DBClient "https://github.com/turbot/steampipe/tree/develop/pkg/db"
    click Introspection "https://github.com/turbot/steampipe/tree/develop/pkg/introspection"
    click ExportMod "https://github.com/turbot/steampipe/tree/develop/pkg/export"
    click SnapshotMod "https://github.com/turbot/steampipe/tree/develop/pkg/snapshot"
```
