# ashish10alex/vscode-dataform-tools architecture

> Architecture diagram of the GitHub repository ashish10alex/vscode-dataform-tools, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ashish10alex/vscode-dataform-tools
- Repository: https://github.com/ashish10alex/vscode-dataform-tools

## Overview

##### Project type and purpose
- This is a VS Code extension—“Dataform tools”—for authoring and running Dataform `.sqlx` projects.
- It provides LSP‑like features (hover, completions, diagnostics, go‑to‑definition), command handlers (run file/tag, cost estimate, format), and rich WebView UIs (compiled query, dependency graph, preview results).
- There is also a standalone Next.js “website” folder that hosts documentation.

##### File structure
- Top‑level “src/” contains the extension’s entrypoint (extension.ts) plus providers (hover, definition, codeLens, codeAction, completions), command implementations (bigqueryDryRun, runFiles, runTag, costEstimator, formatCurrentFile), parsers (sqlxFileParser, dependencyTree), and utilities (logger, bigqueryClient).
- “views/” and “webviews/” hold React/Vite apps for WebView panels (dependency graph, compiled query, query results).
- “media/” supplies static assets for the README.
- “website/” is a Next.js app (React/TS) for marketing and docs, entirely separate at runtime.
- Configuration/build files include tsconfig.json, vite.config.ts (bundles WebViews), package.json, tailwind.config.js.

##### Extract insights from the README
- Core features map 1:1 with providers and commands in “src/”: compilation/dry‑run stats, inline diagnostics, dependency graph, preview results, code generation, cost estimator, formatting via sqlfluff CLI, BigQuery hovers, BigQuery snippets.
- External prerequisites: Dataform CLI, gcloud SDK, sqlfluff, optional Error Lens.

a. Main components
- VS Code Editor (UI shell)
- Extension Host (Node/TypeScript runtime)
– Core modules: command handlers (runQuery, runFile, runTag, costEstimator, formatDocument)
– Language providers: hoverProvider, definitionProvider, completionProvider, codeActionProvider, codeLensProvider, diagnostics (setDiagnostics)
– Parsers & logic: sqlxFileParser, dependencyTree, costEstimator, getLineageMetadata
– Utilities: bigqueryClient (calls BigQuery API), logger
– External CLI invocations: Dataform CLI (compile), sqlfluff CLI (format)
- WebView UIs (embedded React apps)
– Compiled query panel, dependency graph panel, query results panel
- External services
– Dataform CLI (spawned child process)
– Google BigQuery API (via gcloud or SDK)
– SQLFluff (spawned Python CLI)
- File system
– User workspace: `.sqlx` files, JS modules, Dataform project config
– Extension’s internal test workspace under src/test
- Documentation Website (Next.js)

b. Relationships & data flows
1. User triggers a command in VS Code → VS Code API → Extension Host
2. Extension Host → Dataform CLI (spawn child_process) → receives compiled SQL & dry‑run stats
3. Extension Host → BigQuery API (via bigqueryClient/gcloud) → receives query results & cost estimates
4. Extension Host → WebView panel (postMessage) → React UI renders data
5. On `.sqlx` edit or hover → VS Code API → respective provider module → uses parser & metadata → returns completions/hover/diagnostics
6. Format command → Extension Host → SQLFluff CLI → applies edits via VS Code TextEdit
7. Tests drive extension code using vscode-test harness

c. Architectural patterns & principles
- Modular “Command + Provider” design adhering to VS Code Extension API conventions
- Clear separation: core logic (command handlers & providers) vs. UI (WebViews)
- Event‑driven: VS Code events trigger extension reactions
- Micro‑frontend approach for WebView panels (each React app isolated & bundled via Vite)
- “Backend” logic in-process; external CLIs and cloud APIs treated as services

d. Key technologies & frameworks
- TypeScript, VS Code Extension API
- React + Vite for WebView UIs
- Next.js for documentation website
- Dataform CLI (Node child_process)
- Google Cloud SDK / BigQuery client
- SQLFluff for formatting
- Mocha + vscode-test for integration tests

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layers
    VSCode_UI["VS Code Editor UI"]:::editor

    subgraph "Extension Host"
        subgraph "Entrypoint & Core"
            Entrypoint["extension.ts"]:::core
            click Entrypoint "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/extension.ts"
        end

        subgraph "Command Handlers"
            RunFiles["runFiles"]:::core
            click RunFiles "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/runFiles.ts"
            RunTag["runTag"]:::core
            click RunTag "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/runTag.ts"
            DryRun["bigqueryDryRun"]:::core
            click DryRun "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/bigqueryDryRun.ts"
            RunQuery["bigqueryRunQuery"]:::core
            click RunQuery "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/bigqueryRunQuery.ts"
            CostEst["costEstimator"]:::core
            click CostEst "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/costEstimator.ts"
            FormatFile["formatCurrentFile"]:::core
            click FormatFile "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/formatCurrentFile.ts"
            PreviewResults["previewQueryResults"]:::core
            click PreviewResults "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/previewQueryResults.ts"
        end

        subgraph "Language Providers"
            HoverP["hoverProvider"]:::core
            click HoverP "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/hoverProvider.ts"
            DefP["definitionProvider"]:::core
            click DefP "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/definitionProvider.ts"
            CompP["completions"]:::core
            click CompP "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/completions.ts"
            CodeActionP["codeActionProvider"]:::core
            click CodeActionP "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/codeActionProvider.ts"
            CodeLensP["codeLensProvider"]:::core
            click CodeLensP "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/codeLensProvider.ts"
            Diagnostics["setDiagnostics"]:::core
            click Diagnostics "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/setDiagnostics.ts"
        end

        subgraph "Parsers & Metadata"
            Parser["sqlxFileParser"]:::core
            click Parser "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/sqlxFileParser.ts"
            DepTree["dependancyTree"]:::core
            click DepTree "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/dependancyTree.ts"
            DepMeta["dependancyTreeNodeMeta"]:::core
            click DepMeta "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/dependancyTreeNodeMeta.ts"
            Lineage["getLineageMetadata"]:::core
            click Lineage "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/getLineageMetadata.ts"
        end

        subgraph "Utilities & Clients"
            BQClient["bigqueryClient"]:::core
            click BQClient "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/bigqueryClient.ts"
            Logger["logger"]:::core
            click Logger "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/logger.ts"
            Utils["utils/constants"]:::core
            click Utils "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/utils.ts"
            click Utils "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/constants.ts"
        end
    end

    subgraph "WebView UIs"
        DGPanel["Dependency Graph Panel"]:::webview
        click DGPanel "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/webviews/dependancy_graph/index.tsx"
        CompiledPanel["Compiled Query Panel"]:::webview
        click CompiledPanel "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/views/register-preview-compiled-panel.ts"
        ResultsPanel["Query Results Panel"]:::webview
        click ResultsPanel "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/views/register-query-results-panel.ts"
        SidebarPanel["Sidebar Panel"]:::webview
        click SidebarPanel "https://github.com/ashish10alex/vscode-dataform-tools/blob/main/src/views/register-sidebar-panel.ts"
    end

    subgraph "External Services"
        DFCLI["Dataform CLI"]:::externalService
        BQAPI["BigQuery API"]:::externalService
        SQLFluff["SQLFluff CLI"]:::externalService
    end

    FileSystem["File System / Workspace"]:::fs
    DocsWebsite["Documentation Website (Next.js)"]:::website
    click DocsWebsite "https://github.com/ashish10alex/vscode-dataform-tools/tree/main/website/"

    %% Connections
    VSCode_UI -->|"VS Code API calls"| Entrypoint
    Entrypoint --> RunFiles
    Entrypoint --> RunTag
    Entrypoint --> DryRun
    Entrypoint --> RunQuery
    Entrypoint --> CostEst
    Entrypoint --> FormatFile
    Entrypoint --> PreviewResults

    VSCode_UI -->|"edit/hover/events"| HoverP
    HoverP --> Parser
    DefP --> Parser
    CompP --> Parser
    CodeActionP --> Parser
    CodeLensP --> Parser
    Diagnostics --> Parser

    RunFiles --> DFCLI
    RunTag --> DFCLI
    DryRun --> DFCLI
    DryRun --> BQAPI
    RunQuery --> BQAPI
    CostEst --> BQAPI
    FormatFile --> SQLFluff
    PreviewResults --> BQAPI

    Entrypoint --> BQClient
    Entrypoint --> Logger
    Parser --> DepTree
    Parser --> DepMeta
    Parser --> Lineage

    Entrypoint -->|"spawn WebView"| DGPanel
    Entrypoint -->|"spawn WebView"| CompiledPanel
    Entrypoint -->|"spawn WebView"| ResultsPanel
    Entrypoint -->|"spawn WebView"| SidebarPanel

    Entrypoint --> FileSystem
    Parser -.-> FileSystem

    %% Styles
    classDef editor fill:#D0E8FF,stroke:#0366D6;
    classDef core fill:#BDE0FE,stroke:#023E8A;
    classDef webview fill:#D4EDDA,stroke:#2F855A,stroke-width:2px;
    classDef externalService fill:#FFE5B4,stroke:#CC8400,stroke-width:2px;
    classDef fs fill:#F0F0F0,stroke:#999999,stroke-dasharray:2 2;
    classDef website fill:#EDE7F6,stroke:#5E35B1,stroke-width:2px;
```
