How ashish10alex/vscode-dataform-tools works
Project type and purpose
• This is a VS Code extension—“Dataform tools”—for authoring and running Dataform .sqlx projects.
• It provides LSP‑like features (hover, completions, diagnostics, go‑to‑definition), command handlers (run file/tag, cost estimate, format), and rich WebView UIs (compiled query, dependency graph, preview results).
• There is also a standalone Next.js “website” folder that hosts documentation.
File structure
• Top‑level “src/” contains the extension’s entrypoint (extension.ts) plus providers (hover, definition, codeLens, codeAction, completions), command implementations (bigqueryDryRun, runFiles, runTag, costEstimator, formatCurrentFile), parsers (sqlxFileParser, dependencyTree), and utilities (logger, bigqueryClient).
• “views/” and “webviews/” hold React/Vite apps for WebView panels (dependency graph, compiled query, query results).
• “media/” supplies static assets for the README.
• “website/” is a Next.js app (React/TS) for marketing and docs, entirely separate at runtime.
• Configuration/build files include tsconfig.json, vite.config.ts (bundles WebViews), package.json, tailwind.config.js.
Extract insights from the README
• Core features map 1:1 with providers and commands in “src/”: compilation/dry‑run stats, inline diagnostics, dependency graph, preview results, code generation, cost estimator, formatting via sqlfluff CLI, BigQuery hovers, BigQuery snippets.
• External prerequisites: Dataform CLI, gcloud SDK, sqlfluff, optional Error Lens.
a. Main components
• VS Code Editor (UI shell)
• Extension Host (Node/TypeScript runtime)
– Core modules: command handlers (runQuery, runFile, runTag, costEstimator, formatDocument)
– Language providers: hoverProvider, definitionProvider, completionProvider, codeActionProvider, codeLensProvider, diagnostics (setDiagnostics)
– Parsers & logic: sqlxFileParser, dependencyTree, costEstimator, getLineageMetadata
– Utilities: bigqueryClient (calls BigQuery API), logger
– External CLI invocations: Dataform CLI (compile), sqlfluff CLI (format)
• WebView UIs (embedded React apps)
– Compiled query panel, dependency graph panel, query results panel
• External services
– Dataform CLI (spawned child process)
– Google BigQuery API (via gcloud or SDK)
– SQLFluff (spawned Python CLI)
• File system
– User workspace: .sqlx files, JS modules, Dataform project config
– Extension’s internal test workspace under src/test
• Documentation Website (Next.js)
b. Relationships & data flows
1. User triggers a command in VS Code → VS Code API → Extension Host
2. Extension Host → Dataform CLI (spawn child_process) → receives compiled SQL & dry‑run stats
3. Extension Host → BigQuery API (via bigqueryClient/gcloud) → receives query results & cost estimates
4. Extension Host → WebView panel (postMessage) → React UI renders data
5. On .sqlx edit or hover → VS Code API → respective provider module → uses parser & metadata → returns completions/hover/diagnostics
6. Format command → Extension Host → SQLFluff CLI → applies edits via VS Code TextEdit
7. Tests drive extension code using vscode-test harness
c. Architectural patterns & principles
• Modular “Command + Provider” design adhering to VS Code Extension API conventions
• Clear separation: core logic (command handlers & providers) vs. UI (WebViews)
• Event‑driven: VS Code events trigger extension reactions
• Micro‑frontend approach for WebView panels (each React app isolated & bundled via Vite)
• “Backend” logic in-process; external CLIs and cloud APIs treated as services
d. Key technologies & frameworks
• TypeScript, VS Code Extension API
• React + Vite for WebView UIs
• Next.js for documentation website
• Dataform CLI (Node child_process)
• Google Cloud SDK / BigQuery client
• SQLFluff for formatting
• Mocha + vscode-test for integration tests