# evidence-dev/evidence architecture

> Architecture diagram of the GitHub repository evidence-dev/evidence, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/evidence-dev/evidence
- Repository: https://github.com/evidence-dev/evidence

## Overview

##### Project type and purpose
- This is an open-source monorepo for “Evidence”: a Business Intelligence-as-Code toolkit.
- Core deliverables include:
- An Evidence CLI/SDK (packages/evidence, packages/lib/sdk) for scaffolding and building SvelteKit-based data-driven sites.
- A VS Code extension (packages/extension) to bootstrap and interact with Evidence projects.
- A suite of data-source connectors (packages/datasources/*) and libraries (universal-sql, telemetry, db-commons).
- UI component libraries (packages/ui/*) for charts, maps, tables.
- Example/demo SvelteKit sites (sites/docs, sites/example-project, sites/test-env).
- End-to-end tests and CI/CD workflows (.github/workflows).

##### File structure
- Top-level files: pnpm-workspace.yaml, package.json → monorepo management.
- /.github/workflows → CI pipelines for lint, tests, release, e2e.
- /packages → independent npm packages grouped by function:
- datasources: connectors (BigQuery, CSV, DuckDB, etc.)
- evidence & lib/sdk: CLI, Svelte/Vite plugins, configuration, build pipelines
- extension: VS Code integration
- lib: shared utilities (component-utilities, db-commons, preprocess, telemetry, universal-sql)
- /sites → three self-contained SvelteKit sites (docs site, example-project, test-env) showing usage.
- /e2e → Playwright/Vitest based end-to-end test projects for various scenarios.

##### Examining the README
- Describes Evidence as a code-based BI tool generating websites from Markdown + SQL.
- Key architecture:
- CLI/VS Code extension to initialize projects
- Markdown pages embed SQL → powered by universal-sql engine
- SvelteKit + Vite + Evidence SDK for processing pages, extracting queries, rendering charts
- Ability to plugin data-sources, UI components, layouts
- Docker/CLI/Cloud hosting options

a. Main components
– Developer workflows: VS Code extension & CLI
– Monorepo management: pnpm workspace
– Evidence CLI & SDK: build-time processors, plugin architecture
– Data-source connectors (universal-sql engine)
– UI component libraries
– SvelteKit/Vite build pipeline → static site / SSR
– Example & docs front-end apps
– CI/CD pipelines (GitHub Actions + e2e tests)
– External services: cloud hosting, databases (BigQuery, Postgres, DuckDB, CSV files, etc.)
b. Relationships and interactions
– Developer → launches CLI or VS Code extension → generates new project scaffold
– CLI invokes SDK → reads evidence.config, loads plugins (datasources, components, layouts)
– SDK preprocessors scan Markdown pages → extract SQL → universal-sql executes queries on configured data sources → returns results
– Vite/SvelteKit + Evidence SDK Vite plugins transform page code → generates static HTML/JS/CSS or SSR exports
– Build output → deployed to Evidence Cloud, Vercel, Netlify or other host
– E2E tests (Playwright) exercise sample sites → report back via CI
c. Architectural patterns & principles
– Monorepo with modular packages: code reuse and independent versioning
– Plugin architecture: datasources, components, layout, Vite plugins
– Static site generation + SSR fallback via SvelteKit
– Separation of concerns: CLI/SDK vs. runtime UI vs. build-time processors
– Test-driven: unit tests per package, e2e scenarios
d. Key technologies
– Node.js, pnpm workspace
– TypeScript/JavaScript, Zod schemas for config validation
– SvelteKit & Vite with custom processors
– Playwright and Vitest for e2e & unit testing
– DuckDB, SQL connectors, bigquery, CSV, etc.
– GitHub Actions for CI/CD

##### Elements
– Clear labels: “VS Code Extension,” “Evidence CLI/SDK,” “Datasource Plugins,” “Universal-SQL Engine,” “SvelteKit/Vite Build,” “Static Site / SSR,” “CI Pipeline,” “Data Storage” (DuckDB, BigQuery, CSV)
- Developer → CLI/Extension → Scaffold → Build
- Build → Vite plugins → SvelteKit → HTML/JS/CSS
- Markdown → Query extraction → SQL execution → Results
- CI → Tests → Release → npm registry
- Orange for runtime libraries (universal-sql, datasources)
- Purple for UI components
- Gray for external services

##### Be detailed but not overwhelming
- Model each logically distinct package group as its own node or cluster.
- End-to-end flow from writing Markdown + SQL to hosting a fully rendered site.
- CI/CD and testing as first-class citizens in the architecture.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Developer Tools"
        Dev("Developer"):::actor
        Dev --> VSCodeExt["VS Code Extension"]:::frontend
        Dev --> EvidenceCLI["Evidence CLI"]:::frontend
    end

    subgraph "Evidence SDK & Build"
        EvidenceCLI --> EvidenceSDK["Evidence SDK (Vite Plugins)"]:::build
        VSCodeExt --> EvidenceCLI
        EvidenceSDK --> PluginsComp["Component Plugins"]:::build
        EvidenceSDK --> PluginsData["Datasource Plugins"]:::build
        EvidenceSDK --> ConfigSchemas["Configuration Schemas"]:::build
        EvidenceSDK --> CLIEntry["SDK CLI Commands"]:::build
        EvidenceSDK --> ViteVirtual["Vite Virtual Modules"]:::build
    end

    subgraph "Core Libraries"
        UniversalSQL["Universal-SQL Engine"]:::runtime
        Telemetry["Telemetry Library"]:::runtime
        DBCommons["DB Utilities"]:::runtime
        Preprocess["Preprocess Utilities"]:::runtime
        PluginsData --> BigQuery["BigQuery Connector"]:::runtime
        PluginsData --> CSV["CSV Connector"]:::runtime
        PluginsData --> DuckDB["DuckDB Connector"]:::runtime
    end

    subgraph "UI Libraries"
        UIcore["Core Components"]:::ui
        UIicons["Icons"]:::ui
        UItailwind["Tailwind Themes"]:::ui
    end

    subgraph "Sites (Examples)"
        Docs["Documentation Site"]:::app
        Example["Example Project Site"]:::app
        TestEnv["Test Environment Site"]:::app
    end

    subgraph "Build & Deploy"
        EvidenceSDK --> SvelteKit["SvelteKit/Vite Build"]:::build
        SvelteKit --> StaticSite["Static Site / SSR Output"]:::infra
        StaticSite --> Cloud["Hosting (Vercel/Netlify/Evidence Cloud)"]:::infra
    end

    subgraph "CI/CD & Testing"
        MonorepoConfig["Monorepo & CI/CD Config"]:::infra
        MonorepoConfig --> E2E["E2E Test Suites"]:::infra
        MonorepoConfig --> EvidenceSDK
        MonorepoConfig --> Dev
    end

    EvidenceSDK --> UniversalSQL
    EvidenceSDK --> Preprocess
    UniversalSQL --> DBCommons

    Docs --> EvidenceSDK
    Docs --> UIcore
    Docs --> UniversalSQL
    Docs --> Preprocess

    Example --> EvidenceSDK
    Example --> UIcore
    Example --> UniversalSQL

    TestEnv --> EvidenceSDK
    TestEnv --> UIcore

    E2E --> Docs
    E2E --> Example

    %% Click Events
    click EvidenceCLI "https://github.com/evidence-dev/evidence/blob/next/packages/evidence/cli.js"
    click EvidenceCLI "https://github.com/evidence-dev/evidence/blob/next/packages/evidence/index.js"
    click EvidenceSDK "https://github.com/evidence-dev/evidence/tree/next/packages/lib/sdk/src"
    click EvidenceSDK "https://github.com/evidence-dev/evidence/blob/next/packages/lib/sdk/vite.config.js"
    click EvidenceSDK "https://github.com/evidence-dev/evidence/blob/next/packages/lib/sdk/vitest.config.js"
    click VSCodeExt "https://github.com/evidence-dev/evidence/tree/next/packages/extension/src"
    click VSCodeExt "https://github.com/evidence-dev/evidence/blob/next/packages/extension/package.json"
    click BigQuery "https://github.com/evidence-dev/evidence/blob/next/packages/datasources/bigquery/index.cjs"
    click CSV "https://github.com/evidence-dev/evidence/blob/next/packages/datasources/csv/index.cjs"
    click DuckDB "https://github.com/evidence-dev/evidence/blob/next/packages/datasources/duckdb/index.cjs"
    click UniversalSQL "https://github.com/evidence-dev/evidence/tree/next/packages/lib/universal-sql/src"
    click UniversalSQL "https://github.com/evidence-dev/evidence/blob/next/packages/lib/universal-sql/package.json"
    click Telemetry "https://github.com/evidence-dev/evidence/blob/next/packages/lib/telemetry/index.cjs"
    click Telemetry "https://github.com/evidence-dev/evidence/blob/next/packages/lib/telemetry/package.json"
    click DBCommons "https://github.com/evidence-dev/evidence/blob/next/packages/lib/db-commons/index.cjs"
    click DBCommons "https://github.com/evidence-dev/evidence/tree/next/packages/lib/db-commons/src"
    click Preprocess "https://github.com/evidence-dev/evidence/tree/next/packages/lib/preprocess/src"
    click Preprocess "https://github.com/evidence-dev/evidence/blob/next/packages/lib/preprocess/package.json"
    click UIcore "https://github.com/evidence-dev/evidence/tree/next/packages/ui/core-components/src"
    click UIcore "https://github.com/evidence-dev/evidence/blob/next/packages/ui/core-components/package.json"
    click UIicons "https://github.com/evidence-dev/evidence/tree/next/packages/ui/icons/src"
    click UIicons "https://github.com/evidence-dev/evidence/blob/next/packages/ui/icons/package.json"
    click UItailwind "https://github.com/evidence-dev/evidence/tree/next/packages/ui/tailwind/src"
    click UItailwind "https://github.com/evidence-dev/evidence/blob/next/packages/ui/tailwind/package.json"
    click Docs "https://github.com/evidence-dev/evidence/tree/next/sites/docs/"
    click Example "https://github.com/evidence-dev/evidence/tree/next/sites/example-project/src"
    click Example "https://github.com/evidence-dev/evidence/blob/next/sites/example-project/evidence.config.yaml"
    click TestEnv "https://github.com/evidence-dev/evidence/tree/next/sites/test-env/pages"
    click TestEnv "https://github.com/evidence-dev/evidence/blob/next/sites/test-env/evidence.config.yaml"
    click E2E "https://github.com/evidence-dev/evidence/tree/next/e2e/"
    click MonorepoConfig "https://github.com/evidence-dev/evidence/blob/next/pnpm-workspace.yaml"
    click MonorepoConfig "https://github.com/evidence-dev/evidence/blob/next/package.json"
    click MonorepoConfig "https://github.com/evidence-dev/evidence/tree/next/.github/workflows/"
    click ConfigSchemas "https://github.com/evidence-dev/evidence/blob/next/packages/lib/sdk/src/configuration/schemas/*.js"
    click PluginsData "https://github.com/evidence-dev/evidence/tree/next/packages/lib/sdk/src/plugins/datasources"
    click PluginsComp "https://github.com/evidence-dev/evidence/tree/next/packages/lib/sdk/src/plugins/components"
    click ViteVirtual "https://github.com/evidence-dev/evidence/tree/next/packages/lib/sdk/src/build-dev/vite/virtuals"
    click ViteVirtual "https://github.com/evidence-dev/evidence/blob/next/packages/lib/sdk/src/build-dev/vite/plugins.js"
    click CLIEntry "https://github.com/evidence-dev/evidence/blob/next/packages/lib/sdk/src/cli.js"
    click CLIEntry "https://github.com/evidence-dev/evidence/blob/next/packages/lib/sdk/src/cli.root.js"

    classDef actor fill:#cdf,stroke:#333,stroke-width:1px
    classDef frontend fill:#bbdefb,stroke:#333,stroke-width:1px
    classDef build fill:#c8e6c9,stroke:#333,stroke-width:1px
    classDef runtime fill:#ffe0b2,stroke:#333,stroke-width:1px
    classDef ui fill:#e1bee7,stroke:#333,stroke-width:1px
    classDef app fill:#d0d0d0,stroke:#333,stroke-width:1px
    classDef infra fill:#f5f5f5,stroke:#333,stroke-width:1px
```
