# lightdash/lightdash architecture

> Architecture diagram of the GitHub repository lightdash/lightdash, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lightdash/lightdash
- Repository: https://github.com/lightdash/lightdash

## Overview

##### Project type and purpose
- Lightdash is a full-stack, open-source BI platform (an alternative to Looker).
- The monorepo contains:
- packages/backend – a Node.js/TypeScript Express API (TSOA-generated REST)
- packages/frontend – a React/Vite single-page app
- packages/cli – a Node.js CLI to scaffold and manage projects
- packages/common – shared TypeScript types and compiler logic
- packages/warehouses – adapters for external data warehouses (e.g. BigQuery, Snowflake)
- packages/e2e – Cypress end-to-end tests
- It connects to your dbt project, fetches metadata, compiles queries, and serves interactive dashboards, plus a scheduler for reports.

##### File structure
- Top-level “packages” folder holds each component; each has its own src, tests, configs, package.json.
- Backend is organized by routers → controllers → services → models → database (entities/migrations) → clients → analytics → scheduler.
- Frontend is organized by features, components, hooks, providers, plus an “ee” folder for enterprise-only features (AI copilot, embed, SCIM).
- There are Dockerfiles, docker-compose setups, CI/CD pipelines, Helm charts, and Kubernetes configs for deployment.
- The scheduler submodule uses in-process schedulers to run deliveries (email/Slack) and background jobs.
- External integrations: OAuth/OIDC/SSO, S3 cache, email templates, Slack, Google Drive, Prometheus, Sentry.

##### README for additional insights
- Confirms core use cases: connect dbt, define metrics in YAML, build/explore dashboards, schedule delivery, share URLs.
- Describes CI/CD validation, self-hosting guide, 1-click deploy, and contributions.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Users & Clients
    subgraph "Users & Clients"
        Browser["Browser (SPA)"]:::compute
        CLI["CLI"]:::compute
    end
    click CLI "https://github.com/lightdash/lightdash/tree/main/packages/cli/src"

    %% Frontend
    subgraph "Frontend"
        ReactSPA["React SPA (Vite)"]:::compute
        EE_FE["Enterprise Features (ee)"]:::compute
    end
    click ReactSPA "https://github.com/lightdash/lightdash/tree/main/packages/frontend"
    click EE_FE "https://github.com/lightdash/lightdash/tree/main/packages/frontend/src/ee"

    %% Backend
    subgraph "Backend"
        API["Express API (TSOA)"]:::compute
        AuthStrategies["OAuth/OIDC/SSO Strategies"]:::external
        SharedCommon["Shared Libraries"]:::library
        DBMigrations["DB Entities & Migrations"]:::database
        KnexConfig["Knex Config"]:::database
        DbtAPI["dbt Metadata API Client"]:::compute
        EE_BE["Enterprise Features (ee)"]:::compute
    end
    click API "https://github.com/lightdash/lightdash/tree/main/packages/backend/src"
    click AuthStrategies "https://github.com/lightdash/lightdash/tree/main/packages/backend/src/controllers/authentication/strategies"
    click SharedCommon "https://github.com/lightdash/lightdash/tree/main/packages/common/src"
    click DBMigrations "https://github.com/lightdash/lightdash/tree/main/packages/backend/src/database/entities"
    click DBMigrations "https://github.com/lightdash/lightdash/tree/main/packages/backend/src/database/migrations"
    click KnexConfig "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/knexfile.ts"
    click DbtAPI "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/dbt/DbtMetadataApiClient.ts"
    click EE_BE "https://github.com/lightdash/lightdash/tree/main/packages/backend/src/ee"

    %% Scheduler & Worker
    subgraph "Scheduler & Worker"
        SchedulerApp["SchedulerApp"]:::compute
        SchedulerClient["Scheduler Client"]:::compute
    end
    click SchedulerApp "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/SchedulerApp.ts"
    click SchedulerClient "https://github.com/lightdash/lightdash/tree/main/packages/backend/src/scheduler"

    %% Data Stores
    subgraph "Data Stores"
        MetadataDB["PostgreSQL (Metadata DB)"]:::database
        Warehouses["Data Warehouses"]:::database
    end

    %% Warehouses Adapters
    WareAdapters["Warehouse Adapters"]:::library
    click WareAdapters "https://github.com/lightdash/lightdash/tree/main/packages/warehouses/src"

    %% External Services
    subgraph "External Services"
        OAuth["OAuth Providers"]:::external
        S3Cache["S3 Cache & Storage"]:::external
        EmailClient["Email Delivery Client"]:::external
        SlackClient["Slack Integration"]:::external
        GoogleDrive["Google Drive Integration"]:::external
        Prometheus["Prometheus"]:::external
        Sentry["Sentry"]:::external
        PostHog["PostHog"]:::external
    end
    click S3Cache "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/clients/Aws/S3CacheClient.ts"
    click S3Cache "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/clients/Aws/S3ResultsFileStorageClient.ts"
    click EmailClient "https://github.com/lightdash/lightdash/tree/main/packages/backend/src/clients/EmailClient"
    click SlackClient "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/clients/Slack/SlackClient.ts"
    click GoogleDrive "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/clients/Google/GoogleDriveClient.ts"
    click Prometheus "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/prometheus.ts"
    click Sentry "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/sentry.ts"
    click PostHog "https://github.com/lightdash/lightdash/blob/main/packages/backend/src/postHog.ts"

    %% End-to-End Tests
    subgraph "Testing"
        E2E["Cypress E2E Tests"]:::external
    end
    click E2E "https://github.com/lightdash/lightdash/tree/main/packages/e2e"

    %% Relationships
    Browser -->|"HTTPS JSON"| ReactSPA
    CLI -->|"gRPC/HTTP"| API
    ReactSPA -->|"REST API Calls"| API
    API -->|"Knex SQL"| MetadataDB
    API -->|"SQL Queries"| Warehouses
    API -->|"Uses"| SharedCommon
    API -->|"Auth via"| AuthStrategies
    API -->|"Calls"| DbtAPI
    API -->|"Emits Jobs"| SchedulerClient
    SchedulerApp -->|"Triggers Reports"| EmailClient
    SchedulerApp -->|"Triggers Reports"| SlackClient
    SchedulerApp -->|"Uses"| SharedCommon
    SchedulerApp -->|"Knex SQL"| MetadataDB
    SharedCommon -->|"Compiles SQL"| WareAdapters
    WareAdapters -->|"Connects to"| Warehouses
    API -->|"Metrics"| Prometheus
    API -->|"Errors"| Sentry
    API -->|"Telemetry"| PostHog

    %% Styles
    classDef compute fill:#D0E8FF,stroke:#0366d6,color:#0366d6;
    classDef database fill:#DFFFE0,stroke:#10893E,color:#10893E;
    classDef external fill:#F0F0F0,stroke:#666666,color:#666666;
    classDef library fill:#FFF7D6,stroke:#E69E00,color:#E69E00;
```
