# elbwalker/walkeros architecture

> Architecture diagram of the GitHub repository elbwalker/walkeros, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/elbwalker/walkeros
- Repository: https://github.com/elbwalker/walkeros

## Overview

##### Project Type and Purpose
- This is a modular, full-stack monorepo for an open-source event data collection and tag management platform (walkerOS).
- It provides:
- Sources packages (data collection in various environments: browser, Node.js, GCP, dataLayer)
- Destinations packages (mapping and forwarding events to third-party tools)
- Shared utilities and types
- Configuration packages for linting, testing, builds
- Demo apps (React, static tagging demo) and a documentation website

##### Main Architectural Components
a. Monorepo Orchestrator
- Turbo (turbo.json) coordinates builds, tests, and deployments across all packages/apps.
- Root-level config: ESLint, Prettier, VSCode, GitHub workflows.
b. Core Libraries
- packages/types: shared event and API type definitions
- packages/utils: low-level helpers for event construction, consent, session, network calls
- packages/config: ESLint, Jest, tsconfig, tsup presets for consistent setup
c. Sources Layer (Collection)
- packages/sources/*: implement “push” interfaces and state management for each environment
- walkerjs (browser), node, dataLayer, GCP, custom “tagger” CLI
d. Destinations Layer (Activation)
- packages/destinations/*: for each tool (AWS Firehose, BigQuery, Google Ads, GA4, GTM, Meta Pixel, Plausible, Piwik Pro, generic API)
- Each destination package consumes events, applies mapping, forwards via HTTP/SDK
e. Demos and Website
- apps/demos: quickstart React and plain JS tagging examples illustrating integration of Sources → Destinations
- website: Docusaurus-based documentation and marketing site

##### Data Flow and Interactions
- Application Code (demo or customer app) imports a Source package → calls `push(event)`
- Source layer enriches event (session, device, consent) and emits it to a generic pipeline → Utils functions handle retries, error handling
- Destinations layer subscribes to pipeline output, maps events per destination config → forwards to external Tools (Analytics, Storage, Marketing)
- Types package ensures schema consistency end-to-end
- CI/CD pipelines (GitHub Actions) run lint/tests/build for every package

##### Architectural Patterns and Principles
- Monorepo with package-per-concern: clear separation of collection vs activation
- Shared core (utils, types, config) encourages DRY and consistency
- Plugin-style destinations allow easy extensibility (add new third-party integrations)
- Turbo-driven incremental builds for scale
- Test-first approach: each package has its own Jest config and suite

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Monorepo Orchestrator and Root Configs
    subgraph "Monorepo Orchestrator" 
        direction TB
        Turbo["Turbo (turbo.json)"]:::core
        GitHub["GitHub Actions (.github/workflows)"]:::core
        ESLintRC["ESLint config (eslint.config.js)"]:::core
        PrettierRC["Prettier config (.prettierrc)"]:::core
        VSCode["VSCode settings (.vscode/)"]:::core
    end

    %% Config & Build Tools
    subgraph "Config & Build Tools" 
        direction TB
        ESLintPreset["packages/config/eslint"]:::config
        JestPreset["packages/config/jest"]:::config
        TSConfig["packages/config/tsconfig"]:::config
        TSUP["packages/config/tsup"]:::config
    end

    %% Shared Libraries
    subgraph "Shared Libraries" 
        direction TB
        TypesPkg["packages/types"]:::core
        UtilsPkg["packages/utils"]:::core
    end

    %% Sources Layer
    subgraph "Sources Layer (Collection)" 
        direction TB
        WalkerJS["packages/sources/walkerjs"]:::source
        NodeSrc["packages/sources/node"]:::source
        DataLayer["packages/sources/datalayer"]:::source
        GCP["packages/sources/gcp"]:::source
    end

    TaggerCLI["packages/tagger"]:::source

    %% Destinations Layer
    subgraph "Destinations Layer (Activation)"
        direction TB
        subgraph CoreDest
            ETagCore["packages/destinations/core/etag"]:::dest
        end
        subgraph NodeDest
            AWS["packages/destinations/node/aws"]:::dest
            BigQuery["packages/destinations/node/bigquery"]:::dest
            ETagNode["packages/destinations/node/etag"]:::dest
            MetaNode["packages/destinations/node/meta"]:::dest
        end
        subgraph WebDest
            APIWeb["packages/destinations/web/api"]:::dest
            ETagWeb["packages/destinations/web/etag"]:::dest
            Ads["packages/destinations/web/google-ads"]:::dest
            GA4["packages/destinations/web/google-ga4"]:::dest
            GTM["packages/destinations/web/google-gtm"]:::dest
            MetaPixel["packages/destinations/web/meta-pixel"]:::dest
            Piwik["packages/destinations/web/piwikpro"]:::dest
            Plausible["packages/destinations/web/plausible"]:::dest
        end
    end

    %% Demo Apps and Website
    subgraph "Apps & Website" 
        direction TB
        ReactDemo["apps/demos/react"]:::app
        TaggingDemo["apps/demos/tagging"]:::app
        Website["website"]:::app
    end

    %% External Tools
    subgraph "External Systems" 
        direction TB
        Analytics["Analytics & Marketing APIs"]:::external
        Storage["Storage & BigQuery"]:::external
        SDKs["Third-party SDKs"]:::external
    end

    %% Relationships
    GitHub -->|CI/CD| ESLintPreset
    GitHub -->|CI/CD| JestPreset
    GitHub -->|CI/CD| TSConfig
    GitHub -->|CI/CD| TSUP
    GitHub -->|CI/CD| TypesPkg
    GitHub -->|CI/CD| UtilsPkg
    GitHub -->|CI/CD| WalkerJS
    GitHub -->|CI/CD| NodeSrc
    GitHub -->|CI/CD| DataLayer
    GitHub -->|CI/CD| GCP
    GitHub -->|CI/CD| TaggerCLI
    GitHub -->|CI/CD| ETagCore
    GitHub -->|CI/CD| AWS
    GitHub -->|CI/CD| BigQuery
    GitHub -->|CI/CD| ETagNode
    GitHub -->|CI/CD| MetaNode
    GitHub -->|CI/CD| APIWeb
    GitHub -->|CI/CD| ETagWeb
    GitHub -->|CI/CD| Ads
    GitHub -->|CI/CD| GA4
    GitHub -->|CI/CD| GTM
    GitHub -->|CI/CD| MetaPixel
    GitHub -->|CI/CD| Piwik
    GitHub -->|CI/CD| Plausible
    GitHub -->|CI/CD| ReactDemo
    GitHub -->|CI/CD| TaggingDemo
    GitHub -->|CI/CD| Website

    ReactDemo -->|imports| WalkerJS
    TaggingDemo -->|imports| DataLayer

    WalkerJS -->|"push(event)"| UtilsPkg
    NodeSrc -->|"push(event)"| UtilsPkg
    DataLayer -->|"push(event)"| UtilsPkg
    GCP -->|"push(event)"| UtilsPkg
    TaggerCLI -->|"CLI push"| UtilsPkg

    UtilsPkg --> DestinationsLayer
    TypesPkg --> WalkerJS
    TypesPkg --> NodeSrc
    TypesPkg --> DataLayer
    TypesPkg --> GCP
    TypesPkg --> DestinationsLayer

    SourcesLayer -->|pipeline| CoreDest
    SourcesLayer -->|pipeline| NodeDest
    SourcesLayer -->|pipeline| WebDest

    CoreDest -->|forwards| Analytics
    NodeDest -->|forwards| SDKs
    WebDest -->|forwards| Analytics

    Website -->|docs| Users
    Website -->|marketing| Users

    %% Click Events
    click Turbo "https://github.com/elbwalker/walkeros/blob/main/turbo.json"
    click GitHub "https://github.com/elbwalker/walkeros/tree/main/.github/workflows"
    click ESLintRC "https://github.com/elbwalker/walkeros/blob/main/eslint.config.js"
    click PrettierRC "https://github.com/elbwalker/walkeros/blob/main/.prettierrc"
    click VSCode "https://github.com/elbwalker/walkeros/tree/main/.vscode/"
    click ESLintPreset "https://github.com/elbwalker/walkeros/blob/main/packages/config/eslint/index.js"
    click ESLintPreset "https://github.com/elbwalker/walkeros/blob/main/packages/config/eslint/package.json"
    click JestPreset "https://github.com/elbwalker/walkeros/blob/main/packages/config/jest/src/index.js"
    click JestPreset "https://github.com/elbwalker/walkeros/blob/main/packages/config/jest/package.json"
    click TSConfig "https://github.com/elbwalker/walkeros/blob/main/packages/config/tsconfig/base.json"
    click TSConfig "https://github.com/elbwalker/walkeros/blob/main/packages/config/tsconfig/node.json"
    click TSConfig "https://github.com/elbwalker/walkeros/blob/main/packages/config/tsconfig/web.json"
    click TSUP "https://github.com/elbwalker/walkeros/blob/main/packages/config/tsup/index.mjs"
    click TSUP "https://github.com/elbwalker/walkeros/blob/main/packages/config/tsup/package.json"
    click TypesPkg "https://github.com/elbwalker/walkeros/blob/main/packages/types/src/index.ts"
    click TypesPkg "https://github.com/elbwalker/walkeros/blob/main/packages/types/src/schema.ts"
    click UtilsPkg "https://github.com/elbwalker/walkeros/tree/main/packages/utils/src/core"
    click UtilsPkg "https://github.com/elbwalker/walkeros/tree/main/packages/utils/src/web"
    click UtilsPkg "https://github.com/elbwalker/walkeros/tree/main/packages/utils/src/node"
    click WalkerJS "https://github.com/elbwalker/walkeros/blob/main/packages/sources/walkerjs/src/lib/push.ts"
    click NodeSrc "https://github.com/elbwalker/walkeros/blob/main/packages/sources/node/src/lib/push.ts"
    click DataLayer "https://github.com/elbwalker/walkeros/blob/main/packages/sources/datalayer/src/push.ts"
    click GCP "https://github.com/elbwalker/walkeros/blob/main/packages/sources/gcp/src/index.ts"
    click TaggerCLI "https://github.com/elbwalker/walkeros/blob/main/packages/tagger/src/index.ts"
    click ETagCore "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/core/etag/src/index.ts"
    click AWS "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/node/aws/src/push.ts"
    click BigQuery "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/node/bigquery/src/push.ts"
    click ETagNode "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/node/etag/src/push.ts"
    click MetaNode "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/node/meta/src/push.ts"
    click APIWeb "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/web/api/src/index.ts"
    click ETagWeb "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/web/etag/src/index.ts"
    click Ads "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/web/google-ads/src/index.ts"
    click GA4 "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/web/google-ga4/src/index.ts"
    click GTM "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/web/google-gtm/src/index.ts"
    click MetaPixel "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/web/meta-pixel/src/index.ts"
    click Piwik "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/web/piwikpro/src/index.ts"
    click Plausible "https://github.com/elbwalker/walkeros/blob/main/packages/destinations/web/plausible/src/index.ts"
    click ReactDemo "https://github.com/elbwalker/walkeros/blob/main/apps/demos/react/src/App.tsx"
    click TaggingDemo "https://github.com/elbwalker/walkeros/blob/main/apps/demos/tagging/index.html"
    click Website "https://github.com/elbwalker/walkeros/blob/main/website/docusaurus.config.ts"

    %% Styles
    classDef core fill:#cccccc,stroke:#333,stroke-width:1px
    classDef config fill:#eeeeee,stroke:#333,stroke-width:1px
    classDef source fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef dest fill:#d4edda,stroke:#333,stroke-width:1px
    classDef app fill:#ffe5b4,stroke:#333,stroke-width:1px
    classDef external fill:#f8d7da,stroke:#333,stroke-width:1px
    class MonorepoOrchestrator core
    class ConfigAndBuild config
    class Shared core
    class SourcesLayer source
    class DestinationsLayer dest
    class AppsAndWebsite app
    class ExternalSystems external
```
