# gtbitsofgood/bog-analytics architecture

> Architecture diagram of the GitHub repository gtbitsofgood/bog-analytics, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gtbitsofgood/bog-analytics
- Repository: https://github.com/gtbitsofgood/bog-analytics

## Overview

##### Project type and purpose
- This is a full-stack, monorepo analytics platform (“Bits of Good Analytics”) with four core deliverables:
- A Python/Streamlit Dashboard for data visualization
- A TypeScript “Analytics Processing API” (serverless on Netlify) that ingests and stores events
- An NPM “Package” client library (“bog-analytics”) to instrument front-ends and call the API
- A Next.js “Portal” for project and event-type management, backed by its own Node/MongoDB server

##### High-level components (from file tree & README)
- Dashboard: `dashboard/` (Streamlit app, Python, connects to API via HTTP)
- API: `api/` (TypeScript, Netlify functions, models, controllers, migrations, connects to MongoDB analytics DB)
- Package: `package/` (TypeScript library, wraps API calls, exposes event-logging functions)
- Portal: `portal/` (Next.js + React front end under `portal/src/app`, plus server code under `portal/server`, authenticates via MongoDB portal DB)
- Deletion-policy service: `deletion-policy/` (TypeScript cron job/Netlify function for GDPR deletions)
- Probe action: `probe-action/` (GitHub Action to health-check or smoke-test the API)
- Supporting infra: Docker Compose files for local dev (`docker-compose.yml`, etc.)

##### Core data flows & relationships
1. Instrumented client (any Bits of Good project, e.g. `examples/example-project`) installs `bog-analytics` package → calls `analytics API` HTTP endpoints → events are validated, processed and stored in the analytics MongoDB.
2. Deletion-policy service (cron job) calls API’s GDPR endpoints to permanently delete user data per policy.
3. Portal UI & server call their own MongoDB (portal DB) for user/project metadata, and may also call the analytics API to surface event-type counts or example payloads.
4. Dashboard (Streamlit) calls the analytics API to fetch aggregated event data for UI charts.
5. Probe-action runs periodically (GitHub Action) to call health & sample-event endpoints on the analytics API.

##### Architectural patterns & design principles
- Microservices style: separate deployments for API, Portal, Dashboard, Deletion job, Probe action
- Serverless functions (Netlify) for API endpoints
- Single shared analytics MongoDB for event storage, separate portal MongoDB for auth & metadata
- NPM package as a thin client SDK layer over REST API
- Clear MVC in the API: `models/`, `controllers/`, `routes.ts`, `middleware/`
- Next.js App Router for Portal front end + serverless API routes

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        direction TB
        Client["Instrumented Client\n(e.g. Example Project)"]:::client
        Package["bog-analytics SDK\n(TypeScript client)"]:::client
        PortalUI["Portal UI\n(Next.js Front End)"]:::client
        Dashboard["Streamlit Dashboard\n(Python)"]:::client
    end

    %% Backend & Services
    subgraph "Back-end & Services"
        direction TB
        API["Analytics API\n(Netlify Functions,\nTypeScript, Express-style controllers)"]:::service
        PortalServer["Portal Server\n(Node.js, Next.js API Routes)"]:::service
        Deletion["Deletion-Policy Service\n(Cron, GDPR cleanup)"]:::job
        Probe["Probe Action\n(GitHub Action)"]:::ci
    end

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        AnalyticsDB["Analytics DB\n(MongoDB)"]:::db
        PortalDB["Portal DB\n(MongoDB)"]:::db
    end

    %% Data Flows
    Client -->|"calls SDK methods"| Package
    Package -->|"POST /events\nGET /schemas"| API
    API -->|"stores events"| AnalyticsDB

    Deletion -->|"DELETE /gdpr"| API
    Probe -->|"GET /health\nGET /sample-event"| API

    PortalUI -->|"HTTP requests"| PortalServer
    PortalServer -->|"reads/writes metadata"| PortalDB
    PortalUI -->|"GET /event-types"| API

    Dashboard -->|"GET /reports"| API

    %% Click Events
    click Dashboard "https://github.com/gtbitsofgood/bog-analytics/blob/main/dashboard/app.py"
    click API "https://github.com/gtbitsofgood/bog-analytics/blob/main/api/netlify/functions/api.ts"
    click Package "https://github.com/gtbitsofgood/bog-analytics/blob/main/package/src/index.ts"
    click PortalUI "https://github.com/gtbitsofgood/bog-analytics/tree/main/portal/src/app/"
    click PortalServer "https://github.com/gtbitsofgood/bog-analytics/blob/main/portal/server/auth.ts"
    click Deletion "https://github.com/gtbitsofgood/bog-analytics/blob/main/deletion-policy/delete-events.ts"
    click Probe "https://github.com/gtbitsofgood/bog-analytics/blob/main/probe-action/probe.js"

    %% Styles
    classDef client fill:#f9cb9c,stroke:#333,stroke-width:1px;
    classDef service fill:#a4c2f4,stroke:#333,stroke-width:1px;
    classDef db fill:#bde5a4,stroke:#333,stroke-width:1px;
    classDef job fill:#cccccc,stroke:#333,stroke-width:1px;
    classDef ci fill:#e7e0b2,stroke:#333,stroke-width:1px;
```
