# dung0-o/llm-pulse-platform architecture

> Architecture diagram of the GitHub repository dung0-o/llm-pulse-platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dung0-o/llm-pulse-platform
- Repository: https://github.com/dung0-o/llm-pulse-platform

## Overview

Local LLM Sentiment Tracker turns public Reddit discussions into model-family sentiment analytics. The main workflow is RSS scraping to raw GCS data, documented medallion transformations in BigQuery, sentiment backfill through a configurable inference backend, then cached API responses rendered in five dashboard views. The supplied tree has no dbt files, so its documented warehouse stages are represented without invented source paths. API and scraper calls are supported by excerpts; transformation and dashboard-to-API links rely on README documentation.

Main entry points: [`dashboard/src/App.tsx`](https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/App.tsx) (View Router), [`api/main.py`](https://github.com/dung0-o/llm-pulse-platform/blob/main/api/main.py) (FastAPI Routes), [`api/jobs/backfill.py`](https://github.com/dung0-o/llm-pulse-platform/blob/main/api/jobs/backfill.py) (Sentiment Backfill), [`api/inference.py`](https://github.com/dung0-o/llm-pulse-platform/blob/main/api/inference.py) (Inference Backend)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Ingestion"]
  node_scraper["RSS Scraper<br/>[reddit_scraper.py]"]
  node_scraper_runner["Ingestion Runner<br/>[main.py]"]
  node_gcs_upload["GCS Upload<br/>[gcs_client.py]"]
end

subgraph group_warehouse["Warehouse and ML"]
  node_gcs_bronze[("Bronze JSONL")]
  node_silver[("Silver Posts")]
  node_gold_features[("Gold Features")]
  node_gold_predictions[("Sentiment Predictions")]
  node_enriched[("Enriched View")]
  node_backfill["Sentiment Backfill<br/>[backfill.py]"]
  node_inference["Inference Backend<br/>[inference.py]"]
end

subgraph group_serving["API Serving"]
  node_api["FastAPI Routes<br/>[main.py]"]
  node_bigquery["BigQuery Queries<br/>[database.py]"]
  node_redis[("Redis Cache<br/>[cache.py]")]
end

subgraph group_dashboard["Dashboard"]
  node_api_client["API Client<br/>[client.ts]"]
  node_dashboard_routes["View Router<br/>[App.tsx]"]
  node_leaderboard["Leaderboard View<br/>[DashboardPage.tsx]"]
  node_trend["Model Trend View<br/>[ModelPage.tsx]"]
  node_share_voice["Share of Voice"]
  node_search["Posts Search<br/>[SearchPage.tsx]"]
  node_health["System Health<br/>[HealthPage.tsx]"]
end

node_reddit(("Reddit RSS"))
node_hf["Hugging Face API"]
node_local_model["Local Model"]
node_dashboard_user(("Dashboard User"))

node_reddit -->|"provides feed"| node_scraper
node_scraper_runner -->|"scrapes posts"| node_scraper
node_scraper_runner -->|"uploads records"| node_gcs_upload
node_gcs_upload -->|"writes JSONL"| node_gcs_bronze
node_gcs_bronze -->|"transforms"| node_silver
node_silver -->|"transforms"| node_gold_features
node_gold_features -->|"enriches view"| node_enriched
node_gold_predictions -->|"joins view"| node_enriched
node_gold_features -->|"queries mentions"| node_backfill
node_backfill -->|"scores aspects"| node_inference
node_backfill -->|"reads and merges"| node_bigquery
node_inference -->|"requests scores"| node_hf
node_inference -.->|"runs locally"| node_local_model
node_api -->|"triggers backfill"| node_backfill
node_api -->|"queries analytics"| node_bigquery
node_api -->|"reads and writes"| node_redis
node_enriched -->|"serves queries"| node_bigquery
node_dashboard_user -->|"uses views"| node_dashboard_routes
node_dashboard_routes -->|"routes to"| node_leaderboard
node_dashboard_routes -->|"routes to"| node_trend
node_dashboard_routes -->|"routes to"| node_share_voice
node_dashboard_routes -->|"routes to"| node_search
node_dashboard_routes -->|"routes to"| node_health
node_api_client -->|"requests endpoints"| node_api

click node_scraper "https://github.com/dung0-o/llm-pulse-platform/blob/main/scraper/reddit_scraper.py"
click node_scraper_runner "https://github.com/dung0-o/llm-pulse-platform/blob/main/scraper/main.py"
click node_gcs_upload "https://github.com/dung0-o/llm-pulse-platform/blob/main/scraper/gcs_client.py"
click node_backfill "https://github.com/dung0-o/llm-pulse-platform/blob/main/api/jobs/backfill.py"
click node_inference "https://github.com/dung0-o/llm-pulse-platform/blob/main/api/inference.py"
click node_api "https://github.com/dung0-o/llm-pulse-platform/blob/main/api/main.py"
click node_bigquery "https://github.com/dung0-o/llm-pulse-platform/blob/main/api/database.py"
click node_redis "https://github.com/dung0-o/llm-pulse-platform/blob/main/api/cache.py"
click node_api_client "https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/api/client.ts"
click node_dashboard_routes "https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/App.tsx"
click node_leaderboard "https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/DashboardPage.tsx"
click node_trend "https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/ModelPage.tsx"
click node_share_voice "https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/ShareOfVoicePage.tsx"
click node_search "https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/SearchPage.tsx"
click node_health "https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/HealthPage.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scraper,node_scraper_runner,node_gcs_upload,node_dashboard_user toneBlue
class node_gcs_bronze,node_silver,node_gold_features,node_gold_predictions,node_enriched,node_backfill,node_inference toneAmber
class node_api,node_bigquery,node_redis,node_hf toneMint
class node_api_client,node_dashboard_routes,node_leaderboard,node_trend,node_share_voice,node_search,node_health toneRose
class node_reddit,node_local_model toneIndigo
```

## Components

### Ingestion

- [RSS Scraper](https://github.com/dung0-o/llm-pulse-platform/blob/main/scraper/reddit_scraper.py): `scraper/reddit_scraper.py`
- [Ingestion Runner](https://github.com/dung0-o/llm-pulse-platform/blob/main/scraper/main.py): `scraper/main.py`
- [GCS Upload](https://github.com/dung0-o/llm-pulse-platform/blob/main/scraper/gcs_client.py): `scraper/gcs_client.py`

### Warehouse and ML

- **Bronze JSONL**
- **Silver Posts**
- **Gold Features**
- **Sentiment Predictions**
- **Enriched View**
- [Sentiment Backfill](https://github.com/dung0-o/llm-pulse-platform/blob/main/api/jobs/backfill.py): `api/jobs/backfill.py`
- [Inference Backend](https://github.com/dung0-o/llm-pulse-platform/blob/main/api/inference.py): `api/inference.py`

### API Serving

- [FastAPI Routes](https://github.com/dung0-o/llm-pulse-platform/blob/main/api/main.py): `api/main.py`
- [BigQuery Queries](https://github.com/dung0-o/llm-pulse-platform/blob/main/api/database.py): `api/database.py`
- [Redis Cache](https://github.com/dung0-o/llm-pulse-platform/blob/main/api/cache.py): `api/cache.py`

### Dashboard

- [API Client](https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/api/client.ts): `dashboard/src/api/client.ts`
- [View Router](https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/App.tsx): `dashboard/src/App.tsx`
- [Leaderboard View](https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/DashboardPage.tsx): `dashboard/src/pages/DashboardPage.tsx`
- [Model Trend View](https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/ModelPage.tsx): `dashboard/src/pages/ModelPage.tsx`
- [Share of Voice](https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/ShareOfVoicePage.tsx): `dashboard/src/pages/ShareOfVoicePage.tsx`
- [Posts Search](https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/SearchPage.tsx): `dashboard/src/pages/SearchPage.tsx`
- [System Health](https://github.com/dung0-o/llm-pulse-platform/blob/main/dashboard/src/pages/HealthPage.tsx): `dashboard/src/pages/HealthPage.tsx`

### Other components

- **Reddit RSS**
- **Hugging Face API**
- **Local Model**
- **Dashboard User**

## Connections

- Reddit RSS → RSS Scraper: provides feed
- Ingestion Runner → RSS Scraper: scrapes posts (evidence: `scraper/main.py`)
- Ingestion Runner → GCS Upload: uploads records (evidence: `scraper/main.py`)
- GCS Upload → Bronze JSONL: writes JSONL (evidence: `scraper/gcs_client.py`)
- Bronze JSONL → Silver Posts: transforms
- Silver Posts → Gold Features: transforms
- Gold Features → Enriched View: enriches view
- Sentiment Predictions → Enriched View: joins view
- Gold Features → Sentiment Backfill: queries mentions
- Sentiment Backfill → Inference Backend: scores aspects (evidence: `api/jobs/backfill.py`)
- Sentiment Backfill → BigQuery Queries: reads and merges (evidence: `api/jobs/backfill.py`)
- Inference Backend → Hugging Face API: requests scores (evidence: `api/inference.py`)
- Inference Backend → Local Model: runs locally (evidence: `api/inference.py`)
- FastAPI Routes → Sentiment Backfill: triggers backfill (evidence: `api/main.py`)
- FastAPI Routes → BigQuery Queries: queries analytics (evidence: `api/main.py`)
- FastAPI Routes → Redis Cache: reads and writes (evidence: `api/main.py`)
- Enriched View → BigQuery Queries: serves queries (evidence: `api/database.py`)
- Dashboard User → View Router: uses views
- View Router → Leaderboard View: routes to (evidence: `dashboard/src/App.tsx`)
- View Router → Model Trend View: routes to (evidence: `dashboard/src/App.tsx`)
- View Router → Share of Voice: routes to (evidence: `dashboard/src/App.tsx`)
- View Router → Posts Search: routes to (evidence: `dashboard/src/App.tsx`)
- View Router → System Health: routes to (evidence: `dashboard/src/App.tsx`)
- API Client → FastAPI Routes: requests endpoints
