How hex-tech-lab/hex-yt-intel works
This repository is a hybrid edge content-intelligence platform for YouTube analysis. The architecture is split across a Next.js web app on Vercel, a Cloudflare Worker for long-running streaming work, and a Supabase-backed persistence layer. The repo is organized to keep the browser-facing app, the streaming engine, and database access in separate runtime boundaries.
The web app in /web is the main user-facing shell: dashboard, landing pages, auth screens, billing pages, share pages, search, atlas/global graph views, and admin dashboards. It uses Next.js App Router, React client components, route handlers, and local state stores/hooks for analysis, chat, UI, and streaming state. This layer acts as the “bouncer” and orchestration layer rather than doing the heavy synthesis work itself.
The Cloudflare Worker in /worker is the high-throughput execution layer. It exposes routes for chat, analysis, metadata, health, and transcript-related operations using Hono. Its services handle streaming LLM synthesis, transcript extraction, Markdown reconstruction, knowledge-graph synthesis, validation, and persistence preparation. This is the part designed for long-running browser-to-edge SSE flows.
A key architectural decision is the split between interactive requests and streaming synthesis. The web app handles authentication, quota checks, and request initiation. For deep analysis or chat generation, the client is routed into the worker’s SSE stream so the browser stays connected directly to the edge runtime while the worker emits incremental updates.
Persistence is centered on Supabase Postgres with extensive migrations and row-level security. The schema includes analyses, chat conversations/messages, idempotency controls, analysis chunks, knowledge graph tables, app settings, prompt config, billing status, and quota-related columns. The DB is not just storage; it is also the enforcement point for ownership, deduplication, and secure reads/writes.
The repo strongly separates concerns using a ports-and-adapters style. In /web/lib/ports and /worker/src/ports, the code defines abstractions for auth, billing, persistence, ingestion, graph, vector deduplication, model resolution, and streaming-related services. Concrete adapters then connect those ports to Supabase, Redis/Upstash, OpenRouter, Stripe/Paddle, YouTube, and other infrastructure.
The intelligence pipeline is a major subsystem. UCIS v5.1 parsing, dimension extraction, synthesis state, knowledge-graph generation, and relation scoring are split across web libraries, worker services, and UI components. The dashboard visualizes this output through panels such as the knowledge graph canvas, processing log, word cloud, and intelligence panel.
There is a distinct billing and entitlement boundary. The web app contains checkout, billing portal, and billing dashboard flows, while server-side adapters and webhook handlers integrate with Stripe and related billing backends. Billing status is also persisted into the database and used alongside quota enforcement and access control.
The project also includes a search and graph retrieval layer. Web-side hooks and API routes expose search, graph, relations, and atlas endpoints. Under the hood, these connect to persistence and vector/graph adapters so the UI can query saved analyses, global knowledge graphs, and related content views.
Security and reliability are built into the architecture rather than added later. Authentication is handled through Supabase/Vercel auth adapters, middleware, callback routes, and signed server-to-server persistence patterns. The repo also includes rate-limit status, HMAC/token adapters, RLS hardening migrations, and webhook validation routes for external event sources.
Observability is a first-class cross-cutting concern. Sentry is wired into the web app and worker, there are health routes in both runtimes, and the repo includes verification scripts plus deployment/test workflows. This suggests the system is meant to be operated continuously across Vercel, Cloudflare, and Supabase with clear production checks.
The documentation and design-system directories are part of the architecture too, but as supporting assets rather than runtime services. docs/ contains ADRs, specs, ops runbooks, and historical stabilization notes; design-system/ contains UI patterns and theme assets used to keep the console consistent across marketing and dashboard surfaces.
In diagram terms, the important flows are:
• User browser → web app → auth/quota check → worker streaming SSE → Supabase persistence
• User browser → web app route handlers → Supabase/Stripe/Sentry/Redis integrations
• Worker synthesis/validation → HMAC-signed writes → Supabase database
• Saved analyses / chats → web graph/search/share pages → Supabase read models
The main boundaries to show are:
1. Presentation layer in Next.js
2. Streaming compute layer in Cloudflare Worker
3. Persistence/entitlements layer in Supabase plus billing providers
4. Cross-cutting support for auth, observability, rate limiting, and vector/graph intelligence
Main entry points: api Web APIs · routes Worker routes · migrations Supabase DB · adapters Web adapters
Components 18
Next.js web app
Browser-facing console, routes, hooks, and client-side orchestration.
- Web shell · Main Next.js presentation layer for dashboard, landing, auth, billing, share, search, and admin surfaces.
web/app - Web APIs · Browser-facing endpoints for analyses, chat, search, billing, webhooks, health, and graph reads.
web/app/api - Dashboard UI · Interactive analysis workspace with panels for intelligence, graph canvas, streaming log, and metadata.
web/components/templates/console - Client hooks · Browser state and data hooks for auth, search, SSE streaming, graph, relations, and analysis history.
web/hooks - Web adapters · Concrete integrations for Supabase, Stripe, Redis, Upstash vector, worker ingestion, YouTube, and stream tokens.
web/lib/adapters - Use cases · Coordination layer for analysis creation, chat processing, and global graph aggregation.
web/lib/usecases - Ports · Repository boundary contracts for auth, billing, persistence, graph, ingestion, quota, and tokens.
web/lib/ports
Cloudflare Worker
Edge streaming and synthesis runtime for long-running analysis/chat work.
- Worker routes · Edge endpoints for chat, analysis, metadata, transcript, and health operations.
worker/src/routes - Chat stream · Streaming request lifecycle and incremental event emission for browser-connected synthesis.
worker/src/chat-stream.ts - Synthesis services · LLM cascade, prompt building, markdown reconstruction, reasoning, validation, and persistence preparation.
worker/src/services - Persist service · Atomic persistence and repository handoff layer used by worker synthesis flows.
worker/src/services/PersistService.ts - Worker ports · Interfaces for LLM cascade, persistence repository, prompt building, reasoning, and transcript providers.
worker/src/ports
Persistence and entitlements
Supabase schema, billing state, quotas, and durable read models.
- Supabase DB · Analyses, chats, chunks, knowledge graph tables, settings, billing status, idempotency, and RLS rules.
supabase/migrations - Billing providers · Stripe and related checkout/webhook flows used to enforce entitlements and status updates.
web/lib/stripe
Cross-cutting support
Auth, observability, rate limiting, and shared intelligence primitives.
- Auth layer · Supabase/Vercel auth, callbacks, middleware, and signed server-side request handling.
web/lib/auth - Observability · Sentry wiring plus health and verification routes/scripts across web and worker runtimes.
web/lib/monitoring - Intelligence core · UCIS v5.1 parsing, dimension extraction, knowledge graph logic, and relation scoring.
web/lib/intelligence - Traffic guard · Rate limit status, traffic adapters, and quota enforcement used before streaming or persistence.
web/lib/services/traffic.ts
Connections 26
- Web shell → Web APIs: calls · Pages initiate server actions and route-handler requests.
- Web shell → Dashboard UI: renders · The app shell mounts the console and related views.
- Web shell → Client hooks: uses · Client components rely on shared browser state and data hooks.
- Client hooks → Web adapters: reads/writes · Hooks ultimately consume adapter-backed data and streaming state.
- Web APIs → Supabase DB: reads/writes · Route handlers persist and query application state.
- Web APIs → Billing providers: integrates · Checkout and webhook routes connect to billing providers.
- Web APIs → Auth layer: authenticates · Auth routes and middleware establish user identity.
- Web APIs → Observability: reports · API routes emit logs and error telemetry.
- Web APIs → Traffic guard: checks · Inbound requests are gated by traffic and quota rules.
- Web APIs → Worker routes: routes to · Analysis and chat initiation hand off to the edge runtime.
- Use cases → Ports: depends on · Application services speak through abstract repository and service ports.
- Use cases → Web adapters: implemented by · Use cases are realized through concrete Supabase, Redis, and provider adapters.
- Web adapters → Supabase DB: persists · Adapters implement durable reads and writes against Postgres.
- Web adapters → Billing providers: integrates · Billing adapters bridge checkout and subscription state.
- Worker routes → Chat stream: starts · Chat and analysis routes create SSE stream lifecycles.
- Worker routes → Synthesis services: invokes · Route handlers delegate synthesis and validation work.
- Synthesis services → Worker ports: depends on · Edge services are built around explicit provider and repository contracts.
- Synthesis services → Persist service: prepares writes · Synthesis output is packaged for atomic persistence.
- Persist service → Supabase DB: writes · Worker persistence commits analyses, chats, graphs, and chunks.
- Chat stream → Supabase DB: streams state · Incremental synthesis is associated with durable conversation and analysis records.
- Intelligence core → Dashboard UI: drives UI · Graph, dimension, and relation outputs power the console panels.
- Intelligence core → Synthesis services: feeds · UCIS parsing and relation logic inform edge synthesis and validation.
- Traffic guard → Worker routes: guards · Streaming and heavy requests are constrained by quota and traffic policy.
- Auth layer → Supabase DB: authorizes · Ownership and RLS-aware identity flows map users to stored data.
- Observability → Worker routes: monitors · Health and telemetry cover worker execution paths.
- Observability → Web APIs: monitors · Health and telemetry cover web route handlers.