How bracesproul/site-rag works
Project Type and Purpose
• Site RAG is a Chrome extension built as a full-stack, client-side application. It provides an interactive chat UI allowing users to query the current webpage or entire site, performs retrieval-augmented generation (RAG) or context-stuffing with local or remote vector stores, and runs 100% in the browser (optionally integrating Supabase or a local Ollama LLM).
Architectural Overview
At a high level, the extension consists of:
a. A React+TypeScript frontend (popup UI)
b. A set of service libraries handling storage, indexing, API integrations, and query generation
c. External dependencies: LLM providers (OpenAI, Anthropic, Google, etc.), FireCrawl scraper API, and optionally Supabase (vector store)
Main Components
• UI Layer
• Popup UI (src/index.html, src/index.tsx, src/App.tsx)
• ChatView, SettingsForm, model-selector, index-site dialogs
• Reusable UI primitives (src/components/ui/*) and assistant-ui (thread, markdown, syntax highlighting)
• State & Storage
• store.ts: central state management (in-memory + persisted to browser storage)
• clear-docs.ts, last-indexed.ts: helper modules for session/document lifecycle
• index-db.ts: IndexedDB wrapper for local document storage
• Query & RAG Logic
• graphs/query: prompt generation, multi-query mode, utility functions
• constants.ts, types.ts: model/provider definitions, type safety
• API Integrations
• supabase.ts: client for remote vector store (PostgreSQL + pgvector)
• utils.ts: wrappers for fetch calls, FireCrawl scraper integration
• Runtime Glue
• runtimes/assistant-ui.tsx: bootstraps the React popup in extension context
• Build & Config
• Vite (vite.config.ts), TypeScript (tsconfig.json), PostCSS/Tailwind, ESLint, Jest
Component Interactions & Data Flow
1. User opens extension popup → React App mounts via runtimes/assistant-ui.tsx
2. SettingsForm reads/writes API keys and configuration into browser storage (store.ts)
3. User initiates a query in ChatView
• If “context stuffing” mode: src/lib/utils scrapes current page via FireCrawl
• If “RAG” mode:
a. graphs/query/generate-queries.ts generates semantic queries
b. Either index-db (local IndexedDB) or supabase.ts (remote DB) returns nearest documents
• Graphs utilities return a consolidated prompt
4. The prompt is sent to the chosen LLM provider (OpenAI, Anthropic, Google GenAI, local Ollama) via initChatModel wrapper
5. Model response flows back into ChatView and is rendered via assistant-ui components
6. Retrieved documents and session metadata are stored for follow-up queries (sessionID management)
Architectural Patterns & Principles
• Separation of Concerns: UI components ↔ business logic (graphs and lib) ↔ storage ↔ external services
• Modular Design: each service (scraper, vector store, LLM) is encapsulated in its own module
• Single Page UI: reactive React app loaded once by the extension
• Client-side Persistence: browser storage + IndexedDB for offline capability