How rootsongjc/rag-chatbot works
Project type and purpose
• This is a full-stack “Retrieval-Augmented Generation (RAG) Chatbot” project.
• Runtime lives in a Cloudflare Worker (“worker.ts”), exposing a /chat endpoint for RAG queries and admin endpoints for index management.
• There is also a small embeddable JavaScript widget (widget.js generated on deploy) that your Hugo site includes.
• Offline ingestion/reindexing scripts read local Hugo Markdown content, chunk it, embed it, and write vectors to a Cloudflare Vectorize index.
File structure
Top-level:
• src/ – Worker entrypoint + RAG pipeline code
– providers/ – wrappers for embedding and LLM calls (Gemini vs. Qwen)
– rag/ – chunking (chunk.ts), retrieval (retriever.ts), prompt assembly (prompt.ts), title dictionary
– utils/ – markdown parsing, schema definitions
– worker.ts – binds HTTP routes, marshals requests to RAG components
• scripts/ – CLI ingestion utilities
– ingest.ts, manual-ingest.ts, full-reindex.ts, generate-title-dictionary.ts
• tests/ – unit, integration, e2e
• config/build – tsconfig, vitest, playwright, wrangler.toml
README for additional insights
• Architecture described in narrative: Markdown → Plain→Chunks→Embeddings→Vector Store; /chat retrieves top-K chunks + calls LLM; returns answers + source refs.
• Uses Cloudflare Vectorize binding for persistent vector index.
• Widget is a simple JavaScript client that calls the /chat Worker endpoint.
• Supports Gemini (via Google API) or Qwen provider.
• Admin endpoints clear and upsert vectors for full-reindex or manual ingest.