How nayankr8625/wordpressaibot works
Project Type and Purpose
• This is a full-stack chatbot system leveraging Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) techniques to serve context-aware answers on WordPress blogs.
• Key functions: content ingestion (WordPress API), embedding generation (Sentence-BERT), vector indexing (Faiss), RAG processing, CoT logic, LLM query orchestration, and UI integration (Streamlit local demo & WordPress plugin).
File Structure
• Top-level orchestrators: • app.py (likely Flask/FastAPI service) • streamlit_app.py (demo UI) • Dockerfile/requirements.txt
• Data ingestion: • data_collection/data_collection.py
• Embedding/indexing: • embedding/create_index.py
• Storage: • doc_embedding_store/… (prebuilt .faiss + .pkl indexes per document set)
• RAG & LLM logic: • gen_ai/prompts.py • gen_ai/query_llm.py
• Presentation template: • template/htmlTemplate.py (for WordPress plugin)
Extract Architectural Insights from README
• Components called out: Data Retrieval (WP API), Embedding Generator, Vector Database (Faiss), RAG Processor, Chain of Thought Module, User Interface (chat widget or Streamlit).
• Deployment: packaged as a Docker image, integrated via a WordPress plugin.
Main Components & Their Interactions
a. Data Retrieval Service
• Fetches blog content through WordPress REST API (data_collection).
• Outputs raw text to embedding pipeline.
b. Embedding Generator
• Uses create_index.py (Sentence-BERT) to convert text to vectors.
• Stores vectors + metadata in Faiss index + pickle files (doc_embedding_store).
c. Vector Database (Faiss)
• Provides high-speed semantic similarity search for RAG retrieval.
d. RAG Processor & CoT Module
• query_llm.py orchestrates:
1. Receive user query via app.py or streamlit_app.py
2. Retrieve top-k embeddings from Faiss
3. Construct RAG prompt with Chain-of-Thought scaffolding (prompts.py)
4. Send to LLM (OpenAI, etc.) and receive response
e. Application Layer / API
• app.py: exposes REST endpoints for the WordPress plugin or external UIs
• streamlit_app.py: local/demo UI that consumes same RAG endpoints
f. Front-End Integration
• template/htmlTemplate.py: chat widget HTML + JS for WordPress
• Deployed as a WordPress plugin, invoking the external API
Architectural Patterns & Design Principles
• Separation of concerns: ingestion, processing, storage, API, UI
• Micro-service-style modules (each directory = logical service)
• Stateless API layer with external stateful vector DB
• Containerized deployment (Dockerfile) for portability
• Precomputed indexes for efficient startup and scaling
• Optional: Deployment artifacts (Docker container, volumes for doc_embedding_store)
Checklist Before Finalizing:
[ ] All six major components represented
[ ] Two workflows clearly delineated (Indexing vs Query)
[ ] Technology logos or annotations (Faiss, SBERT, OpenAI, Docker)
[ ] Environment boundaries (WP site vs Chatbot service)