How lalanikarim/ai-chatbot-rag works
Main components
• User Interface (UI): Represented by the Streamlit frontend. This is the entry point through which users interact with the chatbot.
• Web Scraper: A component that handles fetching and parsing website content and PDFs using tools like BeautifulSoup and PyMuPDF.
• Data Processing & Embedding Module: The part that processes the scraped text. It leverages Sentence-Transformers and possibly DocArray for converting text into embeddings that can be used for retrieval.
• Conversational Memory Handler: This component maintains session context and follow-up conversation state.
• Retrieval Module (RAG pipeline): This integrates Langchain with the Llama model (via llama-cpp-python and the Mistral model). The retrieval layer fetches the relevant pieces of text based on the embeddings and user queries.
• Model Inference Engine: The backend process that handles natural language generation using the downloaded Mistral model.
Relationships and interactions
• The UI (Streamlit) receives user input and sends queries to the backend engine.
• The Web Scraper feeds content into the Data Processing module to create a knowledge base (embeddings).
• The Embedding Module prepares the content for retrieval and stores it in the appropriate format (possibly within DocArray structures).
• When a user query is received, the Retrieval Module matches it against the pre-processed knowledge base. Results are then passed on to the Model Inference Engine.
• The Model Inference Engine (utilizing Llama-cpp-python with Mistral) processes the query along with context (via conversational memory) to generate responses.
• The conversational memory handler ensures context flow across session turns.
• Finally, the response is returned back to the UI for display.
Architectural patterns & design principles
• Modular design: Though your codebase is small, separations in concerns (scraping, processing, retrieval, model inference, and presentation) are conceptually distinct.
• Data flow pattern: Clearly illustrate the pipeline’s sequential stages—from content ingestion and embedding to retrieval augmented generation.
Relevant technologies and libraries
• Streamlit (for the UI layer).
• BeautifulSoup and PyMuPDF (for web scraping and PDF processing).
• Sentence-Transformers and DocArray (for creating and managing embeddings).
• Langchain (to orchestrate the retrieval augmented generation pipeline).
• llama-cpp-python with the Mistral model (for model inference and natural language generation).
• Python runtime and dependencies managed via requirements.txt.