How maharun0/cse299 works
Project Type and Purpose
• This is a hybrid document retrieval and conversational AI system that combines offline (local document processing and vector-based search) with online retrieval capabilities. The system is designed for securely processing personal or specialized documents, indexing them locally using FAISS, and then engaging users in interactive, conversational searches with the help of adaptive AI models.
File Structure
• Key Directories & Files:
• BotUtils: Contains core modules such as Chain.py, Chunker.py, DocumentLoader.py, OllamaStuffs.py, Prompt.py, Reranker.py, and VectorStore.py. These likely support document ingestion, processing, vector indexing, query formulation, and reranking of results.
• server: Houses main backend scripts (server/main.py, server/terminal_bot.py) and the vector database storage (server/vector_db with FAISS indexes). It also includes sample documents (aida.pdf, physics_book.pdf) used as input files.
• frontend: Contains the UI entry point (frontend.py) and supporting files (images). This represents the chat interface.
• main.py at the project root appears to act as an orchestrator that ties together the frontend, backend, MongoDB sessions, and calls to Ollama (external AI service).
• Reports and Tests: Provide demonstrations, proposals, and test cases but aren’t central to runtime architecture.
• Architectural Patterns:
• The clear separation of frontend, backend, and utility modules points toward a modular, layered approach. The BotUtils modules encapsulate specific RAG pipeline stages (document loading, chunking, vector search, reranking), while the server code manages storage of indexed data and integration with external services.
• The use of both local vector databases (e.g., FAISS) and external AI resources (Ollama) highlights a hybrid retrieval model.
README for Additional Insights
• The README emphasizes that the project’s core is to provide hybrid retrieval combining local indexing (ensuring privacy) and online mode for accessing additional computational resources.
• Key features include efficient document processing, vector-based search, reranking, and adaptive AI models.
• There’s a visual reference to a RAG pipeline image in the README, suggesting that stages of retrieval augmentation should be clearly represented.
a. Main Components:
• Frontend: Chat UI (frontend/frontend.py) from which user queries originate.
• Backend/API: Runs via main.py and server/main.py, managing the conversation flow.
• RAG Pipeline Utilities: Core modules in BotUtils that perform document loading (DocumentLoader), text extraction and chunking (Chunker), prompt generation (Prompt), query chaining (Chain), vector storage handling (VectorStore), and reranking (Reranker). You may also mention Ollama-related functions from OllamaStuffs.
• Vector Database: Represented by FAISS indexes in server/vector_db.
• External Services: MongoDB (for session management) and Ollama (for AI model inference) as indicated in the README.
• File Storage: Documents located in server/files (PDFs, physics book) which are processed by the system.
b. Component Relationships and Data Flow:
• User Query Flow: User inputs are sent from the Frontend to the Backend.
• Request Processing: The Backend orchestrates the RAG Pipeline; it directs the document processing modules (document loading, chunking) to handle local files.
• Vector Search: The processed document text is transformed into embeddings via BotUtils/VectorStore, using FAISS for nearest neighbor search.
• Reranking: The Reranker refines the search results, potentially using further cross-encoder calls.
• Response Generation: Based on the hybrid retrieval (offline processed content combined with possible online/contextual lookups via Ollama), the final conversational response is formulated and returned to the Frontend.
• Data Storage and External Dependencies: MongoDB manages session data and possibly logging of queries; Ollama supplies model inference (as per the prerequisites noted in the README).
c. Important Architectural Patterns and Design Principles:
• Modularity: Each component of the RAG pipeline is compartmentalized (utility modules under BotUtils).
• Hybrid Retrieval: Clear separation between local (on-device, privacy-preserving) processing and online mode (external AI model inference).
• Responsiveness: The system is designed for interactive conversation, indicating asynchronous or real-time data flow between components.
• Security & Privacy: Emphasis on processing sensitive documents locally before invoking external resources.
d. Relevant Technologies & Frameworks:
• FAISS: Used for vector indexing and similarity search.
• MongoDB: Manages sessions and possibly stateful interactions.
• Ollama: External AI service provider for model inference.
• Python Modules: The use of Python scripts throughout; consider using libraries like Flask or similar frameworks if applicable.