How omkanekar28/rag-implementation works
Project type and purpose
• This is a full-stack web application implementing a Retrieval-Augmented Generation (RAG) pipeline for movie recommendation.
• It provides both a command-line interface (CSV-based retrieval and optional LLM-driven explanation) and a Flask web front end for interaction.
File structure
Top-level:
• data/ – raw and cleaned CSV datasets
• src/ – all Python modules and Flask app
• requirements.txt, readme.md
• screenshots/ – UI mockups
Inside src/:
• app.py – Flask entry point, routes, UI glue
• config.py – all hard-coded paths and model identifiers
• csv_cleaning.py – raw → cleaned CSV pipeline
• csv_to_faiss.py – builds FAISS index from cleaned CSV
• faiss_store/ – persisted FAISS index and metadata pickle
• recommend_movies.py – pure retrieval using FAISS
• recommend_movies_llm.py – retrieval + LLM reasoning
• prompts.py – templating LLM prompts
• models.py – wrapper for SentenceTransformer embeddings
• static/ + templates/ – web UI assets and HTML
Key insights from README
• Two modes: simple FAISS retrieval, and RAG (retrieve + LLM via Hugging Face)
• Tech stack: Python, Flask, FAISS, SentenceTransformers, Hugging Face models
• Modular design: each stage is a separate script/module
Main components
A. Data Layer
– Raw CSV (data/movie_dataset.csv)
– Cleaned CSV (data/cleaned_movie_dataset.csv)
– FAISS Index (src/faiss_store/movies.index + metadata pickle)
B. Indexing Pipeline
– csv_cleaning.py → output cleaned CSV
– csv_to_faiss.py → build FAISS index from embeddings
C. Embedding Service
– models.py using SentenceTransformer
– Generates vector representations for documents and queries
D. Retrieval Service
– FAISS index search (recommend_movies.py)
– Returns top-N nearest movie vectors + distances
E. RAG/Generation Service
– prompts.py constructs LLM prompt from query + retrieved docs
– recommend_movies_llm.py calls Hugging Face model via config.py to generate reasoning
F. Web Application
– app.py defines Flask routes:
• “/” → index.html
• “/recommend_movies” → display_results.html
• “/recommend_movies_llm” → display_results_llm.html
– static assets (CSS, images) and Jinja2 templates
G. External Dependencies
– FAISS library for similarity search
– SentenceTransformers for embeddings
– Hugging Face Transformers for LLM inference
Relationships & data flow
• Offline: Raw CSV → [csv_cleaning.py] → Cleaned CSV → [csv_to_faiss.py] → FAISS Index
• Online user query:
1. User submits a description + N via web form → Flask route
2. models.py encodes query → vector
3. FAISS index search → top-N results
4a. Simple mode: return titles + distances → display_results.html
4b. RAG mode: combine retrieved docs + user query → prompt → LLM → structured explanation → display_results_llm.html
• Config parameters (model path, index path) come from config.py
Architectural patterns & principles
• Layered architecture: Data → Indexing → Retrieval → Generation → Presentation
• Separation of concerns: each major function (clean, index, retrieve, generate, serve) lives in its own module