How gurmeet289/agentic-rag-system-multi-file-upload---embed---chat- works
Project Type and Purpose
• This is a “full-stack” single-page application built with Streamlit for the UI and Python scripts for the backend logic.
• Purpose: let users upload multiple PDF/TXT docs, embed them into a local vector store (ChromaDB), then chat and automatically route questions to a calculator, RAG (using embedded docs), or web search.
File Structure
• Top-level directory:
• app.py – Streamlit UI, tabbed interface (“Upload & Embed” and “Chat/Ask”)
• rag_core.py – defines the agentic routing logic (calculator / RAG / web search) and LLM calls via LangChain & Gemini
• ingest_incremental.py – handles incremental ingestion: file load, chunking, embedding, saving into ChromaDB
• data/ – raw uploaded documents
• chroma_db_hf/ – persistent vector store on disk
• .env – API keys (Gemini)
• No traditional “frontend vs backend” directories, but the separation is logical: app.py (presentation) vs rag_core.py & ingest_incremental.py (business logic) vs vector store & data folder (persistence).
README for Additional Insights
• Confirms three workflows: Upload & Embed, Chat/Ask, and internal routing logic.
• Lists key technologies (Streamlit, LangChain, ChromaDB, local HF embeddings, Gemini, duckduckgo-search).
a. Main Components
• User Browser (Streamlit UI)
• Streamlit App (app.py)
• Ingestion Module (ingest_incremental.py)
• Routing & LLM Module (rag_core.py)
• Vector Store (ChromaDB on disk)
• Embedding Engine (local HuggingFace model)
• LLM Service (Gemini via LangChain)
• Tools / External Services:
• Calculator (native Python)
• Web Search (DuckDuckGo API)
• Data Persistence: raw docs folder + vector store folder + .env for config
b. Relationships & Data Flows
1. Upload & Embed Flow:
• User → UI → Ingestion Module → Chunk → Embed → store in ChromaDB
2. Chat / Ask Flow:
• User question → UI → Routing Module
– If math → Calculator → return answer
– If docs context → Vector Store lookup + LLM → return answer + sources
– If time-sensitive → Web Search → return answer + snippet
3. Both flows return results back to UI for display
c. Architectural Patterns & Design Principles
• Agentic Tool Routing Pattern (deterministic routing to tools)
• Retrieval-Augmented Generation (RAG)
• Incremental Ingestion (idempotent embeddings)
• Persistent Vector Store for statefulness
d. Key Technologies & Frameworks
• Presentation: Streamlit (app.py)
• Business Logic: Python scripts, LangChain wrappers
• Embeddings: local HuggingFace models (no API cost)
• Vector Store: ChromaDB
• LLM: Google Gemini via LangChain-Google-GenAI
• Search: duckduckgo-search