How gurpreetkaurjethra/multi-pdfs_chatapp_ai-agent works
Project type and purpose
• This is a full-stack, single‐page Streamlit web application (“chatapp.py”) that empowers users to upload multiple PDF documents, convert them into embeddings via LangChain + Google Gemini Pro (and other LLMs), store them in a FAISS vector store, and interact with a conversational QA chatbot in real time.
File structure
• Top-level files:
• chatapp.py – main Streamlit entry point (backend + frontend glue)
• requirements.txt – Python dependencies (Streamlit, langchain, google-generativeai, faiss-cpu, PyPDF2, etc.)
• docs/ – sample PDF corpus
• img/ – architecture & demo screenshots
• README.md, .gitignore, LICENSE
• No separate “frontend” vs “backend” folders—Streamlit serves both UI and backend code from one script.
• External config: expects a .env with GOOGLE_API_KEY.
Extract architecture details from README
• Pipeline:
1. PDF Loading (PyPDF2)
2. Text Chunking (LangChain Sliding Window)
3. Embedding Generation (Google Gemini Pro via langchain_google_genai or other LLMs)
4. Vector Storage & Similarity Search (FAISS)
5. Chat Interface & Response Generation (LLM + chunk retrieval)
• External dependencies: Google Generative AI API (embeddings + chat), optionally OpenAI, Anthropic, Llama2.
• UI: Streamlit front end in the same Python process.
Main components
a. Users (Browser) – end users interact via web browser
b. Streamlit Application Server (chatapp.py)
• UI Layer (Streamlit widgets: file uploader, chat interface)
• Controller/Orchestration layer that invokes processing modules
c. PDF Loader Module (PyPDF2)
d. Text Chunker Module (LangChain text-splitter, Sliding Window)
e. Embedding Service (LangChain abstraction calling Google Generative AI API or alternative LLM endpoints)
f. Vector Store (FAISS in-memory or on-disk)
g. Retriever Module (calls FAISS similarity search)
h. Response Generator (passes selected chunks + user query to LLM API, returns answer)
i. External Services
• Google Generative AI API (embeddings + chat completions)
• Optional: OpenAI / Anthropic / Llama2 endpoints
j. Storage/Config
• Local .env for API keys
• docs/ folder for sample inputs (not in production)
Relationships and data flows
1. Browser → Streamlit UI: user uploads PDFs / sends chat queries
2. Streamlit UI → PDF Loader → extracted text
3. Extracted text → Text Chunker → text chunks
4. Chunks → Embedding Service → vector embeddings
5. Embeddings → FAISS Vector Store → stored vectors
6. On user question:
a. Streamlit UI → Retriever → queries FAISS → top-k chunks
b. Selected chunks + user question → Response Generator → LLM API → formatted answer
7. Answer → Streamlit UI → Browser
Architectural patterns and principles
• Pipeline/RAG pattern: ingestion → vectorization → retrieval → generation
• Separation of concerns within the monolithic script: loader, chunker, embedding, storage, retrieval, UI
• Externalize secrets/config via .env
• Pluggable LLM backends via LangChain abstractions
Relevant technologies/frameworks
• Streamlit (UI + server)
• PyPDF2 (PDF parsing)
• LangChain (text splitting, embeddings, LLM chaining)
• langchain_google_genai (Google API adapter)
• faiss‐cpu (vector database)
• python‐dotenv (config)
• Python 3.x
Summary
• Capturing the essential pipeline stages and their interconnections is key.
• Be exhaustive but logical: group related steps under “Ingestion”, “Vector DB & Retrieval”, “LLM Interaction”, and “UI”.