How bhushanmaheshwari/langchain-chatwithfiles works
Project type and purpose
• This is a full-stack conversational-AI web application built in Python (likely using Streamlit given the pages/ folder and .py naming).
• It lets users upload PDF documents, ingests and vectorizes their content, and then provides an interactive chat interface to query those documents. There is also a logs page for transaction history.
File structure
• Top-level directories:
• pages/ : the Streamlit front-end entrypoints (1_📁_Chat_with_PDF.py, 2_📝_Logs.py)
• features/ : business-logic modules (upload.py handles PDF uploads, conversation.py manages the QA loop)
• processing/ : data-pipeline modules (ingest.py, chunk.py, vectordb.py, storage.py, start.py)
• utils/ : supporting utilities (login.py, session.py, logs.py)
• data/ : persistent storage (raw PDFs organized by project ID and possibly logs)
• htmlTemplates.py : custom HTML snippets for UI
• 👋_Hello.py : greeting/landing-page logic
• Supporting files:
• requirements.txt (Python dependencies, likely includes LLM SDK, vector-DB client, Streamlit)
• .devcontainer/ (development container config)
README for additional insights
• Screenshots show an introduction page, basic conversational AI interface, extraction prompts and a logs view.
a. Main components
1. User Interface (Streamlit front-end):
• Pages: Chat with PDF, Logs
• HTML templates for layout
2. Features Layer:
• Upload handler (PDF upload & validation)
• Conversation manager (query vector DB, call LLM API, format responses)
3. Processing Pipeline:
• Ingestion orchestrator (start.py)
• Chunking service (chunk.py)
• Vectorization & storage (vectordb.py + storage.py)
4. Data Stores:
• File system (raw PDFs under data/)
• Vector database (Chroma/FAISS or cloud-hosted)
• Transaction logs (could be flat files or a DB, surfaced via utils/logs.py)
5. External Services:
• LLM API (OpenAI, Anthropic, etc.)
• Authentication provider (if any, via utils/login.py)
b. Relationships and interactions
• User → Streamlit UI → Features.upload → data storage
• User question → Features.conversation → Processing.vectordb (query embeddings) → LLM API
• Ingestion job starts (manual or on upload) wired by processing/start → chunk → vectordb → storage
• All user actions logged by utils/logs → logs store → surfaced by Logs page
c. Architectural patterns/design principles
• Separation of concerns: UI (pages), business logic (features), data pipeline (processing), utilities (utils)
• Modular pipeline for ingestion (chunking, vectorization, storage)
• Stateless UI with session management (utils/session.py)
d. Key technologies/frameworks
• Python 3.x, Streamlit (front end), possibly FastAPI or direct Streamlit callbacks
• Vector DB client (Chroma/FAISS)
• LLM SDK (OpenAI or similar)
• PDF parser (PyPDF2, pdfminer)
• Containerization (via .devcontainer)