How kartikeya2006/pdf_qa works
Project Type and Purpose
• This is a full-stack web application (“PDF Q&A App”) that lets users upload PDF documents and then ask natural-language questions about their contents.
• Frontend: Next.js + Tailwind CSS (deployed to Vercel)
• Backend: FastAPI + Uvicorn (deployed on AWS EC2)
• Core AI: LangChain/LlamaIndex for vector-based indexing and OpenAI LLM calls
High-Level File Structure
• frontend/qa_pdf
• Next.js app folder with React components, global styles, config (tsconfig, next.config.ts)
• UI components for upload and chat
• backend
• main.py (FastAPI entrypoint)
• pdf_processor.py (extracts text via PyMuPDF)
• qa_engine.py (builds/loads LlamaIndex vector stores and issues LLM queries)
• db.py + models.py + Alembic migrations (SQLite “pdfs.db” for metadata)
• uploads/ (raw PDFs)
• indices/ (per-PDF vector stores: docstore.json, graph_store.json, index_store.json, etc.)
Main Components
A. User’s Web Browser (Client)
B. Frontend App (Next.js server + static assets)
C. Backend API (FastAPI/Uvicorn)
D. PDF Storage (uploads/ on server disk)
E. PDF Processing Service (pdf_processor.py)
F. Indexing & QA Service (qa_engine.py with LlamaIndex/LangChain)
G. Persistent Metadata Store (SQLite via db.py + Alembic)
H. Vector Store Filesystem (indices/ per-PDF vector data)
I. External LLM Service (OpenAI API)
Relationships & Data Flow
1. User → Next.js Frontend: Upload PDF or ask a question
2. Next.js → FastAPI “/upload” endpoint: HTTP POST with file
3. FastAPI → Disk: save PDF to uploads/
4. FastAPI → pdf_processor.py: extract text from PDF
5. pdf_processor → qa_engine.py: ingest text into LlamaIndex → writes vector files under indices/
6. Backend → SQLite via db.py: record PDF metadata (filename, index path)
7. User → Frontend: renders upload success or file list
8. User → Frontend: enter question, Frontend → FastAPI “/query” endpoint with {pdf_id, question}
9. FastAPI → qa_engine.py: load vector store from indices/, call LlamaIndex QA chain which issues requests to OpenAI API
10. OpenAI API → qa_engine → FastAPI → Frontend → User: return contextual answer
Architectural Patterns & Design Principles
• Separation of Concerns:
• Frontend handles UI, routing, styling
• Backend handles ingestion pipeline, indexing, QA logic, persistence
• Layered Service:
• Storage layer (PDF files + vector files + SQLite)
• Processing layer (pdf_processor, qa_engine)
• API layer (FastAPI endpoints)
• Modular Indexing Pipeline: ingest → index → query
• CI/CD Targets: Vercel for frontend, EC2 for backend
• Frontend: React, Next.js, Tailwind CSS, TypeScript
• Backend: Python 3.9+, FastAPI, Uvicorn, PyMuPDF, LangChain/LlamaIndex, SQLite + Alembic
• External: OpenAI REST API