How gaurav4050/pdfreader2 works
Project type and purpose
• This is a Python-based backend service (FastAPI) for document ingestion, embedding, vector storage, semantic search, reranking, and summarization using an external LLM (Gemini).
• It provides APIs (via uvicorn main:app) to load documents, generate embeddings, query a vector store, rerank hits, and produce summaries.
File structure
• Top level:
• main.py – FastAPI entry point
• embedding/ – code to generate embeddings for text chunks
• ingest/ – ingestion scripts (e.g., ingest from Google Drive)
• vectorstore/ – Milvus client wrapper
• utils/ – helper modules (query embedding, grouping, hashing, PDF parsing, reranking, summarization)
• ground_truth.json – evaluation/reference data
• No frontend; single backend service.
• Configuration handled in code; external dependencies: Milvus, LLM API.
Insights from README
• Minimal: runs with “uvicorn main:app --reload”.
• Implies FastAPI with endpoints defined in main.py.
Detail
• Every util module as a subcomponent if desired (e.g., PDF parsing, hashing, grouping).
• Ground_truth.json only if illustrating evaluation flows.