How harshalakkaraju/pdf-whisperer works
Project Identification
• PDF-Whisperer is an intelligent PDF reader powered by LLMs with two modes of operation: cloud-based using OpenAI’s API and local using Ollama combined with Litellm.
• Its purpose is to extract, analyze, and summarize text from PDFs with additional features like entity recognition and sentiment analysis.
Analyzing the File Structure
• Top-level files include:
• README.md: Contains feature descriptions and quick-start instructions.
• app.py: Likely the main application file that runs the interactive Streamlit interface.
• requirements.txt: Lists Python dependencies.
• The faiss_db directory holds:
• index.faiss and index.pkl: Represent the FAISS database used to store or index vector representations from PDF documents.
• The presence of these files suggests a straightforward architecture with a single entry point (app.py), while the FAISS database forms an integral part of the data retrieval or text-search mechanism.
Key System Components
a. User Interface (UI) Layer:
• Represented by the Streamlit application launched via app.py.
• UI components enable user interactions by submitting PDFs and receiving results such as summaries or answers.
b. Controller/Application Logic:
• Embedded within app.py that acts as the orchestration layer.
• Handles routing requests to either the cloud-based LLM (via OpenAI API) or the local processing pathway.
c. Cloud-based Processing Module:
• An “OpenAI API” component which receives requests for LLM processing.
d. Local Processing Chain:
• Separate components for “Ollama” and “Litellm Proxy.”
• These work together to enable offline processing. Indicate that the Litellm proxy interfaces with local models (illustrate a connection to “Local LLM Models”).
e. Data Storage / FAISS Database:
• FAISS database (index.faiss and index.pkl) as a critical component.
• Links between the document ingestion module and the FAISS storage system for indexing and retrieving vector representations.
f. PDF Parsing and Content Extraction Module:
Relationships and Interactions
• From the User Interface to the Controller.
• The Controller splits the workflow into two parallel paths: one towards the OpenAI API and one towards the local LLM chain using Ollama and Litellm.
• The PDF parsing/extraction module should feed data into both processing pipelines.
• Results from both processing paths flow back to the Controller and then to the UI for user presentation.
Architectural Patterns and Design Principles
• Separation of concerns:
• UI versus backend processing.
• Cloud-based versus offline processing pipelines.
• Use of a vector database (FAISS) as an underlying search/indexing mechanism.
• Dependency on external services (OpenAI, Ollama, Litellm) and local file/data management.