How jfagan/llm-rag-invoice-haystack works
Project Type and Purpose
• This is a Retrieval-Augmented Generation (RAG) pipeline for invoice data extraction and Q&A, using:
• Haystack 2 for document retrieval
• Weaviate as the vector store
• Llama2 (hosted on Hugging Face) as the LLM
• Docker Compose for full containerization
File Structure
• Root:
• docker-compose.yml & dockerfile → container orchestration
• ingest.py → ingestion pipeline (PDF → embeddings)
• main.py → query/answer pipeline (CLI → answer)
• llm/ folder:
• llm.py, prompts.py, wrapper.py → LLM integration (prompt templating + HF API calls)
• data/ folder:
• invoice PDFs fed to ingest.py
• config.yml, requirements.txt → configuration & dependencies
README Insights
• “docker-compose up --build” spins up:
1. Application container
2. Weaviate vector database container
• ingest.py loads PDFs → vector embeddings in Weaviate
• main.py executes a Haystack retrieval + LLM generation pipeline
Main Components & Their Interactions
a. Data Source
• Local data/ directory (invoice_1.pdf, etc.)
b. Ingestion Service
• ingest.py reads PDFs → uses Haystack DocumentStore + Weaviate client → stores embeddings
c. Vector Store
• Weaviate container (Docker Compose service) — holds embeddings & metadata
d. Retrieval & LLM Service
• main.py initializes Haystack pipeline:
• Retriever queries Weaviate → retrieves top-k document chunks
• Reader/Generator uses Llama2 via Hugging Face API through llm.wrapper
e. External Services
• Hugging Face Model Hub (Llama2) — accessed via API token
f. Orchestration
• Docker Compose network connecting the app container and Weaviate
Architectural Patterns & Design Principles
• Pipeline pattern: ingest → index → retrieve → generate
• Separation of concerns:
• Ingest module vs. Query module
• Document storage (Weaviate) vs. compute (app container)
• Containerization ensures environment parity
Essential Details to Capture
• The llm/ folder’s role in abstracting prompt logic and API calls
• The fact that Weaviate is spun up locally but acts as an external vector DB
• How PDF files flow from disk → embeddings → vector store → downstream retrieval
• The user-facing CLI (main.py) feeding natural-language queries into the pipeline