How ilyarice/rag-challenge-2 works
Project Type and Purpose
• This is a data-processing and question-answering pipeline built around Retrieval-Augmented Generation (RAG).
• It ingests PDF annual reports, parses and serializes them, builds a vector index, retrieves relevant contexts, reranks them, and finally issues LLM prompts to produce structured answers.
High-Level Architectural Components
A. CLI / Orchestrator
• Entry point for user commands (download models, parse PDFs, build index, answer questions).
• Parses flags/configs, invokes pipeline stages in correct order.
B. Ingestion Layer
• PDF Ingestion Service
– Reads raw PDF files (from local disk, S3, etc.)
– Optionally unzips archives and preserves metadata (company, year).
• Preprocessing Interface
– Responsible for batching, parallelization parameters, and dispatching to the PDF Parser.
C. Parsing & Serialization
• PDF Parsing Service
– Uses Docling (or similar) to extract raw text, tables, images.
– Outputs a structured JSON representation per document.
• Table Serialization Service
– Normalizes table structures into a serialized, queryable form.
D. Indexing & Storage
• Document Store
– A blob store or file system for parsed JSON artifacts.
• Vector Indexer
– Embedding encoder (OpenAI/Gemini API or on-prem model).
– Builds a vector search index (e.g., Faiss, Annoy).
• Metadata Store
– Tracks document IDs, parent-child relationships, and vector pointers.
E. Retrieval & Reranking
• Retriever
– Given a question, queries the vector index for top-k candidate chunks.
– Applies parent-document retrieval logic to group related chunks.
• Reranker
– Ranks candidate chunks via a lightweight LLM call or heuristic model.
F. Prompt Construction & LLM Interface
• Prompt Generator
– Applies structured output templates, chain-of-thought prompting.
– Inserts reranked contexts and question into a final LLM prompt.
• LLM Client
– Connects to OpenAI/Gemini API, handles retries, rate limits.
– Returns the final answer.
G. Query Router (for multi-company mode)
• Dispatches the question to multiple company indices.
• Merges results, applies comparison logic.
Relationships and Data Flow
1) User issues a command via CLI → Orchestrator
2) Orchestrator calls Ingestion → parsed documents → Table Serializer → stored JSON
3) Orchestrator calls Indexer → builds vector index + metadata
4) Upon question: Orchestrator passes query → Retriever → Reranker → Prompt Generator → LLM Client
5) Answer returned to CLI output or written to file
CLI/Orchestrator → Ingestion → Parsing → Serialization → Indexing → Retrieval → Reranking → Prompting → LLM → Output
Architectural Patterns & Design Principles
• Modular, stage-based pipeline (each stage can be developed/tested independently).
• Config-driven (via YAML/JSON) for easy switching of models/configs.
• Parallelism abstraction (each stage exposes a “max_workers” or “batch_size” parameter).
• Clear separation of compute-intensive (PDF parsing, embedding) vs. I/O (disk, API calls).
• Stateless services where possible (makes scaling simpler).
Summary
Recode this project using a strictly modular pipeline architecture. Ensure each stage:
• Has a single responsibility
• Communicates via well-defined payloads
• Can be independently tested and scaled
Following this design will let you reconstruct the solution from scratch in a clean, maintainable way, while preserving all of the original RAG functionality.