How ashishsinha5/rag_api works
Project Type and Purpose
• This is a backend service (microservice) for Retrieval Augmented Generation (RAG), exposing a RESTful API via FastAPI.
• It ingests user documents (PDF/HTML), builds a vector database for semantic search, and performs QnA by retrieving relevant chunks and passing them to an open-source LLM (llama.cpp).
High-Level Components
a. Client
• SwaggerUI or any HTTP client that sends requests to the API.
b. API Layer (FastAPI)
• main.py: defines endpoints (upload, query, init_llm).
• routes → delegates to service functions in load_data.py, vector_db.py, load_llm.py, prompts.py, utils.py.
c. Vector Database
• vector_db.py: abstracts creation and querying of the vector store (e.g., FAISS or in-memory index).
• Persists document embeddings by collection_name.
d. Document Processing
• load_data.py: handles file ingestion, chunking, text extraction.
• utils.py: shared utilities (e.g., text normalization).
e. LLM Service
• load_llm.py & llama2_config.yaml: initializes llama.cpp with configurable parameters.
• prompts.py: formats retrieved context + user query into a prompt.
f. Data Storage
• Local file system for uploaded docs and vector index files.
g. Configuration
• llama2_config.yaml: default model settings.
• requirements.txt / .gitignore.
Interactions & Data Flow
• Upload Document:
Client → POST /upload → FastAPI → load_data.py (extract/chunk) → vector_db.py (store embeddings) → Response.
• Query Document:
Client → POST /query → FastAPI → vector_db.py (retrieve top-N embeddings) → prompts.py (assemble prompt) → load_llm.py (call llama.cpp) → return generated answer.
• Init LLM Parameters:
Client → POST /init_llm → FastAPI → load_llm.py (update llama2_config.yaml and reinitialize model).
Architectural Patterns & Principles
• Layered Architecture:
• API Layer (HTTP interface)
• Service Layer (business logic: ingestion, retrieval, prompting)
• Data Layer (vector DB, file storage)
• Integration Layer (llama.cpp engine)
• Single Responsibility: each module (load_data, vector_db, load_llm, prompts) focuses on one concern.
• Configuration-as-Code: llama2_config.yaml drives model settings.
Technologies & Frameworks
• FastAPI (API framework + auto-generated SwaggerUI)
• llama.cpp (local LLM inference)
• Langchain (potentially for chaining; not in code but referenced)
• Python 3.x
• Vector store (e.g., FAISS under the hood)
• Uvicorn (ASGI server)