How datatalksclub/llm-zoomcamp works
Project Type and Purpose
• This is not a production codebase but an educational “course repository” for the LLM Zoomcamp. It guides students through building Retrieval‐Augmented Generation (RAG) systems in Python, covering:
• Introduction to LLMs and RAG (Elasticsearch, OpenAI API)
• Vector search (embeddings, vector DBs)
• Evaluation (search/LLM evaluation)
• Monitoring (dashboards, online evaluation)
• Best practices (hybrid search, reranking)
• A full end‐to‐end example project
File Structure
• Top‐level directories “01-intro” through “06-project-example” correspond to modules, each containing scripts/notebooks, data, and READMEs.
• The “cohorts” directory houses hands‐on lab code, Docker configurations, and data for in‐class exercises.
• Supporting files include:
• docker-compose.yaml, Dockerfile (for reproducible environments)
• .ipynb notebooks (illustrating pipeline steps interactively)
• JSON/CSV data artifacts (documents, embeddings, ground‐truth)
• Monitoring configs (Grafana JSON, dashboards)
Core System Architecture for a RAG Pipeline
A. Content Ingestion & Preprocessing
– Source documents (PDFs, JSON, CSV)
– Preprocessor component (tokenization, cleaning)
– Output: embeddings + metadata
B. Indexing & Storage Layer
– Elasticsearch (text search)
– Vector database (e.g., FAISS, Pinecone)
C. Retrieval & Orchestration Service
– API / Python orchestrator (LangChain or custom)
– Query dispatcher:
• Keyword search → ES
• Vector query → Vector DB
– Results aggregator
D. LLM Inference Engine
– OpenAI API / HF inference service
– Prompts with retrieved context → generates answers
E. Evaluation Pipeline
– Offline evaluator (notebooks scoring search and LLM outputs)
– Online evaluator (LLM-as-judge, user feedback)
– Ground-truth comparator (CSV of Q&A)
F. Monitoring & Dashboarding
– Metrics collector (logging API calls, latencies, accuracy)
– Grafana (dashboard JSON, alerts)
G. User Interface (optional)
– Jupyter demos or minimal web UI (Streamlit/Flask)
Relationships and Data Flow
1. Documents → Preprocessor → Indexes (ES & Vector DB)
2. User Query → Orchestrator → Retrieval Services → Top-K contexts
3. Context + Query → LLM API → Answer
4. Answer → Evaluation Module → Metrics
5. Metrics → Monitoring Dashboard
• Bi-directional feedback from Monitoring back to Best Practices (e.g., retraining, prompt tuning).
Architectural Patterns & Design Principles
• Pipeline pattern: clearly separate ingestion, retrieval, inference, evaluation, monitoring.
• Modular microservices (or Python modules) for each step—easy to swap components (e.g., replace ES with Pinecone).
• Configuration‐as‐code (Docker/Docker Compose, JSON dashboards).
• Reproducibility via notebooks and versioned data artifacts.
Relevant Technologies / Libraries
• Python + Jupyter Notebooks
• Elasticsearch (text search)
• FAISS / other vector store
• OpenAI API / Hugging Face models
• LangChain or custom orchestrator scripts
• Docker, Docker Compose
• Grafana for monitoring
• Pandas/NumPy for evaluation scripts