How varunnthakuur/hands-on-ai-rag-using-llamaindex-3830207 works
Project type and purpose
• This is an educational code repository accompanying the “Hands-On AI: RAG using LlamaIndex” LinkedIn Learning course.
• Its purpose is to guide learners through the concepts and code needed to build, evaluate, and extend Retrieval-Augmented Generation (RAG) systems using the LlamaIndex framework.
File structure
• Top-level directories “01_Getting_Started” through “07_Modular_RAG” each contain notebooks or markdown files illustrating a specific stage in the RAG pipeline: environment setup, fundamentals, ingestion, querying, evaluation, advanced pre-/post-retrieval, and modular RAG.
• The “helpers” package contains reusable Python modules for prompts, text cleaning, and utilities.
• Configuration files (.devcontainer, .env_example), CI workflows (.github/workflows/main.yml), and requirements.txt support reproducible environments and continuous integration.
README for additional insights
• The README describes the course scope: fundamentals of RAG, LlamaIndex integration, end-to-end pipelines, and critical evaluation.
Main components
a. Data Sources
• Raw data files (PDFs, text corpora) used in “01_Getting_Started” notebooks
b. Pre-processing
• helpers/text_cleaning_helpers.py
• helpers/utils.py
c. Ingestion Pipeline
• LlamaIndex loaders (02_Fundamental_Concepts_in_LlamaIndex/02_03_Loading_Data.ipynb)
• Indexing (02_04_Indexing.ipynb)
• Storage backends (vector DB of choice, configured in .env_example)
d. Query Pipeline
• Retrieval engines (02_06_Querying.ipynb, 03_Introduction_to_RAG/03_03_Query_Pipeline.ipynb)
• Prompt engineering (03_04_Prompt_Engineering_for_RAG.md)
e. Post-Retrieval Enhancements
• Node re-ranking, prompt compression, self-correcting (06_Advanced_RAG_Post_Retrieval)
f. Modular RAG Extensions
• Hybrid retrievers, agentic RAG, ensembles (07_Modular_RAG)
g. Evaluation
• Metrics and evaluation set creation (04_RAG_Evaluation)
h. Orchestration & Environment
• Jupyter notebooks as choreographers
• .devcontainer for environment, CI workflows for testing
Relationships and data flow
1. Raw Data → Pre-processing → Ingestion Pipeline → Vector Store
2. User Query → Query Pipeline → Retrieved Documents
3. Retrieved Documents → Post-Retrieval Modules → Final Prompt
4. Final Prompt → LLM → Model Response
5. Model Response + Ground Truth → Evaluation Module
Architectural patterns and design principles
• Pipeline pattern: clear separation of ingestion, retrieval, post-processing, and evaluation stages.
• Modular design: each stage is implemented in its own directory/notebook, helpers package centralizes reusable code.
• Pluggable backends: vector database and LLM providers are configurable via environment variables.