How azure-samples/design-and-evaluation-of-rag-solutions works
Project Type and Purpose
• This is a learning-resource/open-source toolkit for building a Retrieval-Augmented Generation (RAG) pipeline using Jupyter notebooks.
• It demonstrates best practices for:
• Document preparation (synthetic docs, HTML→Markdown, analysis)
• Chunking (semantic and rule-based)
• Indexing & retrieval (Azure AI Search)
• Augmentation & generation (Azure OpenAI)
• Testing & evaluation (QA tests, synthetic QA generations, end-to-end)
File Structure
• Top-level staging directories correspond to each pipeline phase:
2. Preparation and document analysis
3. Chunk processing
4. Search and retrieval
5. Testing and evaluation
6. Recommendations, code snippets, appendices
• Utility code lives in rag_utils/ (shared functions).
• data_in/ holds raw HTML source docs.
• .env-sample files in each stage indicate use of environment variables for Azure service keys.
• requirements.txt lists Python dependencies (Azure SDKs, Langchain, etc.).
Relationships and Data Flow
1. Raw Data → Preparation Phase
2. Prepared Markdown/JSON → Chunk Processing
3. Chunks → Indexing → Azure AI Search
4. Query → Azure AI Search → Retrieved Chunks
5. Retrieved Chunks + Query → Azure OpenAI → Generated Response
6. Generated Response → Testing & Evaluation
Architectural Patterns & Design Principles
• Modular pipeline with clear separations of concern (prep, chunking, indexing, retrieval, generation, evaluation).
• Environment-driven configuration (.env samples) for secrets/configs.
• Notebook-driven orchestration for educational transparency.
• Service-oriented integration points with Azure services.
Technologies & Frameworks
• Python 3.x, Jupyter Notebooks
• Azure AI Search SDK
• Azure OpenAI SDK
• Langchain (in 3.2)
• CSV/JSON for metadata
• pandas (implied) for data manipulation