# azure-samples/design-and-evaluation-of-rag-solutions architecture

> Architecture diagram of the GitHub repository azure-samples/design-and-evaluation-of-rag-solutions, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/azure-samples/design-and-evaluation-of-rag-solutions
- Repository: https://github.com/azure-samples/design-and-evaluation-of-rag-solutions

## Overview

##### Project Type and Purpose
- This is a learning-resource/open-source toolkit for building a Retrieval-Augmented Generation (RAG) pipeline using Jupyter notebooks.
- It demonstrates best practices for:
- Document preparation (synthetic docs, HTML→Markdown, analysis)
- Chunking (semantic and rule-based)
- Indexing & retrieval (Azure AI Search)
- Augmentation & generation (Azure OpenAI)
- Testing & evaluation (QA tests, synthetic QA generations, end-to-end)

##### File Structure
- Top-level staging directories correspond to each pipeline phase:
2. Preparation and document analysis
3. Chunk processing
4. Search and retrieval
5. Testing and evaluation
6. Recommendations, code snippets, appendices
- Utility code lives in `rag_utils/` (shared functions).
- `data_in/` holds raw HTML source docs.
- `.env-sample` files in each stage indicate use of environment variables for Azure service keys.
- `requirements.txt` lists Python dependencies (Azure SDKs, Langchain, etc.).

##### Relationships and Data Flow
1. Raw Data → Preparation Phase
2. Prepared Markdown/JSON → Chunk Processing
3. Chunks → Indexing → Azure AI Search
4. Query → Azure AI Search → Retrieved Chunks
5. Retrieved Chunks + Query → Azure OpenAI → Generated Response
6. Generated Response → Testing & Evaluation

##### Architectural Patterns & Design Principles
- Modular pipeline with clear separations of concern (prep, chunking, indexing, retrieval, generation, evaluation).
- Environment-driven configuration (.env samples) for secrets/configs.
- Notebook-driven orchestration for educational transparency.
- Service-oriented integration points with Azure services.

##### Technologies & Frameworks
- Python 3.x, Jupyter Notebooks
- Azure AI Search SDK
- Azure OpenAI SDK
- Langchain (in 3.2)
- CSV/JSON for metadata
- pandas (implied) for data manipulation

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Storage
    RawData["Raw Data Ingestion"]:::dataStorage
    click RawData "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/tree/main/data_in/"

    %% Preparation Phase
    subgraph "Preparation Phase"
        genSynth["Synthetic Document Generator"]:::compute
        convHTML["HTML→Markdown Converter"]:::compute
        analyzeLen["Document Length Analyzer"]:::compute
        classifySec["Section/Sub-document Classifier"]:::compute
        detectDup["Duplicate Document Detector"]:::compute
    end
    click genSynth "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/2.-preparation-phase-and-document-analysis/2.1.-Generate-synthetic-documents/generate-synthetic-documents.ipynb"
    click convHTML "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/2.-preparation-phase-and-document-analysis/2.2.-Convert-HTML-to-markdown/convert_html_to_markdown.ipynb"
    click analyzeLen "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/2.-preparation-phase-and-document-analysis/2.3.-Length-of-the-documents/calculate_length_of_the_documents.ipynb"
    click classifySec "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/2.-preparation-phase-and-document-analysis/2.4.-Sub-document-analysis/classify_markdown_content.ipynb"
    click detectDup "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/2.-preparation-phase-and-document-analysis/2.5.-Duplicate-documents/find_duplicates.ipynb"

    %% Chunk Processing
    subgraph "Chunk Processing"
        semanticChunk["Semantic Chunking (GPT-4)"]:::compute
        ruleChunk["Rule-based Chunking (Langchain)"]:::compute
    end
    click semanticChunk "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/3.-chunk-processing/3.1.-Semantic-Chunking-with-GPT-4o/semantic_chunking_with_aoai.ipynb"
    click ruleChunk "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/3.-chunk-processing/3.2.-Chunking-with-Langchain-with-max-tokens-and-overlapping/chunking_with_max_tokens.ipynb"

    %% Indexing & Search
    subgraph "Indexing & Search"
        createIndex["Index Creation & Ingestion"]:::compute
        AzureSearch["Azure AI Search"]:::external
        rerank["Retrieval & Reranking"]:::compute
    end
    click createIndex "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/4.-search-and-retrieval/4.1.-create-index-and-index-documents/create_index_and_index_documents.ipynb"
    click rerank "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/4.-search-and-retrieval/4.4.-Search-results-re-ranking/rerank_chunks_and_generate_answer.ipynb"

    %% Augmentation & Generation
    AzureOpenAI["Azure OpenAI Service"]:::external

    %% Testing & Evaluation
    subgraph "Testing & Evaluation"
        testQuality["Search-Result Quality Tests"]:::compute
        genQA["Synthetic QA Pair Generation"]:::compute
        evalAns["Answer-Quality Evaluation"]:::compute
        endToEnd["End-to-End Pipeline Tests"]:::compute
    end
    click testQuality "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/5.-testing-and-evaluation/5.1.-Testing-search-results/search_and_answer_generation_tests.ipynb"
    click genQA "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/5.-testing-and-evaluation/5.2.-Automatic-generation-of-synthetic-QA-pairs/generate_synthetic_qa_pairs.ipynb"
    click evalAns "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/5.-testing-and-evaluation/5.3.-Evaluate-answer-quality/evaluate_answer.ipynb"
    click endToEnd "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/5.-testing-and-evaluation/5.4.-Testing-the-end-to-end-process/end_to_end_process.ipynb"

    %% Utilities & Config
    AIUtils["AI Helper Functions"]:::util
    TextUtils["Text Helper Functions"]:::util
    click AIUtils "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/rag_utils/ai_functions.py"
    click TextUtils "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/rag_utils/text_functions.py"
    Env["Environment Config (.env-sample)"]:::util
    click Env "https://github.com/azure-samples/design-and-evaluation-of-rag-solutions/blob/main/.env-sample"

    %% Data Flow Arrows
    RawData -->|"HTML files"| genSynth
    RawData -->|"HTML files"| convHTML
    RawData -->|"HTML files"| analyzeLen
    RawData -->|"HTML files"| classifySec
    RawData -->|"HTML files"| detectDup

    genSynth -->|"Prepared docs"| semanticChunk
    convHTML -->|"Markdown docs"| semanticChunk
    classifySec -->|"Sub-documents"| semanticChunk
    detectDup -->|"Filtered docs"| semanticChunk

    semanticChunk -->|"Chunks"| createIndex
    ruleChunk -->|"Chunks"| createIndex
    semanticChunk -->|"Chunks"| ruleChunk
    ruleChunk -->|"Chunks"| semanticChunk

    createIndex -->|"Index documents"| AzureSearch
    AzureSearch -->|"Document ingestion ack"| rerank

    rerank -->|"Search query & results"| AzureSearch
    AzureSearch -->|"Search results"| rerank

    rerank -->|"Augmented prompt"| AzureOpenAI
    AzureOpenAI -->|"Generated answer"| rerank

    rerank -->|"Generated answer"| testQuality
    rerank -->|"Generated answer"| genQA
    rerank -->|"Generated answer"| evalAns
    rerank -->|"Generated answer"| endToEnd

    %% Utility Calls
    genSynth -.-> AIUtils
    genSynth -.-> TextUtils
    convHTML -.-> AIUtils
    convHTML -.-> TextUtils
    analyzeLen -.-> AIUtils
    classifySec -.-> TextUtils
    detectDup -.-> TextUtils
    semanticChunk -.-> AIUtils
    ruleChunk -.-> AIUtils
    createIndex -.-> AIUtils
    rerank -.-> AIUtils
    testQuality -.-> AIUtils
    genQA -.-> AIUtils
    evalAns -.-> AIUtils
    endToEnd -.-> AIUtils

    %% Styles
    classDef dataStorage fill:#f2f2f2,stroke:#333,stroke-width:1px;
    classDef compute fill:#cce5ff,stroke:#333,stroke-width:1px;
    classDef external fill:#fff2cc,stroke:#333,stroke-width:1px;
    classDef util fill:#d9ead3,stroke:#333,stroke-width:1px;
```
