# azure-samples/azure-open-ai-rag-oyd-text-images architecture

> Architecture diagram of the GitHub repository azure-samples/azure-open-ai-rag-oyd-text-images, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/azure-samples/azure-open-ai-rag-oyd-text-images
- Repository: https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images

## Overview

##### Project Type and Purpose
- This is a full-stack, serverless RAG (Retrieval-Augmented Generation) application specialized for PDF documents containing both text and embedded images.
- Purpose: Ingest PDFs, extract text chunks and images, index them via Azure AI Search with embeddings, and serve a chat-style demo (web app or notebook) that returns LLM responses with text and image citations.

##### Top-Level Components (based on file tree & README)
a. Infrastructure as Code
- Bicep files (`bicep/`, `bicep/helpers/`, `helper.sh`) define Azure resources: Blob Storage, Function App, Cognitive Search, OpenAI deployments.
b. Data Ingestion Pipeline
- Azure Blob Storage: `raw_data/` and `prepared_data/` paths.
- Azure Function (`azure-function/`): Python code triggered on blob upload to split PDFs, extract text & images, and write processed artifacts back to blob.
c. Indexing Service
- Azure AI Search (`azure-ai-search` referenced in README): configured with data source, skillset, index, and indexer to vectorize text via OAI embedding model and link images to text chunks.
d. Application Runtime
- Demo Web App (`demo-app/`): Python (Streamlit) front end and server-side chat logic that calls Azure OpenAI (chat + embedding deployments) and retrieves context from AI Search.
- Demo Notebook (`demo-notebook/`): Jupyter notebook version mirroring the client/server interactions.
e. Supporting Artifacts
- CI/CD workflows (`.github/workflows/hugo.yml` for docs/site), Dockerfile for containerizing demo environment, workshop content (`workshop/`).

##### Relationships and Data Flows
- PDF Upload → Blob Storage (`raw_data`)
- Blob Trigger → Azure Function
- Extract text chunks + images → upload to Blob Storage (`prepared_data`)
- Scheduled/On-demand → AI Search Indexer
- Reads from `prepared_data` → embeds text via OAI → populates Search Index with text + image metadata
- User Query → Demo App Client
- Forwards to Server Chat logic → consults Azure OpenAI (chat) which under the hood queries AI Search for context → returns enriched response with citations
- Demo App Server → Blob Storage
- Serves image URLs for cited images back to client UI
- Client UI → Renders text + image citations

##### Architectural Patterns & Principles
- Event-driven pipeline: Blob storage events triggering Functions
- Serverless compute: Azure Functions + managed Azure AI Search + OpenAI service
- Separation of concerns: ingestion, indexing, AI chat, UI layers
- Infrastructure as Code: declarative Bicep definitions, reproducible deployments
- Extensibility: workshop content and helper scripts for learning and customization

##### Final Advice
- Don’t omit the workshop and notebook; indicate them as educational overlays.
- Extension points: where users could plug in custom PDFs or swap embedding models.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Infrastructure as Code
    subgraph "Infrastructure as Code"
        IaC["Bicep Files"]:::infra
        Helpers["Bicep Helpers"]:::infra
        Script["helper.sh"]:::infra
    end

    %% Data Ingestion
    subgraph "Data Ingestion" 
        RawStorage["Azure Blob Storage\n(raw_data)"]:::storage
        Function["Azure Function\n(blob trigger)"]:::compute
        PreparedStorage["Azure Blob Storage\n(prepared_data)"]:::storage
    end

    %% Indexing Service
    subgraph "Indexing Service"
        SearchService["Azure AI Search"]:::ai
        DataSource["Data Source"]:::ai
        Skillset["Skillset"]:::ai
        IndexEntity["Index"]:::ai
        Indexer["Indexer"]:::ai
    end

    %% AI Services
    subgraph "AI Services"
        OpenAIEmbeddings["OpenAI Embeddings"]:::ai
        OpenAIChat["OpenAI Chat"]:::ai
    end

    %% Application Runtime
    subgraph "Client/Server"
        DemoApp["Demo App\n(Streamlit)"]:::app
        DemoNotebook["Demo Notebook"]:::app
        UI["Client UI"]:::app
    end

    %% Supporting Artifacts
    subgraph "Supporting Artifacts"
        CICD["CI/CD Workflow"]:::infra
        DockerDef["Dockerfile"]:::infra
        SampleDocs["Sample Documents"]:::storage
        Workshop["Workshop Content"]:::infra
        Docs["README.md"]:::infra
    end

    %% Data Flows
    RawStorage -->|"BlobCreated event"| Function
    Function -->|"split PDFs,\nextract text + images"| PreparedStorage
    PreparedStorage -->|"indexer pull\ntext & image metadata"| DataSource
    DataSource --> Skillset
    Skillset --> IndexEntity
    IndexEntity --> Indexer
    Indexer -->|"embedding vectors via OAI"| SearchService
    UI -->|"HTTP /query"| DemoApp
    DemoApp -->|"calls Chat API"| OpenAIChat
    DemoApp -->|"requests context"| SearchService
    SearchService -->|"returns results"| DemoApp
    DemoApp -->|"serves image URLs"| UI
    DemoNotebook -->|"alternative client\n/hquery"| OpenAIChat

    %% Infra provisioning arrow
    IaC --> IaC
    Script --> IaC
    IaC -->|deploys| RawStorage
    IaC -->|deploys| Function
    IaC -->|deploys| SearchService
    IaC -->|deploys| OpenAIEmbeddings
    IaC -->|deploys| OpenAIChat

    %% Click Events
    click IaC "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/tree/main/bicep/"
    click Helpers "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/tree/main/bicep/helpers/"
    click Script "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/blob/main/helper.sh"
    click Function "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/tree/main/azure-function/"
    click DataSource "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/blob/main/bicep/helpers/datasource.json"
    click Skillset "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/blob/main/bicep/helpers/skillset.json"
    click IndexEntity "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/blob/main/bicep/helpers/index.json"
    click Indexer "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/blob/main/bicep/helpers/indexer.json"
    click DemoApp "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/tree/main/demo-app/"
    click DemoNotebook "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/tree/main/demo-notebook/"
    click CICD "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/blob/main/.github/workflows/hugo.yml"
    click DockerDef "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/tree/main/Dockerfile"
    click SampleDocs "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/tree/main/sample-documents/"
    click Workshop "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/tree/main/workshop/"
    click Docs "https://github.com/azure-samples/azure-open-ai-rag-oyd-text-images/blob/main/README.md"

    %% Styles
    classDef storage fill:#bbdefb,stroke:#2196f3;
    classDef compute fill:#c8e6c9,stroke:#4caf50;
    classDef ai fill:#ffe0b2,stroke:#ff9800;
    classDef infra fill:#e0e0e0,stroke:#9e9e9e;
    classDef app fill:#f0f4c3,stroke:#cddc39;
```
