# anirudhjbabu1/multimodel-rag-using-clip architecture

> Architecture diagram of the GitHub repository anirudhjbabu1/multimodel-rag-using-clip, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anirudhjbabu1/multimodel-rag-using-clip
- Repository: https://github.com/anirudhjbabu1/multimodel-rag-using-clip

## Overview

This notebook implements a PDF-based multimodal RAG workflow: PDF content is split into text and images, embedded with CLIP, stored and retrieved through FAISS, then formatted for OpenAI GPT-4.1. The repository contains only the notebook and README, and no source excerpts were available; relationships below reflect the README’s documented pipeline, not inspected notebook code.

Main entry points: [`Multimodel-RAG.ipynb`](https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb) (Text/image extraction)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["PDF Ingestion"]
  node_extract["Text/image extraction"]
end

subgraph group_retrieval["Embedding Retrieval"]
  node_text_embed["Text embeddings"]
  node_image_embed["Image embeddings"]
  node_faiss[("FAISS index")]
  node_retriever["Retriever"]
  node_topk["Top-K results"]
end

subgraph group_generation["Answer Generation"]
  node_format["LLM formatting"]
  node_gpt["OpenAI GPT-4.1"]
end

node_user(("User"))
node_pdf["PDF document"]

node_user -->|"provides"| node_pdf
node_pdf -->|"extracts content"| node_extract
node_extract -->|"embeds text"| node_text_embed
node_extract -->|"embeds images"| node_image_embed
node_text_embed -->|"stores embeddings"| node_faiss
node_image_embed -->|"stores embeddings"| node_faiss
node_faiss -->|"retrieves from"| node_retriever
node_retriever -->|"returns top-K"| node_topk
node_topk -->|"formats results"| node_format
node_format -->|"supplies context"| node_gpt

click node_extract "https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb"
click node_text_embed "https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb"
click node_image_embed "https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb"
click node_faiss "https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb"
click node_retriever "https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb"
click node_topk "https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb"
click node_format "https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_extract,node_user toneBlue
class node_text_embed,node_image_embed,node_faiss,node_retriever,node_topk toneAmber
class node_format,node_gpt toneMint
class node_pdf toneIndigo
```

## Components

### PDF Ingestion

- [Text/image extraction](https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb): `Multimodel-RAG.ipynb`

### Embedding Retrieval

- [Text embeddings](https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb): `Multimodel-RAG.ipynb`
- [Image embeddings](https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb): `Multimodel-RAG.ipynb`
- [FAISS index](https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb): `Multimodel-RAG.ipynb`
- [Retriever](https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb): `Multimodel-RAG.ipynb`
- [Top-K results](https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb): `Multimodel-RAG.ipynb`

### Answer Generation

- [LLM formatting](https://github.com/anirudhjbabu1/multimodel-rag-using-clip/blob/main/Multimodel-RAG.ipynb): `Multimodel-RAG.ipynb`
- **OpenAI GPT-4.1**

### Other components

- **User**
- **PDF document**

## Connections

- User → PDF document: provides (evidence: `README.md`)
- PDF document → Text/image extraction: extracts content (evidence: `README.md`)
- Text/image extraction → Text embeddings: embeds text (evidence: `README.md`)
- Text/image extraction → Image embeddings: embeds images (evidence: `README.md`)
- Text embeddings → FAISS index: stores embeddings (evidence: `README.md`)
- Image embeddings → FAISS index: stores embeddings (evidence: `README.md`)
- FAISS index → Retriever: retrieves from (evidence: `README.md`)
- Retriever → Top-K results: returns top-K (evidence: `README.md`)
- Top-K results → LLM formatting: formats results (evidence: `README.md`)
- LLM formatting → OpenAI GPT-4.1: supplies context (evidence: `README.md`)
