# mukeshram-07/multimodal-rag architecture

> Architecture diagram of the GitHub repository mukeshram-07/multimodal-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mukeshram-07/multimodal-rag
- Repository: https://github.com/mukeshram-07/multimodal-rag

## Overview

Multimodal RAG is an authenticated PDF knowledge-base and question-answering application. Users sign in, choose a collection, upload PDFs for parsing, chunking, embedding, and vector storage, then submit queries that retrieve relevant chunks and generate cited answers. The React client communicates with FastAPI; the backend also provides OTP/Google authentication and collection management. Some subsystem links are documented in the README or SOURCE INDEX rather than sampled implementation excerpts, so uncertain connections are omitted.

Main entry points: [`rag-frontend/src/App.jsx`](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/App.jsx) (App shell), [`src/rag/api/main.py`](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/main.py) (FastAPI app), [`src/rag/api/routes/auth.py`](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/auth.py) (Auth routes), [`src/rag/api/routes/query.py`](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/query.py) (Query route)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Web client"]
  node_app["App shell<br/>[App.jsx]"]
  node_authctx["Auth context<br/>[AuthContext.jsx]"]
  node_api_client["API client<br/>[api.js]"]
  node_collection_hook["Collections hook<br/>[useCollections.js]"]
  node_chat_hook["Chat hook<br/>[useChat.js]"]
  node_sidebar["Collection sidebar<br/>[Sidebar.jsx]"]
  node_upload["PDF upload panel<br/>[UploadPanel.jsx]"]
  node_chatview["Chat and citations<br/>[ChatWindow.jsx]"]
  node_queryinput["Query input<br/>[QueryInput.jsx]"]
end

subgraph group_api["API and identity"]
  node_fastapi["FastAPI app<br/>[main.py]"]
  node_auth_routes["Auth routes<br/>[auth.py]"]
  node_ingest_route["Ingest route<br/>[ingest.py]"]
  node_query_route["Query route<br/>[query.py]"]
  node_collection_routes["Collection routes<br/>[collections.py]"]
  node_auth_service["Auth and collections<br/>[service.py]"]
  node_otp_service["OTP service<br/>[otp_service.py]"]
  node_email_service["Email service<br/>[email_service.py]"]
  node_google_service["Google identity<br/>[google_service.py]"]
end

subgraph group_ingestion["Document ingestion"]
  node_ingest_pipeline["Ingestion pipeline<br/>[pipeline.py]"]
  node_pdf_parser["PDF parser<br/>[pdf_parser.py]"]
  node_chunker["Text chunker<br/>[chunker.py]"]
  node_embedding["Embedding service"]
end

subgraph group_qa["Retrieval and answers"]
  node_retriever["Retriever<br/>[retriever.py]"]
  node_generator["Response generator"]
end

subgraph group_state["Persistence and contracts"]
  node_vector_store[("Vector store<br/>[vector_store.py]")]
  node_auth_models["User and OTP models<br/>[models.py]"]
  node_auth_db[("Auth database<br/>[database.py]")]
  node_contracts["RAG data contracts<br/>[models.py]"]
end

node_user(("User"))

node_user -->|"uses"| node_app
node_app -->|"provides auth"| node_authctx
node_app -->|"renders"| node_sidebar
node_app -->|"renders"| node_chatview
node_app -->|"renders"| node_queryinput
node_app -->|"uses"| node_collection_hook
node_app -->|"uses"| node_chat_hook
node_sidebar -->|"renders"| node_upload
node_authctx -->|"calls auth API"| node_api_client
node_collection_hook -->|"calls collections API"| node_api_client
node_chat_hook -->|"calls query API"| node_api_client
node_api_client -->|"HTTP requests"| node_fastapi
node_fastapi -->|"mounts"| node_auth_routes
node_fastapi -->|"mounts"| node_ingest_route
node_fastapi -->|"mounts"| node_query_route
node_fastapi -->|"mounts"| node_collection_routes
node_auth_routes -->|"calls"| node_auth_service
node_auth_routes -->|"uses"| node_otp_service
node_auth_routes -->|"uses"| node_email_service
node_auth_routes -->|"uses"| node_google_service
node_auth_service -->|"reads and writes"| node_auth_models
node_auth_models -->|"declares ORM tables"| node_auth_db
node_collection_routes -.->|"manages collections"| node_auth_service
node_ingest_route -->|"resolves collection"| node_auth_service
node_ingest_route -->|"ingests PDF"| node_ingest_pipeline
node_ingest_pipeline -->|"parses pages"| node_pdf_parser
node_ingest_pipeline -->|"chunks text"| node_chunker
node_ingest_pipeline -.->|"embeds chunks"| node_embedding
node_ingest_pipeline -->|"stores chunks"| node_vector_store
node_query_route -->|"resolves collection"| node_auth_service
node_query_route -->|"retrieves chunks"| node_retriever
node_query_route -->|"generates answer"| node_generator
node_retriever -->|"searches"| node_vector_store
node_retriever -.->|"embeds query"| node_embedding
node_generator -->|"uses response models"| node_contracts
node_ingest_route -->|"returns ingest result"| node_contracts
node_query_route -->|"accepts and returns"| node_contracts

click node_app "https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/App.jsx"
click node_authctx "https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/context/AuthContext.jsx"
click node_api_client "https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/services/api.js"
click node_collection_hook "https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/hooks/useCollections.js"
click node_chat_hook "https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/hooks/useChat.js"
click node_sidebar "https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/components/Sidebar.jsx"
click node_upload "https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/components/UploadPanel.jsx"
click node_chatview "https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/components/ChatWindow.jsx"
click node_queryinput "https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/components/QueryInput.jsx"
click node_fastapi "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/main.py"
click node_auth_routes "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/auth.py"
click node_ingest_route "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/ingest.py"
click node_query_route "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/query.py"
click node_collection_routes "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/collections.py"
click node_auth_service "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/service.py"
click node_otp_service "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/otp_service.py"
click node_email_service "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/email_service.py"
click node_google_service "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/google_service.py"
click node_ingest_pipeline "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/ingestion/pipeline.py"
click node_pdf_parser "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/ingestion/pdf_parser.py"
click node_chunker "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/ingestion/chunker.py"
click node_embedding "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/embedding/embedding_service.py"
click node_retriever "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/retrieval/retriever.py"
click node_generator "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/generation/response_generator.py"
click node_vector_store "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/storage/vector_store.py"
click node_auth_models "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/models.py"
click node_auth_db "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/database.py"
click node_contracts "https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/models.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_authctx,node_api_client,node_collection_hook,node_chat_hook,node_sidebar,node_upload,node_chatview,node_queryinput,node_user toneBlue
class node_fastapi,node_auth_routes,node_ingest_route,node_query_route,node_collection_routes,node_auth_service,node_otp_service,node_email_service,node_google_service toneAmber
class node_ingest_pipeline,node_pdf_parser,node_chunker,node_embedding toneMint
class node_retriever,node_generator toneRose
class node_vector_store,node_auth_models,node_auth_db,node_contracts toneIndigo
```

## Components

### Web client

- [App shell](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/App.jsx): `rag-frontend/src/App.jsx`
- [Auth context](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/context/AuthContext.jsx): `rag-frontend/src/context/AuthContext.jsx`
- [API client](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/services/api.js): `rag-frontend/src/services/api.js`
- [Collections hook](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/hooks/useCollections.js): `rag-frontend/src/hooks/useCollections.js`
- [Chat hook](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/hooks/useChat.js): `rag-frontend/src/hooks/useChat.js`
- [Collection sidebar](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/components/Sidebar.jsx): `rag-frontend/src/components/Sidebar.jsx`
- [PDF upload panel](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/components/UploadPanel.jsx): `rag-frontend/src/components/UploadPanel.jsx`
- [Chat and citations](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/components/ChatWindow.jsx): `rag-frontend/src/components/ChatWindow.jsx`
- [Query input](https://github.com/mukeshram-07/multimodal-rag/blob/main/rag-frontend/src/components/QueryInput.jsx): `rag-frontend/src/components/QueryInput.jsx`

### API and identity

- [FastAPI app](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/main.py): `src/rag/api/main.py`
- [Auth routes](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/auth.py): `src/rag/api/routes/auth.py`
- [Ingest route](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/ingest.py): `src/rag/api/routes/ingest.py`
- [Query route](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/query.py): `src/rag/api/routes/query.py`
- [Collection routes](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/api/routes/collections.py): `src/rag/api/routes/collections.py`
- [Auth and collections](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/service.py): `src/rag/auth/service.py`
- [OTP service](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/otp_service.py): `src/rag/auth/otp_service.py`
- [Email service](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/email_service.py): `src/rag/auth/email_service.py`
- [Google identity](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/google_service.py): `src/rag/auth/google_service.py`

### Document ingestion

- [Ingestion pipeline](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/ingestion/pipeline.py): `src/rag/ingestion/pipeline.py`
- [PDF parser](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/ingestion/pdf_parser.py): `src/rag/ingestion/pdf_parser.py`
- [Text chunker](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/ingestion/chunker.py): `src/rag/ingestion/chunker.py`
- [Embedding service](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/embedding/embedding_service.py): `src/rag/embedding/embedding_service.py`

### Retrieval and answers

- [Retriever](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/retrieval/retriever.py): `src/rag/retrieval/retriever.py`
- [Response generator](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/generation/response_generator.py): `src/rag/generation/response_generator.py`

### Persistence and contracts

- [Vector store](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/storage/vector_store.py): `src/rag/storage/vector_store.py`
- [User and OTP models](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/models.py): `src/rag/auth/models.py`
- [Auth database](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/auth/database.py): `src/rag/auth/database.py`
- [RAG data contracts](https://github.com/mukeshram-07/multimodal-rag/blob/main/src/rag/models.py): `src/rag/models.py`

### Other components

- **User**

## Connections

- User → App shell: uses (evidence: `rag-frontend/src/App.jsx`)
- App shell → Auth context: provides auth (evidence: `rag-frontend/src/App.jsx`)
- App shell → Collection sidebar: renders (evidence: `rag-frontend/src/App.jsx`)
- App shell → Chat and citations: renders (evidence: `rag-frontend/src/App.jsx`)
- App shell → Query input: renders (evidence: `rag-frontend/src/App.jsx`)
- App shell → Collections hook: uses (evidence: `rag-frontend/src/App.jsx`)
- App shell → Chat hook: uses (evidence: `rag-frontend/src/App.jsx`)
- Collection sidebar → PDF upload panel: renders (evidence: `rag-frontend/src/components/Sidebar.jsx`)
- Auth context → API client: calls auth API (evidence: `rag-frontend/src/context/AuthContext.jsx`)
- Collections hook → API client: calls collections API (evidence: `rag-frontend/src/hooks/useCollections.js`)
- Chat hook → API client: calls query API (evidence: `rag-frontend/src/hooks/useChat.js`)
- API client → FastAPI app: HTTP requests (evidence: `rag-frontend/src/services/api.js`)
- FastAPI app → Auth routes: mounts (evidence: `src/rag/api/main.py`)
- FastAPI app → Ingest route: mounts (evidence: `src/rag/api/main.py`)
- FastAPI app → Query route: mounts (evidence: `src/rag/api/main.py`)
- FastAPI app → Collection routes: mounts (evidence: `src/rag/api/main.py`)
- Auth routes → Auth and collections: calls (evidence: `src/rag/api/routes/auth.py`)
- Auth routes → OTP service: uses (evidence: `src/rag/api/routes/auth.py`)
- Auth routes → Email service: uses (evidence: `src/rag/api/routes/auth.py`)
- Auth routes → Google identity: uses (evidence: `src/rag/api/routes/auth.py`)
- Auth and collections → User and OTP models: reads and writes (evidence: `src/rag/auth/service.py`)
- User and OTP models → Auth database: declares ORM tables (evidence: `src/rag/auth/models.py`)
- Collection routes → Auth and collections: manages collections
- Ingest route → Auth and collections: resolves collection (evidence: `src/rag/api/routes/ingest.py`)
- Ingest route → Ingestion pipeline: ingests PDF (evidence: `src/rag/api/routes/ingest.py`)
- Ingestion pipeline → PDF parser: parses pages (evidence: `src/rag/ingestion/pipeline.py`)
- Ingestion pipeline → Text chunker: chunks text (evidence: `src/rag/ingestion/pipeline.py`)
- Ingestion pipeline → Embedding service: embeds chunks
- Ingestion pipeline → Vector store: stores chunks (evidence: `src/rag/ingestion/pipeline.py`)
- Query route → Auth and collections: resolves collection (evidence: `src/rag/api/routes/query.py`)
- Query route → Retriever: retrieves chunks (evidence: `src/rag/api/routes/query.py`)
- Query route → Response generator: generates answer (evidence: `src/rag/api/routes/query.py`)
- Retriever → Vector store: searches (evidence: `src/rag/retrieval/retriever.py`)
- Retriever → Embedding service: embeds query
- Response generator → RAG data contracts: uses response models (evidence: `src/rag/generation/response_generator.py`)
- Ingest route → RAG data contracts: returns ingest result (evidence: `src/rag/api/routes/ingest.py`)
- Query route → RAG data contracts: accepts and returns (evidence: `src/rag/api/routes/query.py`)
