How mukeshram-07/multimodal-rag works
Multimodal RAG is an authenticated PDF knowledge-base and question-answering application. Users sign in, choose a collection, upload PDFs for parsing, chunking, embedding, and vector storage, then submit queries that retrieve relevant chunks and generate cited answers. The React client communicates with FastAPI; the backend also provides OTP/Google authentication and collection management. Some subsystem links are documented in the README or SOURCE INDEX rather than sampled implementation excerpts, so uncertain connections are omitted.
Main entry points: App.jsx App shell · main.py FastAPI app · auth.py Auth routes · query.py Query route
Components 29
Web client
- App shell
rag-frontend/src/App.jsx - Auth context
rag-frontend/src/context/AuthContext.jsx - API client
rag-frontend/src/services/api.js - Collections hook
rag-frontend/src/hooks/useCollections.js - Chat hook
rag-frontend/src/hooks/useChat.js - Collection sidebar
rag-frontend/src/components/Sidebar.jsx - PDF upload panel
rag-frontend/src/components/UploadPanel.jsx - Chat and citations
rag-frontend/src/components/ChatWindow.jsx - Query input
rag-frontend/src/components/QueryInput.jsx
API and identity
- FastAPI app
src/rag/api/main.py - Auth routes
src/rag/api/routes/auth.py - Ingest route
src/rag/api/routes/ingest.py - Query route
src/rag/api/routes/query.py - Collection routes
src/rag/api/routes/collections.py - Auth and collections
src/rag/auth/service.py - OTP service
src/rag/auth/otp_service.py - Email service
src/rag/auth/email_service.py - Google identity
src/rag/auth/google_service.py
Document ingestion
- Ingestion pipeline
src/rag/ingestion/pipeline.py - PDF parser
src/rag/ingestion/pdf_parser.py - Text chunker
src/rag/ingestion/chunker.py - Embedding service
src/rag/embedding/embedding_service.py
Retrieval and answers
- Retriever
src/rag/retrieval/retriever.py - Response generator
src/rag/generation/response_generator.py
Persistence and contracts
- Vector store
src/rag/storage/vector_store.py - User and OTP models
src/rag/auth/models.py - Auth database
src/rag/auth/database.py - RAG data contracts
src/rag/models.py
Other components
- User
Connections 37
- User → App shell: uses
rag-frontend/src/App.jsx - App shell → Auth context: provides auth
rag-frontend/src/App.jsx - App shell → Collection sidebar: renders
rag-frontend/src/App.jsx - App shell → Chat and citations: renders
rag-frontend/src/App.jsx - App shell → Query input: renders
rag-frontend/src/App.jsx - App shell → Collections hook: uses
rag-frontend/src/App.jsx - App shell → Chat hook: uses
rag-frontend/src/App.jsx - Collection sidebar → PDF upload panel: renders
rag-frontend/src/components/Sidebar.jsx - Auth context → API client: calls auth API
rag-frontend/src/context/AuthContext.jsx - Collections hook → API client: calls collections API
rag-frontend/src/hooks/useCollections.js - Chat hook → API client: calls query API
rag-frontend/src/hooks/useChat.js - API client → FastAPI app: HTTP requests
rag-frontend/src/services/api.js - FastAPI app → Auth routes: mounts
src/rag/api/main.py - FastAPI app → Ingest route: mounts
src/rag/api/main.py - FastAPI app → Query route: mounts
src/rag/api/main.py - FastAPI app → Collection routes: mounts
src/rag/api/main.py - Auth routes → Auth and collections: calls
src/rag/api/routes/auth.py - Auth routes → OTP service: uses
src/rag/api/routes/auth.py - Auth routes → Email service: uses
src/rag/api/routes/auth.py - Auth routes → Google identity: uses
src/rag/api/routes/auth.py - Auth and collections → User and OTP models: reads and writes
src/rag/auth/service.py - User and OTP models → Auth database: declares ORM tables
src/rag/auth/models.py - Collection routes → Auth and collections: manages collections
- Ingest route → Auth and collections: resolves collection
src/rag/api/routes/ingest.py - Ingest route → Ingestion pipeline: ingests PDF
src/rag/api/routes/ingest.py - Ingestion pipeline → PDF parser: parses pages
src/rag/ingestion/pipeline.py - Ingestion pipeline → Text chunker: chunks text
src/rag/ingestion/pipeline.py - Ingestion pipeline → Embedding service: embeds chunks
- Ingestion pipeline → Vector store: stores chunks
src/rag/ingestion/pipeline.py - Query route → Auth and collections: resolves collection
src/rag/api/routes/query.py - Query route → Retriever: retrieves chunks
src/rag/api/routes/query.py - Query route → Response generator: generates answer
src/rag/api/routes/query.py - Retriever → Vector store: searches
src/rag/retrieval/retriever.py - Retriever → Embedding service: embeds query
- Response generator → RAG data contracts: uses response models
src/rag/generation/response_generator.py - Ingest route → RAG data contracts: returns ingest result
src/rag/api/routes/ingest.py - Query route → RAG data contracts: accepts and returns
src/rag/api/routes/query.py