# anirudh2627/youtube-rag architecture

> Architecture diagram of the GitHub repository anirudh2627/youtube-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anirudh2627/youtube-rag
- Repository: https://github.com/anirudh2627/youtube-rag

## Overview

YouTube RAG Assistant ingests video transcripts into timestamped chunks, indexes them for lexical and semantic search, and answers questions with reranked evidence and clickable citations. Users enter through the Streamlit UI or FastAPI; video processing and chat are coordinated by services, while retrieval and generation form the query path. Evaluation is a separate capability. Most pipeline links follow the README; source excerpts confirm service wiring and several implementation calls, but not every internal stage.

Main entry points: [`app/generation/rag_engine.py`](https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/rag_engine.py) (RAG engine), [`app/retrieval/pipeline.py`](https://github.com/anirudh2627/youtube-rag/blob/main/app/retrieval/pipeline.py) (Retrieval pipeline), [`app/services/container.py`](https://github.com/anirudh2627/youtube-rag/blob/main/app/services/container.py) (Service container), [`app/api/routes_chat.py`](https://github.com/anirudh2627/youtube-rag/blob/main/app/api/routes_chat.py) (FastAPI routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["Application"]
  node_streamlit["Streamlit UI<br/>[streamlit_app.py]"]
  node_api["FastAPI routes<br/>[routes_chat.py]"]
  node_video_routes["Video routes<br/>[routes_videos.py]"]
  node_container["Service container<br/>[container.py]"]
end

subgraph group_ingest["Ingestion and Indexing"]
  node_video_service["Video processing<br/>[video_service.py]"]
  node_transcript["Transcript providers"]
  node_cleaning["Transcript cleaning<br/>[cleaning.py]"]
  node_chunking["Timestamp chunking<br/>[chunking.py]"]
  node_embedder["Embeddings<br/>[st_embedder.py]"]
  node_vector_store[("Vector store<br/>[chroma_store.py]")]
end

subgraph group_query["Retrieval"]
  node_bm25["BM25 index<br/>[bm25.py]"]
  node_retrieval["Retrieval pipeline<br/>[pipeline.py]"]
  node_fusion["RRF fusion<br/>[hybrid.py]"]
  node_reranker["Cross-encoder reranker<br/>[reranker.py]"]
end

subgraph group_generation["Answering"]
  node_rewriter["Query rewriter<br/>[rewriter.py]"]
  node_memory["Conversation memory<br/>[conversation.py]"]
  node_rag_engine["RAG engine<br/>[rag_engine.py]"]
  node_prompts["Grounded prompts<br/>[prompts.py]"]
  node_llm["LLM provider<br/>[openai_compat.py]"]
  node_citations["Citation rendering<br/>[rag_engine.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_eval_runner["Evaluation runner<br/>[runner.py]"]
  node_eval_metrics["Evaluation metrics<br/>[metrics.py]"]
  node_eval_judges["LLM judges<br/>[judges.py]"]
end

node_user(("User"))

node_user -->|"asks through"| node_streamlit
node_streamlit -->|"calls API"| node_api
node_api -->|"gets services"| node_container
node_video_routes -->|"gets services"| node_container
node_video_routes -->|"processes video"| node_video_service
node_container -->|"constructs"| node_video_service
node_video_service -->|"fetches transcript"| node_transcript
node_transcript -->|"provides segments"| node_cleaning
node_cleaning -->|"cleans segments"| node_chunking
node_chunking -->|"creates vectors"| node_embedder
node_embedder -->|"indexes vectors"| node_vector_store
node_chunking -->|"indexes text"| node_bm25
node_api -->|"answers query"| node_rag_engine
node_container -->|"constructs"| node_retrieval
node_rag_engine -->|"rewrites query"| node_rewriter
node_rag_engine -->|"retrieves evidence"| node_retrieval
node_retrieval -->|"embeds query"| node_embedder
node_retrieval -->|"queries vectors"| node_vector_store
node_retrieval -->|"searches terms"| node_bm25
node_retrieval -->|"fuses rankings"| node_fusion
node_retrieval -->|"reranks candidates"| node_reranker
node_container -->|"constructs"| node_rag_engine
node_container -->|"constructs"| node_memory
node_rag_engine -->|"loads history"| node_memory
node_rag_engine -->|"builds context"| node_prompts
node_rag_engine -->|"generates answer"| node_llm
node_rag_engine -->|"renders citations"| node_citations
node_eval_runner -->|"computes metrics"| node_eval_metrics
node_eval_runner -->|"runs judges"| node_eval_judges

click node_streamlit "https://github.com/anirudh2627/youtube-rag/blob/main/frontend/streamlit_app.py"
click node_api "https://github.com/anirudh2627/youtube-rag/blob/main/app/api/routes_chat.py"
click node_video_routes "https://github.com/anirudh2627/youtube-rag/blob/main/app/api/routes_videos.py"
click node_container "https://github.com/anirudh2627/youtube-rag/blob/main/app/services/container.py"
click node_video_service "https://github.com/anirudh2627/youtube-rag/blob/main/app/services/video_service.py"
click node_transcript "https://github.com/anirudh2627/youtube-rag/blob/main/app/ingestion/youtube_transcript.py"
click node_cleaning "https://github.com/anirudh2627/youtube-rag/blob/main/app/ingestion/cleaning.py"
click node_chunking "https://github.com/anirudh2627/youtube-rag/blob/main/app/ingestion/chunking.py"
click node_embedder "https://github.com/anirudh2627/youtube-rag/blob/main/app/embeddings/st_embedder.py"
click node_vector_store "https://github.com/anirudh2627/youtube-rag/blob/main/app/vectorstore/chroma_store.py"
click node_bm25 "https://github.com/anirudh2627/youtube-rag/blob/main/app/retrieval/bm25.py"
click node_retrieval "https://github.com/anirudh2627/youtube-rag/blob/main/app/retrieval/pipeline.py"
click node_fusion "https://github.com/anirudh2627/youtube-rag/blob/main/app/retrieval/hybrid.py"
click node_reranker "https://github.com/anirudh2627/youtube-rag/blob/main/app/retrieval/reranker.py"
click node_rewriter "https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/rewriter.py"
click node_memory "https://github.com/anirudh2627/youtube-rag/blob/main/app/memory/conversation.py"
click node_rag_engine "https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/rag_engine.py"
click node_prompts "https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/prompts.py"
click node_llm "https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/openai_compat.py"
click node_citations "https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/rag_engine.py"
click node_eval_runner "https://github.com/anirudh2627/youtube-rag/blob/main/app/evaluation/runner.py"
click node_eval_metrics "https://github.com/anirudh2627/youtube-rag/blob/main/app/evaluation/metrics.py"
click node_eval_judges "https://github.com/anirudh2627/youtube-rag/blob/main/app/evaluation/judges.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_streamlit,node_api,node_video_routes,node_container,node_user toneBlue
class node_video_service,node_transcript,node_cleaning,node_chunking,node_embedder,node_vector_store toneAmber
class node_bm25,node_retrieval,node_fusion,node_reranker toneMint
class node_rewriter,node_memory,node_rag_engine,node_prompts,node_llm,node_citations toneRose
class node_eval_runner,node_eval_metrics,node_eval_judges toneIndigo
```

## Components

### Application

- [Streamlit UI](https://github.com/anirudh2627/youtube-rag/blob/main/frontend/streamlit_app.py): `frontend/streamlit_app.py`
- [FastAPI routes](https://github.com/anirudh2627/youtube-rag/blob/main/app/api/routes_chat.py): `app/api/routes_chat.py`
- [Video routes](https://github.com/anirudh2627/youtube-rag/blob/main/app/api/routes_videos.py): `app/api/routes_videos.py`
- [Service container](https://github.com/anirudh2627/youtube-rag/blob/main/app/services/container.py): `app/services/container.py`

### Ingestion and Indexing

- [Video processing](https://github.com/anirudh2627/youtube-rag/blob/main/app/services/video_service.py): `app/services/video_service.py`
- [Transcript providers](https://github.com/anirudh2627/youtube-rag/blob/main/app/ingestion/youtube_transcript.py): `app/ingestion/youtube_transcript.py`
- [Transcript cleaning](https://github.com/anirudh2627/youtube-rag/blob/main/app/ingestion/cleaning.py): `app/ingestion/cleaning.py`
- [Timestamp chunking](https://github.com/anirudh2627/youtube-rag/blob/main/app/ingestion/chunking.py): `app/ingestion/chunking.py`
- [Embeddings](https://github.com/anirudh2627/youtube-rag/blob/main/app/embeddings/st_embedder.py): `app/embeddings/st_embedder.py`
- [Vector store](https://github.com/anirudh2627/youtube-rag/blob/main/app/vectorstore/chroma_store.py): `app/vectorstore/chroma_store.py`

### Retrieval

- [BM25 index](https://github.com/anirudh2627/youtube-rag/blob/main/app/retrieval/bm25.py): `app/retrieval/bm25.py`
- [Retrieval pipeline](https://github.com/anirudh2627/youtube-rag/blob/main/app/retrieval/pipeline.py): `app/retrieval/pipeline.py`
- [RRF fusion](https://github.com/anirudh2627/youtube-rag/blob/main/app/retrieval/hybrid.py): `app/retrieval/hybrid.py`
- [Cross-encoder reranker](https://github.com/anirudh2627/youtube-rag/blob/main/app/retrieval/reranker.py): `app/retrieval/reranker.py`

### Answering

- [Query rewriter](https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/rewriter.py): `app/generation/rewriter.py`
- [Conversation memory](https://github.com/anirudh2627/youtube-rag/blob/main/app/memory/conversation.py): `app/memory/conversation.py`
- [RAG engine](https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/rag_engine.py): `app/generation/rag_engine.py`
- [Grounded prompts](https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/prompts.py): `app/generation/prompts.py`
- [LLM provider](https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/openai_compat.py): `app/generation/openai_compat.py`
- [Citation rendering](https://github.com/anirudh2627/youtube-rag/blob/main/app/generation/rag_engine.py): `app/generation/rag_engine.py`

### Evaluation

- [Evaluation runner](https://github.com/anirudh2627/youtube-rag/blob/main/app/evaluation/runner.py): `app/evaluation/runner.py`
- [Evaluation metrics](https://github.com/anirudh2627/youtube-rag/blob/main/app/evaluation/metrics.py): `app/evaluation/metrics.py`
- [LLM judges](https://github.com/anirudh2627/youtube-rag/blob/main/app/evaluation/judges.py): `app/evaluation/judges.py`

### Other components

- **User**

## Connections

- User → Streamlit UI: asks through (evidence: `README.md`)
- Streamlit UI → FastAPI routes: calls API (evidence: `README.md`)
- FastAPI routes → Service container: gets services (evidence: `app/api/routes_chat.py`)
- Video routes → Service container: gets services (evidence: `app/api/routes_videos.py`)
- Video routes → Video processing: processes video (evidence: `app/api/routes_videos.py`)
- Service container → Video processing: constructs (evidence: `app/services/container.py`)
- Video processing → Transcript providers: fetches transcript (evidence: `app/services/video_service.py`)
- Transcript providers → Transcript cleaning: provides segments (evidence: `README.md`)
- Transcript cleaning → Timestamp chunking: cleans segments (evidence: `README.md`)
- Timestamp chunking → Embeddings: creates vectors (evidence: `README.md`)
- Embeddings → Vector store: indexes vectors (evidence: `README.md`)
- Timestamp chunking → BM25 index: indexes text (evidence: `README.md`)
- FastAPI routes → RAG engine: answers query (evidence: `README.md`)
- Service container → Retrieval pipeline: constructs (evidence: `app/services/container.py`)
- RAG engine → Query rewriter: rewrites query (evidence: `app/generation/rag_engine.py`)
- RAG engine → Retrieval pipeline: retrieves evidence (evidence: `app/generation/rag_engine.py`)
- Retrieval pipeline → Embeddings: embeds query (evidence: `app/retrieval/pipeline.py`)
- Retrieval pipeline → Vector store: queries vectors (evidence: `app/retrieval/pipeline.py`)
- Retrieval pipeline → BM25 index: searches terms (evidence: `app/retrieval/pipeline.py`)
- Retrieval pipeline → RRF fusion: fuses rankings (evidence: `app/retrieval/pipeline.py`)
- Retrieval pipeline → Cross-encoder reranker: reranks candidates (evidence: `app/retrieval/pipeline.py`)
- Service container → RAG engine: constructs (evidence: `app/services/container.py`)
- Service container → Conversation memory: constructs (evidence: `app/services/container.py`)
- RAG engine → Conversation memory: loads history (evidence: `app/generation/rag_engine.py`)
- RAG engine → Grounded prompts: builds context (evidence: `app/generation/rag_engine.py`)
- RAG engine → LLM provider: generates answer (evidence: `README.md`)
- RAG engine → Citation rendering: renders citations (evidence: `app/generation/rag_engine.py`)
- Evaluation runner → Evaluation metrics: computes metrics (evidence: `app/evaluation/runner.py`)
- Evaluation runner → LLM judges: runs judges (evidence: `app/evaluation/runner.py`)
