# vipinkumarv0107/ai-video-assistant architecture

> Architecture diagram of the GitHub repository vipinkumarv0107/ai-video-assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vipinkumarv0107/ai-video-assistant
- Repository: https://github.com/vipinkumarv0107/ai-video-assistant

## Overview

AI Video Assistant accepts an uploaded media file or URL, converts it to chunked audio, transcribes it, and returns a transcript with a title, summary, action items, decisions, and open questions. Users can then ask transcript-grounded questions in the analysis session. The backend coordinates these stages; the browser interface displays results and submits questions. The excerpts show the RAG retrieval setup, but also contain an undefined `llm` reference, so successful question answering is not assured. Some frontend analysis-request wiring is omitted from the excerpt.

Main entry points: [`main.py`](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/main.py) (Analysis pipeline), [`core/rag_engine.py`](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/rag_engine.py) (RAG question answering), [`core/transcriber.py`](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/transcriber.py) (Transcription), [`core/summarize.py`](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/summarize.py) (Transcript analysis)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User interface"]
  node_browser["Web interface<br/>[script.js]"]
end

subgraph group_api["API orchestration"]
  node_api_app["FastAPI endpoints<br/>[main.py]"]
  node_pipeline["Analysis pipeline<br/>[main.py]"]
  node_analysis_session[("In-memory analysis sessions<br/>[main.py]")]
end

subgraph group_media["Media and transcription"]
  node_audio_processing["Media processing<br/>[audio_processor.py]"]
  node_audio_chunks["Audio chunks<br/>[audio_processor.py]"]
  node_transcription["Transcription<br/>[transcriber.py]"]
  node_transcript["Transcript<br/>[transcriber.py]"]
end

subgraph group_intelligence["Transcript intelligence"]
  node_analysis["Transcript analysis<br/>[summarize.py]"]
  node_analysis_result["Analysis results<br/>[summarize.py]"]
  node_rag_engine["RAG question answering<br/>[rag_engine.py]"]
  node_vector_store[("Transcript vector store<br/>[rag_engine.py]")]
  node_extractor["Legacy extractors<br/>[extractor.py]"]
end

node_user(("User"))
node_media_source["Uploaded media or URL"]
node_online_media["Online media downloader"]
node_whisper["Whisper"]
node_sarvam["Sarvam speech API"]
node_mistral_analysis["Mistral analysis model"]
node_mistral_rag["Mistral answer model"]

node_user -->|"uses"| node_browser
node_browser -->|"checks health"| node_api_app
node_browser -->|"submits questions"| node_api_app
node_browser -.->|"requests analysis"| node_api_app
node_api_app -->|"runs analysis"| node_pipeline
node_pipeline -->|"processes source"| node_audio_processing
node_media_source -->|"provides media"| node_audio_processing
node_audio_processing -->|"downloads URL media"| node_online_media
node_audio_processing -->|"creates chunks"| node_audio_chunks
node_pipeline -->|"transcribes chunks"| node_transcription
node_audio_chunks -->|"supplies audio"| node_transcription
node_transcription -->|"transcribes"| node_whisper
node_transcription -->|"transcribes Hinglish"| node_sarvam
node_transcription -->|"assembles text"| node_transcript
node_pipeline -->|"analyzes transcript"| node_analysis
node_transcript -->|"provides text"| node_analysis
node_analysis -->|"invokes model"| node_mistral_analysis
node_analysis -->|"parses sections"| node_analysis_result
node_pipeline -->|"builds question chain"| node_rag_engine
node_transcript -->|"provides text"| node_rag_engine
node_rag_engine -->|"stores chunks"| node_vector_store
node_rag_engine -->|"retrieves chunks"| node_vector_store
node_rag_engine -->|"invokes model"| node_mistral_rag
node_pipeline -->|"stores chain"| node_analysis_session
node_api_app -->|"looks up chain"| node_analysis_session
node_api_app -->|"asks question"| node_rag_engine
node_extractor -->|"delegates analysis"| node_analysis

click node_browser "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/script.js"
click node_api_app "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/main.py"
click node_pipeline "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/main.py"
click node_audio_processing "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/utils/audio_processor.py"
click node_audio_chunks "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/utils/audio_processor.py"
click node_transcription "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/transcriber.py"
click node_transcript "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/transcriber.py"
click node_analysis "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/summarize.py"
click node_analysis_result "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/summarize.py"
click node_rag_engine "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/rag_engine.py"
click node_vector_store "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/rag_engine.py"
click node_analysis_session "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/main.py"
click node_extractor "https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/extractor.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser,node_user toneBlue
class node_api_app,node_pipeline,node_analysis_session toneAmber
class node_audio_processing,node_audio_chunks,node_transcription,node_transcript,node_sarvam toneMint
class node_analysis,node_analysis_result,node_rag_engine,node_vector_store,node_extractor toneRose
class node_media_source,node_online_media,node_whisper,node_mistral_analysis,node_mistral_rag toneIndigo
```

## Components

### User interface

- [Web interface](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/script.js): `script.js`

### API orchestration

- [FastAPI endpoints](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/main.py): `main.py`
- [Analysis pipeline](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/main.py): `main.py`
- [In-memory analysis sessions](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/main.py): `main.py`

### Media and transcription

- [Media processing](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/utils/audio_processor.py): `utils/audio_processor.py`
- [Audio chunks](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/utils/audio_processor.py): `utils/audio_processor.py`
- [Transcription](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/transcriber.py): `core/transcriber.py`
- [Transcript](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/transcriber.py): `core/transcriber.py`

### Transcript intelligence

- [Transcript analysis](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/summarize.py): `core/summarize.py`
- [Analysis results](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/summarize.py): `core/summarize.py`
- [RAG question answering](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/rag_engine.py): `core/rag_engine.py`
- [Transcript vector store](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/rag_engine.py): `core/rag_engine.py`
- [Legacy extractors](https://github.com/vipinkumarv0107/ai-video-assistant/blob/main/core/extractor.py): `core/extractor.py`

### Other components

- **User**
- **Uploaded media or URL**
- **Online media downloader**
- **Whisper**
- **Sarvam speech API**
- **Mistral analysis model**
- **Mistral answer model**

## Connections

- User → Web interface: uses (evidence: `script.js`)
- Web interface → FastAPI endpoints: checks health (evidence: `script.js`)
- Web interface → FastAPI endpoints: submits questions (evidence: `script.js`)
- Web interface → FastAPI endpoints: requests analysis
- FastAPI endpoints → Analysis pipeline: runs analysis (evidence: `main.py`)
- Analysis pipeline → Media processing: processes source (evidence: `main.py`)
- Uploaded media or URL → Media processing: provides media
- Media processing → Online media downloader: downloads URL media (evidence: `utils/audio_processor.py`)
- Media processing → Audio chunks: creates chunks (evidence: `utils/audio_processor.py`)
- Analysis pipeline → Transcription: transcribes chunks (evidence: `main.py`)
- Audio chunks → Transcription: supplies audio (evidence: `main.py`)
- Transcription → Whisper: transcribes (evidence: `core/transcriber.py`)
- Transcription → Sarvam speech API: transcribes Hinglish (evidence: `core/transcriber.py`)
- Transcription → Transcript: assembles text (evidence: `core/transcriber.py`)
- Analysis pipeline → Transcript analysis: analyzes transcript (evidence: `main.py`)
- Transcript → Transcript analysis: provides text (evidence: `main.py`)
- Transcript analysis → Mistral analysis model: invokes model (evidence: `core/summarize.py`)
- Transcript analysis → Analysis results: parses sections (evidence: `core/summarize.py`)
- Analysis pipeline → RAG question answering: builds question chain (evidence: `main.py`)
- Transcript → RAG question answering: provides text (evidence: `main.py`)
- RAG question answering → Transcript vector store: stores chunks (evidence: `core/rag_engine.py`)
- RAG question answering → Transcript vector store: retrieves chunks (evidence: `core/rag_engine.py`)
- RAG question answering → Mistral answer model: invokes model (evidence: `core/rag_engine.py`)
- Analysis pipeline → In-memory analysis sessions: stores chain (evidence: `main.py`)
- FastAPI endpoints → In-memory analysis sessions: looks up chain (evidence: `main.py`)
- FastAPI endpoints → RAG question answering: asks question (evidence: `main.py`)
- Legacy extractors → Transcript analysis: delegates analysis (evidence: `core/extractor.py`)
