How vipinkumarv0107/ai-video-assistant works
AI Video Assistant accepts an uploaded media file or URL, converts it to chunked audio, transcribes it, and returns a transcript with a title, summary, action items, decisions, and open questions. Users can then ask transcript-grounded questions in the analysis session. The backend coordinates these stages; the browser interface displays results and submits questions. The excerpts show the RAG retrieval setup, but also contain an undefined llm reference, so successful question answering is not assured. Some frontend analysis-request wiring is omitted from the excerpt.
Main entry points: main.py Analysis pipeline · rag_engine.py RAG question answering · transcriber.py Transcription · summarize.py Transcript analysis
Components 20
User interface
- Web interface
script.js
API orchestration
- FastAPI endpoints
main.py - Analysis pipeline
main.py - In-memory analysis sessions
main.py
Media and transcription
- Media processing
utils/audio_processor.py - Audio chunks
utils/audio_processor.py - Transcription
core/transcriber.py - Transcript
core/transcriber.py
Transcript intelligence
- Transcript analysis
core/summarize.py - Analysis results
core/summarize.py - RAG question answering
core/rag_engine.py - Transcript vector store
core/rag_engine.py - Legacy extractors
core/extractor.py
Other components
- User
- Uploaded media or URL
- Online media downloader
- Whisper
- Sarvam speech API
- Mistral analysis model
- Mistral answer model
Connections 27
- User → Web interface: uses
script.js - Web interface → FastAPI endpoints: checks health
script.js - Web interface → FastAPI endpoints: submits questions
script.js - Web interface → FastAPI endpoints: requests analysis
- FastAPI endpoints → Analysis pipeline: runs analysis
main.py - Analysis pipeline → Media processing: processes source
main.py - Uploaded media or URL → Media processing: provides media
- Media processing → Online media downloader: downloads URL media
utils/audio_processor.py - Media processing → Audio chunks: creates chunks
utils/audio_processor.py - Analysis pipeline → Transcription: transcribes chunks
main.py - Audio chunks → Transcription: supplies audio
main.py - Transcription → Whisper: transcribes
core/transcriber.py - Transcription → Sarvam speech API: transcribes Hinglish
core/transcriber.py - Transcription → Transcript: assembles text
core/transcriber.py - Analysis pipeline → Transcript analysis: analyzes transcript
main.py - Transcript → Transcript analysis: provides text
main.py - Transcript analysis → Mistral analysis model: invokes model
core/summarize.py - Transcript analysis → Analysis results: parses sections
core/summarize.py - Analysis pipeline → RAG question answering: builds question chain
main.py - Transcript → RAG question answering: provides text
main.py - RAG question answering → Transcript vector store: stores chunks
core/rag_engine.py - RAG question answering → Transcript vector store: retrieves chunks
core/rag_engine.py - RAG question answering → Mistral answer model: invokes model
core/rag_engine.py - Analysis pipeline → In-memory analysis sessions: stores chain
main.py - FastAPI endpoints → In-memory analysis sessions: looks up chain
main.py - FastAPI endpoints → RAG question answering: asks question
main.py - Legacy extractors → Transcript analysis: delegates analysis
core/extractor.py