How talsperre/lecturesummarizer works
Project Type and Purpose
• This is a full-stack web application built with Django (Python) that automatically summarizes lecture videos using AI (speech recognition), computer vision (shot detection), and NLP (summarization & tag generation). Users upload videos via a web UI; the system splits videos into shots, extracts audio, converts speech to text, summarizes each clip, generates tags, indexes results in a database, and presents summaries and search results back to the user.
File Structure Analysis
• Top-level:
• videoSummarizer/ ← Django project root
• requirements.txt ← Python dependencies
• documents/ ← design docs and presentation
• Under videoSummarizer/:
• manage.py ← Django CLI entry point
• videoSummarizer/ ← Django settings, URL routing, WSGI/ASGI
• summarize/ ← Core Django app: models, views, templates, static assets, forms, URLs
• Modular AI/processing code (each with Interface + implementation + tests):
• shot_detection/
• audio_generation/
• speech_recognition/
• summarization/
• tag_generation/
• Media storage under videoSummarizer/media/: videos, audio clips, metadata CSVs.
README Insights
• MVT (Model-View-Template) design pattern
• SOLID, clean code, interfaces abstract external libraries (PySceneDetect, FFMPEG, Google Speech API, YAKE)
• MySQL backend (VideoSummarizer database)
• Unit tests for each module
Main Components
a. User Interface
• Web browser → Django “summarize” app (views + templates + static)
b. Application Server
• Django core (manage.py, settings, URLs, WSGI/ASGI)
• Shot Detection
• Audio Generation
• Speech Recognition
• Summarization
• Tag Generation
d. Data Storage
• File system (media/videos, media/audio, metadata CSVs)
• MySQL database (video, clip, transcript, summary, tags tables)
e. External Services & Libraries
• PySceneDetect
• FFMPEG
• Google Speech Recognition API
• YAKE (keyword extraction)
Relationships & Data Flow
1. User uploads video → saved to media/videos
2. Shot Detection reads raw video → outputs video clips + split_times CSV → saved under media/
3. Audio Generation reads each clip → uses FFMPEG → writes audio files under media/audio
4. Speech Recognition reads audio files → calls Google API → writes transcripts
5. Summarization & Tag Generation read transcripts → generate summaries and keywords → stored in DB
6. Django Views query DB + media files → render HTML pages for search, summaries, playback
Architectural Patterns & Principles
• MVT (Django) for web layer
• Interface + Implementation modules for each AI step (SOLID, Open/Closed)
• Single Responsibility per processing module
• Clear separation: Web layer vs. Processing pipeline vs. Storage