How adamixa/wathiq_video_analysis works
Project type and purpose
• This is a video-analysis application (a modular “full-stack” capstone tool) with:
• A user-facing Streamlit front-end for upload, parameter selection, and report display
• A back-end pipeline of specialized Python modules that perform video and audio analysis
• No persistent database; intermediate data are stored on the file system (CSVs, WAVs, model weights)
• An external API (OpenAI) for text summarization and recommendations
File structure
• Top-level directories map one-to-one with analysis domains:
• AudioProcessing/ (notebooks & extracted_audio.wav)
• BodyLanguage/ (notebooks, coords.csv, model weights)
• EyeContact/, FaceEmotion/, HandMovement/ likewise
• PythonScripts/ contains the production code:
• streamlit_app.py (UI controller)
• AudioProcessor.py, BodyLanguageRecognizer.py, EmotionDetector.py, EyeContact.py, HandMovementDetector.py (processing modules)
• ObjectTracker.py (shared utility)
• main.py (CLI orchestration)
• Config and dependency files: requirements.txt, .streamlit/config.toml
• Models & data stores on disk: .pkl, .pt, .wav, .csv, Haarcascade XMLs
Extract key insights from README
• Core features: body language, hand movement, eye contact, emotion, stutter detection & transcription, OpenAI-powered summarization
• Usage pattern: user launches Streamlit → selects or uploads video → “Analyze” triggers all modules → report visualized in app
a. Main components
1. User & Streamlit UI (Front-end)
2. Orchestrator / Controller (streamlit_app.py or main.py)
3. Video Preprocessor (ffmpeg wrapper)
4. Visual Analysis Modules (each runs on video frames)
– BodyLanguageRecognizer
– HandMovementDetector
– EyeContactAnalyzer
– EmotionDetector (DeepFace + YOLO)
– ObjectTracker (shared)
5. Audio Analysis Module (AudioProcessor)
– Stutter detector
– Speech-to-text (whisper or similar)
– Summarization (OpenAI API)
6. Aggregator & Report Generator
7. File System (raw videos, intermediate CSV/WAV, model weights)
8. External Service: OpenAI API
b. Relationships & flows
• User → (upload/select video) → Streamlit UI
• UI → Orchestrator invokes Video Preprocessor → frame stream + extracted audio
• Frame stream → forked in parallel to each Visual Analysis Module → outputs CSV/metrics
• Extracted audio → AudioProcessor → stutter report + transcription
• Transcription → OpenAI API → text summary & recommendations
• All module outputs → Aggregator → final data structure
• Aggregator → Streamlit UI → display charts, videos, text report
c. Architectural patterns & design principles
• Modular pipeline: each analysis domain is a separate component in a data-flow graph
• Single-process “monolith” with clear separation of concerns (no microservices)
• Shared utility layer (ObjectTracker, ModelLoader) for DRY code
• Layered: UI layer → orchestration layer → processing layer → external layer
d. Key technologies & frameworks
• Python 3.x, Streamlit
• MediaPipe, OpenCV, Haarcascades
• Ultralytics YOLO (yolov8n.pt)
• DeepFace (emotion)
• ffmpeg (audio extraction)
• OpenAI API (summaries & recommendations)
Diagramming recommendations
[Streamlit UI], [Orchestrator], [BodyLanguageRecognizer], [AudioProcessor], [Aggregator]
(raw_videos), (extracted_audio.wav), (model_weights/*.pkl/.pt), (coords.csv)
• Use a cloud symbol or rounded rectangle for external service:
{OpenAI API}
User → UI → Orchestrator → {Video Preprocessor} → Visual Modules → Aggregator → UI
• Blue = UI / Orchestration
• Green = Processing Modules
• Yellow = Data Storage
• Purple = External Services
Detail and clarity
• Parallel flows for the five visual modules
• Final step of feeding the aggregated results back to the UI for rendering