How mharis-code/videoanalyzer works
1. What this repo is
- This is a Claude Code skill, not a standalone app or service.
- Its job is to take a video URL or local file, gather visual and spoken evidence, and produce a structured markdown analysis report.
2. Top-level architecture
- The repo has three main layers:
- Skill/config layer: SKILL.md, CLAUDE.md
- Installer/setup layer: install.py
- Pipeline layer: scripts/
- The pipeline is executed on demand inside Claude Code when the user invokes /video-analyze.
3. Execution flow
- The core flow is: input URL/path → download media/subtitles → extract frames → build transcript → optionally transcribe audio → generate report.
- watch.py is the orchestration entry point that ties these stages together and prints/saves the final result.
4. Media acquisition boundary
- scripts/download.py is the ingestion boundary.
- It uses yt-dlp to handle any supported platform, which is the repo’s main external dependency for cross-platform video retrieval.
- It also fetches metadata and subtitle files, preferring native captions when available.
5. Visual analysis subsystem
- scripts/frames.py turns the video into a bounded set of JPEG frames.
- It uses ffprobe to inspect duration and ffmpeg to extract frames.
- The important architectural detail is that frame extraction is budget-based, not fixed-rate: long videos are sampled sparsely, while short or focused ranges get denser coverage to control token cost.
6. Transcript processing subsystem
- scripts/transcribe.py parses WebVTT subtitles into timestamped segments.
- It includes normalization for YouTube-style rolling captions by deduplicating repeated lines and merging their time ranges.
- This gives the report a clean temporal text track when captions exist.
7. Fallback speech-to-text path
- scripts/whisper.py is the fallback when no usable captions are present.
- It extracts audio with ffmpeg, converts to a compact mono 16 kHz MP3, and sends it to Groq Whisper or OpenAI Whisper through API calls.
- The repo treats this as optional: if no API key is configured, analysis still works but may be frames-only.
8. Orchestration and report generation
- scripts/watch.py is the main coordinator.
- It supports both remote URLs and local files, plus optional --start / --end range zooming for long videos.
- Its output is a markdown report that combines frame observations and transcript context into the final analysis structure.
9. Installation and environment management
- install.py is the one-shot installer that prepares the environment for Claude Code usage.
- It checks Python version, installs or guides installation of yt-dlp and ffmpeg/ffprobe, copies the skill into Claude’s global skills directory, patches script paths in SKILL.md, and scaffolds a config file for optional Whisper keys.
10. Skill integration boundary
- SKILL.md is the prompt Claude Code reads when the skill is invoked.
- CLAUDE.md acts as repo-guidance for opening the repository directly in Claude Code, so the assistant can run the installer and validate readiness.
- This means the repo is designed around Claude-driven orchestration, with the scripts doing the actual media work.
11. Configuration and secrets
- Optional API credentials live in ~/.config/watch/.env.
- The only material secret inputs are GROQ_API_KEY or OPENAI_API_KEY, used solely for the Whisper fallback path.
12. Output boundary
- The pipeline writes analysis files to a dated markdown path under a content-creation drafts folder.
- The final output is therefore both machine-readable in structure and portable as a markdown artifact, which makes the repo suitable for content strategy workflows rather than live serving.
Main entry points: watch.py watch.py · install.py install.py · frames.py frames.py · whisper.py whisper.py
Components 15
Skill layer
Claude Code entrypoints and usage guidance.
Installer layer
One-shot setup and environment preparation.
- install.py · Prepares Claude Code setup, dependencies, and local configuration.
install.py - .env config · Optional local credentials for Whisper fallback.
Pipeline layer
Media analysis workflow under scripts/.
- scripts/setup.py · Shared environment and path setup used by pipeline scripts.
scripts/setup.py - watch.py · Main coordinator that runs the analysis pipeline and writes the report.
scripts/watch.py - download.py · Fetches video media, metadata, and subtitles via yt-dlp.
scripts/download.py - frames.py · Samples video frames with a budget-based ffprobe/ffmpeg strategy.
scripts/frames.py - transcribe.py · Converts WebVTT captions into cleaned timestamped transcript segments.
scripts/transcribe.py - whisper.py · Extracts audio and sends it to Whisper APIs when captions are unavailable.
scripts/whisper.py - yt-dlp · Cross-platform media retrieval backend for downloads and metadata.
- ffmpeg · Used for frame extraction and audio conversion.
- ffprobe · Inspects duration and stream properties for frame budgeting.
- Whisper API · Groq Whisper or OpenAI Whisper used for optional transcription.
- Markdown report · Final dated markdown artifact combining visual and transcript evidence.
Connections 21
- CLAUDE.md → SKILL.md: guides use · Repo guidance points to the skill prompt used by Claude Code.
- install.py → SKILL.md: installs · Installer copies the skill into Claude's global skills directory.
- install.py → .env config: scaffolds · Installer creates or prepares the optional secrets/config file.
- install.py → yt-dlp: checks · Installer verifies yt-dlp availability or guides installation.
- install.py → ffmpeg: checks · Installer verifies ffmpeg availability or guides installation.
- install.py → ffprobe: checks · Installer verifies ffprobe availability or guides installation.
- watch.py → scripts/setup.py: uses · Orchestrator relies on shared setup helpers.
- watch.py → download.py: downloads · Main flow starts by acquiring media and subtitles.
- watch.py → frames.py: extracts frames · Main flow creates a bounded frame set for analysis.
- watch.py → transcribe.py: builds transcript · Main flow consumes captions when available.
- watch.py → whisper.py: falls back · Main flow can invoke speech transcription when captions are missing.
- watch.py → Markdown report: writes · Orchestrator emits the final markdown analysis report.
- download.py → yt-dlp: depends on · Media ingestion is built on yt-dlp.
- frames.py → ffprobe: probes with · Frame budgeting depends on video duration inspection.
- frames.py → ffmpeg: extracts with · Frames are rendered from the source video via ffmpeg.
- transcribe.py → Markdown report: feeds · Cleaned subtitle segments contribute to the final report.
- whisper.py → ffmpeg: converts with · Audio is extracted and compacted with ffmpeg before transcription.
- whisper.py → Whisper API: calls · Fallback transcription is sent to a Whisper provider.
- .env config → whisper.py: configures · API keys in the env file enable the whisper fallback path.
- download.py → Markdown report: supports · Downloaded metadata and subtitles support the final analysis.
- frames.py → Markdown report: supports · Frame observations are incorporated into the report.