How wizlijun/claude_video_tranlater works
Project Type and Purpose
• This is a command-line, full-workflow video subtitle extraction, translation, and dubbing pipeline.
• It stitches together download, audio extraction, Whisper ASR, subtitle optimization, Claude AI translation, IndexTTS voice synthesis, and FFmpeg–based video composition into a single automated toolchain.
Main Components
A. Ingestion & Download
• Scripts: getvideo.sh, download_and_process.sh
• External: YouTube-dl or similar fetcher
• Output: raw video file (e.g., video.mp4)
B. Pre-Processing (process_video_part1.sh)
• Audio Extraction: FFmpeg → original_audio.wav
• ASR & Subtitle Extraction: get_srt_by_wisper.py (OpenAI Whisper) → step2_whisper.srt
• Subtitle Optimization: built-in in get_srt_by_wisper.py (merging, segmentation, clean-up)
C. Translation (translate_by_claude.sh)
• Input: step2_whisper.srt
• Service: Claude CLI → step3_translated.srt
D. Post-Processing (process_video_part2.sh / process_video_part2_plus.sh)
• Text-to-Speech: IndexTTS → step5_chinese_audio.wav
• Final Composition: FFmpeg overlays translated SRT + dubs audio → final video_translated.mp4
E. Offline Text-Normalization Library
• Path: models/wetext/… containing FSTs for tokenization, punctuation removal, ITN, etc.
• Used by get_srt_by_wisper.py for local text cleanup.
F. Auxiliary Tools
• Markdown Generation: genmarkdown_by_claude.sh → {video}.md
• Publishing: post_to_bilibili.sh, post_to_xiaohongshu.sh
Data Flow & Relationships
raw video → audio (.wav) → ASR SRT → optimized SRT → translated SRT → TTS audio → final video
• Each stage reads upstream artifacts from {video}_temp/ folder.
• Scripts detect existing files for checkpoint/ resume.
• External services (Whisper, Claude, IndexTTS) are called via Python or CLI wrappers.
Architectural Patterns & Principles
• Batch Processing Pipeline: clear sequential stages with checkpoint support.
• Modular CLI Components: each logical step (download, ASR, translate, TTS, compose) isolated in its own script.
• Pluggable External Services: Whisper, Claude AI, IndexTTS, FFmpeg.
• Local Model Library: FST-based normalization for offline subtitle optimization.
Technologies & Frameworks
• Bash / shell scripts for orchestration.
• Python 3.8+ for ASR (get_srt_by_wisper.py) and utilities (utils/).
• FFmpeg for audio/video processing.
• OpenAI Whisper (pip install openai-whisper).
• Claude CLI for translation.
• IndexTTS for Chinese voice synthesis.
• FST models under models/wetext/.
• Notes for critical design choices (e.g., “supports resume”, “modular CLI”).