naufalrizqullah/opensource-clipping

1. The repository has two main surfaces: a batch video-processing pipeline driven from the CLI, and a web control plane built from the FastAPI backend plus React dashboard. The CLI path does the heavy media work; the web path mainly manages jobs, settings, and file access.

2. The core pipeline lives in clipping/ and is orchestrated by main.py → clipping.runner. config.py parses the large CLI surface, runner.py coordinates the stages, and engine.py handles the “analysis” side of the workflow. This is the backbone for turning one source URL into one or more finished clips.

3. Source ingestion and analysis form the first data plane. The pipeline can pull from YouTube, TikTok, Instagram, or Google Drive, then transcribe with Faster-Whisper, optionally reuse built-in subtitles, and send transcript/context to an LLM provider. engine.py, metadata.py, and hook_manager.py are the main pieces involved in extracting moments, normalizing metadata, and selecting hook material.

4. AI selection is separated from rendering. Gemini is the default analysis provider, with NVIDIA NIM as an alternative. The AI layer decides clip ranks, titles, descriptions, tags, hook candidates, and segment trimming decisions; the render layer consumes those decisions rather than re-analyzing content.

5. clipping/studio/ is the media-rendering subsystem. It contains the FFmpeg-driven composition logic for subtitles, typography, transitions, effects, B-roll, BGM, thumbnail generation, and layout modes. The modular files here show that rendering is composed from specialized renderers rather than one monolith.

6. Face tracking and podcast layouts are a major rendering boundary. face_detection.py, render_split_screen.py, render_camera_switch.py, and render_hybrid.py handle vertical framing, split-screen podcasts, and active-speaker switching. These modes depend on face detection and, for audio-driven speaker logic, diarization.py plus HuggingFace/Pyannote.

7. Story Clip mode is its own mini-pipeline under clipping/story/. loader.py, source_manager.py, and assembler.py suggest a multi-source narrative workflow where a recipe and multiple assets are assembled into a single story sequence. This is distinct from the normal “one source URL, many highlights” clip pipeline.

8. Outputs are file-based and job-scoped. The README indicates each run produces an outputs/ folder with final MP4s, thumbnails, render manifests, and metadata previews. That makes the pipeline effectively a file-producing batch job, with intermediate artifacts likely used for retries and post-processing.

9. The YouTube uploader is a separate executable boundary. run_upload.py and youtube_uploader/uploader.py consume the rendered outputs and scheduling metadata to publish clips to YouTube. This is intentionally decoupled from rendering so uploads can be run later or independently.

10. The web backend is a job-management layer around the same media pipeline. web/api/app.py, routes/jobs.py, routes/files.py, and routes/settings.py expose job submission, status, artifact access, and configuration. store.py implies persistent job/state storage, while worker.py suggests an asynchronous background executor for running clipping jobs.

11. The dashboard is a separate frontend client for that API. web/dashboard/ is a Vite/React app that calls the API through src/api.js and presents job creation, job detail, and settings pages. It is not the processing engine; it is the operator UI for the web layer.

12. External dependencies materially shape the architecture. FFmpeg is a hard runtime dependency for rendering, Faster-Whisper requires GPU-friendly compute for transcription, Gemini/NVIDIA NIM are used for semantic clip selection, Pexels/Pixabay provide optional media assets, and HuggingFace tokens enable diarization-based podcast modes.

13. Deployment is container-friendly but still batch-oriented. The repo includes a Dockerfile, docker-compose, env sample, and a Colab notebook, which indicates the project is designed to run as a reproducible media batch system across local GPU, container, and notebook environments.

14. Architecturally, the key boundary is: analysis → render → publish. The analysis stage creates structured decisions, the render stage converts them into media assets, and the upload/web layers consume those assets for distribution and management. An architecture diagram should keep those as separate boxes with file-based handoff between them.