# naufalrizqullah/opensource-clipping architecture

> Architecture diagram of the GitHub repository naufalrizqullah/opensource-clipping, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/naufalrizqullah/opensource-clipping
- Repository: https://github.com/naufalrizqullah/opensource-clipping

## Overview

1. **The repository has two main surfaces:** a **batch video-processing pipeline** driven from the CLI, and a **web control plane** built from the FastAPI backend plus React dashboard. The CLI path does the heavy media work; the web path mainly manages jobs, settings, and file access.

2. **The core pipeline lives in `clipping/` and is orchestrated by `main.py` → `clipping.runner`.** `config.py` parses the large CLI surface, `runner.py` coordinates the stages, and `engine.py` handles the “analysis” side of the workflow. This is the backbone for turning one source URL into one or more finished clips.

3. **Source ingestion and analysis form the first data plane.** The pipeline can pull from YouTube, TikTok, Instagram, or Google Drive, then transcribe with Faster-Whisper, optionally reuse built-in subtitles, and send transcript/context to an LLM provider. `engine.py`, `metadata.py`, and `hook_manager.py` are the main pieces involved in extracting moments, normalizing metadata, and selecting hook material.

4. **AI selection is separated from rendering.** Gemini is the default analysis provider, with NVIDIA NIM as an alternative. The AI layer decides clip ranks, titles, descriptions, tags, hook candidates, and segment trimming decisions; the render layer consumes those decisions rather than re-analyzing content.

5. **`clipping/studio/` is the media-rendering subsystem.** It contains the FFmpeg-driven composition logic for subtitles, typography, transitions, effects, B-roll, BGM, thumbnail generation, and layout modes. The modular files here show that rendering is composed from specialized renderers rather than one monolith.

6. **Face tracking and podcast layouts are a major rendering boundary.** `face_detection.py`, `render_split_screen.py`, `render_camera_switch.py`, and `render_hybrid.py` handle vertical framing, split-screen podcasts, and active-speaker switching. These modes depend on face detection and, for audio-driven speaker logic, `diarization.py` plus HuggingFace/Pyannote.

7. **Story Clip mode is its own mini-pipeline under `clipping/story/`.** `loader.py`, `source_manager.py`, and `assembler.py` suggest a multi-source narrative workflow where a recipe and multiple assets are assembled into a single story sequence. This is distinct from the normal “one source URL, many highlights” clip pipeline.

8. **Outputs are file-based and job-scoped.** The README indicates each run produces an `outputs/` folder with final MP4s, thumbnails, render manifests, and metadata previews. That makes the pipeline effectively a file-producing batch job, with intermediate artifacts likely used for retries and post-processing.

9. **The YouTube uploader is a separate executable boundary.** `run_upload.py` and `youtube_uploader/uploader.py` consume the rendered outputs and scheduling metadata to publish clips to YouTube. This is intentionally decoupled from rendering so uploads can be run later or independently.

10. **The web backend is a job-management layer around the same media pipeline.** `web/api/app.py`, `routes/jobs.py`, `routes/files.py`, and `routes/settings.py` expose job submission, status, artifact access, and configuration. `store.py` implies persistent job/state storage, while `worker.py` suggests an asynchronous background executor for running clipping jobs.

11. **The dashboard is a separate frontend client for that API.** `web/dashboard/` is a Vite/React app that calls the API through `src/api.js` and presents job creation, job detail, and settings pages. It is not the processing engine; it is the operator UI for the web layer.

12. **External dependencies materially shape the architecture.** FFmpeg is a hard runtime dependency for rendering, Faster-Whisper requires GPU-friendly compute for transcription, Gemini/NVIDIA NIM are used for semantic clip selection, Pexels/Pixabay provide optional media assets, and HuggingFace tokens enable diarization-based podcast modes.

13. **Deployment is container-friendly but still batch-oriented.** The repo includes a Dockerfile, docker-compose, env sample, and a Colab notebook, which indicates the project is designed to run as a reproducible media batch system across local GPU, container, and notebook environments.

14. **Architecturally, the key boundary is: analysis → render → publish.** The analysis stage creates structured decisions, the render stage converts them into media assets, and the upload/web layers consume those assets for distribution and management. An architecture diagram should keep those as separate boxes with file-based handoff between them.

Main entry points: [`clipping/runner.py`](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/runner.py) (Runner), [`clipping/studio.py`](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio.py) (Studio), [`clipping/engine.py`](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/engine.py) (Engine), [`clipping/story_runner.py`](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/story_runner.py) (Story runner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cli["CLI pipeline"]
  node_main["CLI entry<br/>entrypoint<br/>[main.py]"]
  node_config["Config<br/>cli parser<br/>[config.py]"]
  node_runner["Runner<br/>orchestrator<br/>[runner.py]"]
  node_engine["Engine<br/>analysis core<br/>[engine.py]"]
  node_metadata["Metadata<br/>normalizer<br/>[metadata.py]"]
  node_hook_manager["Hook manager<br/>selector<br/>[hook_manager.py]"]
  node_studio["Studio<br/>render subsystem<br/>[studio.py]"]
  node_face_detection["Face tracking<br/>render aid<br/>[face_detection.py]"]
  node_split_screen["Split screen<br/>podcast layout"]
  node_camera_switch["Camera switch<br/>podcast layout"]
  node_hybrid["Hybrid render<br/>layout mode<br/>[render_hybrid.py]"]
  node_story["Story clip<br/>mini pipeline<br/>[assembler.py]"]
  node_story_loader["Story loader<br/>input loader<br/>[loader.py]"]
  node_story_sources["Source manager<br/>source planner<br/>[source_manager.py]"]
  node_story_runner["Story runner<br/>story entrypoint<br/>[story_runner.py]"]
end

subgraph group_web["Web control plane"]
  node_api_app["API app<br/>backend app<br/>[app.py]"]
  node_jobs["Jobs API<br/>route<br/>[jobs.py]"]
  node_files["Files API<br/>route<br/>[files.py]"]
  node_settings["Settings API<br/>route<br/>[settings.py]"]
  node_store["Store<br/>state storage<br/>[store.py]"]
  node_worker["Worker<br/>background executor<br/>[worker.py]"]
  node_dashboard["Dashboard<br/>frontend app<br/>[App.jsx]"]
  node_api_client["API client<br/>frontend api<br/>[api.js]"]
end

subgraph group_publish["Publishing"]
  node_upload["YouTube upload<br/>publisher<br/>[run_upload.py]"]
  node_uploader["Uploader<br/>publish worker<br/>[uploader.py]"]
end

subgraph group_external["External systems"]
  node_ffmpeg[("FFmpeg<br/>runtime")]
  node_whisper["Faster-Whisper<br/>transcription"]
  node_llm["Gemini/NIM<br/>ai provider"]
  node_diarization["Pyannote<br/>speaker diarization<br/>[diarization.py]"]
  node_assets["Media sources<br/>source systems"]
end

node_main -->|"parse args"| node_config
node_main -->|"start job"| node_runner
node_runner -->|"analyze"| node_engine
node_runner -->|"build metadata"| node_metadata
node_runner -->|"select hooks"| node_hook_manager
node_engine -->|"ingest source"| node_assets
node_engine -->|"transcribe"| node_whisper
node_engine -->|"rank clips"| node_llm
node_engine -->|"speaker map"| node_diarization
node_runner -->|"render"| node_studio
node_studio -->|"encode"| node_ffmpeg
node_studio -->|"track faces"| node_face_detection
node_studio -->|"compose mode"| node_split_screen
node_studio -->|"compose mode"| node_camera_switch
node_studio -->|"compose mode"| node_hybrid
node_runner -->|"story mode"| node_story_runner
node_story_runner -->|"load recipe"| node_story_loader
node_story_runner -->|"plan sources"| node_story_sources
node_story_runner -->|"assemble"| node_story
node_runner -->|"publish handoff"| node_upload
node_upload -->|"upload clip"| node_uploader
node_api_app -->|"route"| node_jobs
node_api_app -->|"route"| node_files
node_api_app -->|"route"| node_settings
node_jobs -->|"enqueue"| node_worker
node_worker -->|"run pipeline"| node_runner
node_jobs -->|"persist state"| node_store
node_files -->|"read artifacts"| node_store
node_dashboard -->|"call api"| node_api_client
node_api_client -->|"http"| node_api_app

click node_main "https://github.com/naufalrizqullah/opensource-clipping/blob/main/main.py"
click node_config "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/config.py"
click node_runner "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/runner.py"
click node_engine "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/engine.py"
click node_metadata "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/metadata.py"
click node_hook_manager "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/hook_manager.py"
click node_studio "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio.py"
click node_face_detection "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio/face_detection.py"
click node_split_screen "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio/render_split_screen.py"
click node_camera_switch "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio/render_camera_switch.py"
click node_hybrid "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio/render_hybrid.py"
click node_story "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/story/assembler.py"
click node_story_loader "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/story/loader.py"
click node_story_sources "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/story/source_manager.py"
click node_story_runner "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/story_runner.py"
click node_upload "https://github.com/naufalrizqullah/opensource-clipping/blob/main/run_upload.py"
click node_uploader "https://github.com/naufalrizqullah/opensource-clipping/blob/main/youtube_uploader/uploader.py"
click node_api_app "https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/app.py"
click node_jobs "https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/routes/jobs.py"
click node_files "https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/routes/files.py"
click node_settings "https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/routes/settings.py"
click node_store "https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/store.py"
click node_worker "https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/worker.py"
click node_dashboard "https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/dashboard/src/App.jsx"
click node_api_client "https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/dashboard/src/api.js"
click node_diarization "https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/diarization.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_config,node_runner,node_engine,node_metadata,node_hook_manager,node_studio,node_face_detection,node_split_screen,node_camera_switch,node_hybrid,node_story,node_story_loader,node_story_sources,node_story_runner toneBlue
class node_api_app,node_jobs,node_files,node_settings,node_store,node_worker,node_dashboard,node_api_client toneAmber
class node_upload,node_uploader toneMint
class node_ffmpeg,node_whisper,node_llm,node_diarization,node_assets toneRose
```

## Components

### CLI pipeline

Batch clip generation and rendering pipeline driven from the command line.

- [CLI entry](https://github.com/naufalrizqullah/opensource-clipping/blob/main/main.py): entrypoint · Starts the batch clipping workflow. · `main.py`
- [Config](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/config.py): cli parser · Parses the large command-line surface and run options. · `clipping/config.py`
- [Runner](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/runner.py): orchestrator · Coordinates the end-to-end clip job pipeline. · `clipping/runner.py`
- [Engine](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/engine.py): analysis core · Runs source ingestion, transcription, analysis, and moment selection. · `clipping/engine.py`
- [Metadata](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/metadata.py): normalizer · Builds and normalizes clip metadata and output descriptors. · `clipping/metadata.py`
- [Hook manager](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/hook_manager.py): selector · Selects and manages hook material and attention-grabbing segments. · `clipping/hook_manager.py`
- [Studio](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio.py): render subsystem · FFmpeg-driven media rendering and composition layer. · `clipping/studio.py`
- [Face tracking](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio/face_detection.py): render aid · Detects faces for framing and speaker-aware layouts. · `clipping/studio/face_detection.py`
- [Split screen](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio/render_split_screen.py): podcast layout · Renders split-screen podcast and multi-camera compositions. · `clipping/studio/render_split_screen.py`
- [Camera switch](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio/render_camera_switch.py): podcast layout · Performs active-speaker switching and dynamic framing. · `clipping/studio/render_camera_switch.py`
- [Hybrid render](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/studio/render_hybrid.py): layout mode · Combines speaker, face, and layout logic for mixed formats. · `clipping/studio/render_hybrid.py`
- [Story clip](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/story/assembler.py): mini pipeline · Multi-source story workflow for assembling narrative clips. · `clipping/story/assembler.py`
- [Story loader](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/story/loader.py): input loader · Loads story assets and source definitions. · `clipping/story/loader.py`
- [Source manager](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/story/source_manager.py): source planner · Coordinates multiple sources for story assembly. · `clipping/story/source_manager.py`
- [Story runner](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/story_runner.py): story entrypoint · Runs the story-specific pipeline from recipe to output. · `clipping/story_runner.py`

### Web control plane

FastAPI backend and React dashboard for job management and file access.

- [API app](https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/app.py): backend app · FastAPI application that exposes job and file APIs. · `web/api/app.py`
- [Jobs API](https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/routes/jobs.py): route · Submits jobs and reports job status. · `web/api/routes/jobs.py`
- [Files API](https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/routes/files.py): route · Serves rendered artifacts and job files. · `web/api/routes/files.py`
- [Settings API](https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/routes/settings.py): route · Exposes configuration and runtime settings. · `web/api/routes/settings.py`
- [Store](https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/store.py): state storage · Persists job state and backend data. · `web/api/store.py`
- [Worker](https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/api/worker.py): background executor · Runs clipping jobs asynchronously in the web backend. · `web/api/worker.py`
- [Dashboard](https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/dashboard/src/App.jsx): frontend app · React/Vite operator UI for jobs, details, and settings. · `web/dashboard/src/App.jsx`
- [API client](https://github.com/naufalrizqullah/opensource-clipping/blob/main/web/dashboard/src/api.js): frontend api · Frontend client for calling the FastAPI backend. · `web/dashboard/src/api.js`

### Publishing

Decoupled upload boundary for distributing rendered clips.

- [YouTube upload](https://github.com/naufalrizqullah/opensource-clipping/blob/main/run_upload.py): publisher · Consumes rendered outputs and scheduling metadata to publish clips. · `run_upload.py`
- [Uploader](https://github.com/naufalrizqullah/opensource-clipping/blob/main/youtube_uploader/uploader.py): publish worker · Implements the YouTube upload flow. · `youtube_uploader/uploader.py`

### External systems

Third-party runtimes, model providers, and asset sources used by the pipeline.

- **FFmpeg**: runtime · Media encoding and composition runtime.
- **Faster-Whisper**: transcription · Speech transcription engine for source media.
- **Gemini/NIM**: ai provider · Semantic analysis provider for clip selection and copy generation.
- [Pyannote](https://github.com/naufalrizqullah/opensource-clipping/blob/main/clipping/diarization.py): speaker diarization · Speaker segmentation for podcast and active-speaker modes. · `clipping/diarization.py`
- **Media sources**: source systems · YouTube, TikTok, Instagram, Drive, and optional stock asset providers.

## Connections

- CLI entry → Config: parse args · Command-line options are parsed before the job starts.
- CLI entry → Runner: start job · Entry point hands control to the pipeline orchestrator.
- Runner → Engine: analyze · Runner invokes analysis and ingestion stages.
- Runner → Metadata: build metadata · Runner collects normalized clip metadata.
- Runner → Hook manager: select hooks · Runner requests hook material and segment choices.
- Engine → Media sources: ingest source · Engine pulls media from supported source systems.
- Engine → Faster-Whisper: transcribe · Speech is converted to transcript text for analysis.
- Engine → Gemini/NIM: rank clips · Transcript and context are sent to an AI provider for clip decisions.
- Engine → Pyannote: speaker map · Podcast modes use diarization for speaker-aware analysis.
- Runner → Studio: render · Structured analysis results are handed to the render subsystem.
- Studio → FFmpeg: encode · Rendering is executed through FFmpeg-based composition.
- Studio → Face tracking: track faces · Layout modes use face detection for framing.
- Studio → Split screen: compose mode · Split-screen podcast layouts are rendered here.
- Studio → Camera switch: compose mode · Active-speaker switching layouts are rendered here.
- Studio → Hybrid render: compose mode · Hybrid render mode combines multiple layout strategies.
- Runner → Story runner: story mode · Story clip jobs follow a dedicated entry path.
- Story runner → Story loader: load recipe · Story runner loads narrative sources and assets.
- Story runner → Source manager: plan sources · Story runner coordinates multiple source inputs.
- Story runner → Story clip: assemble · Assets are assembled into a story sequence.
- Runner → YouTube upload: publish handoff · Rendered outputs can be passed to the uploader boundary.
- YouTube upload → Uploader: upload clip · Upload executable delegates to the YouTube uploader implementation.
- API app → Jobs API: route · Job submission and status endpoints are mounted on the API.
- API app → Files API: route · Artifact access endpoints are mounted on the API.
- API app → Settings API: route · Settings endpoints are mounted on the API.
- Jobs API → Worker: enqueue · Submitted jobs are handed to the background worker.
- Worker → Runner: run pipeline · The web backend executes the same clipping pipeline asynchronously.
- Jobs API → Store: persist state · Job records and status updates are stored persistently.
- Files API → Store: read artifacts · File views resolve outputs and job artifacts from backend storage.
- Dashboard → API client: call api · Frontend pages use a shared API client.
- API client → API app: http · Dashboard communicates with the FastAPI backend.