How gaoharimran29-glitch/clip_forge works
1. What this repo is
- ClipForge is a two-tier application: a Next.js frontend for user interaction and a FastAPI backend that runs a long-running media-processing pipeline.
- The backend turns a YouTube URL into one or more vertical, captioned short clips using a graph-based workflow.
2. Primary request/response boundary
- The user starts in the frontend, enters a YouTube link, and triggers generation.
- The frontend talks to the backend over HTTP for job creation and over SSE-style streaming for live progress updates.
- The backend does not appear to be a simple synchronous API; it runs jobs in the background and streams status as the pipeline advances.
3. Frontend architecture
- The frontend is a Next.js App Router app under frontend/app.
- Key UI pieces are split into reusable components: input card, hero/header, source video embed, clip cards, and results section.
- frontend/hooks/useClipGeneration.ts is the main interaction hook for starting a job and consuming its status/results.
- frontend/lib/backend.ts centralizes backend API calls, while frontend/lib/youtube.ts handles YouTube-specific URL/embed logic.
- frontend/types/job.ts and frontend/types/clip.ts define the frontend’s data contract for job state and generated clips.
4. Backend entrypoint and API layer
- backend/app.py is the FastAPI entrypoint.
- This layer is responsible for exposing the generation endpoint, job/status endpoints, and the progress stream.
- backend/exceptions.py likely maps pipeline and input failures into API-friendly error responses.
5. Job orchestration boundary
- Long-running work is separated from request handling through backend/job_manager.py.
- That subsystem likely creates and tracks per-request jobs, so the HTTP request can return quickly while processing continues asynchronously.
- This is an important architectural boundary: web request lifecycle is distinct from media generation lifecycle.
6. Pipeline composition
- backend/graph.py and backend/graph_runner.py define and execute the core workflow.
- The README shows the pipeline as a LangGraph-based sequence of stages, so the graph is the central orchestration mechanism rather than ad hoc scripting.
- backend/state.py defines the shared state passed between stages, which is how intermediate artifacts and metadata move through the pipeline.
7. Pipeline stages as separate nodes
- The backend/nodes/ directory contains the execution units:
- yt_download.py downloads the source YouTube video.
- transcript.py generates speech-to-text transcripts.
- llm_analysis.py evaluates transcript segments for viral/high-retention potential.
- cut_clips.py extracts the selected moments into final clips, likely using FFmpeg.
- cleanup_disk.py removes temporary artifacts after processing.
- This node split is the clearest place to draw the internal data flow: download → transcript → LLM scoring → clip extraction → cleanup.
8. LLM and prompt subsystem
- backend/model/llm.py and backend/model/prompts.py isolate model configuration and prompt templates from the rest of the pipeline.
- This keeps the analysis stage decoupled from the graph execution logic and makes the “viral moment detection” behavior easy to change without touching the pipeline structure.
- The README indicates an external LLM service is used for analysis, so the backend depends on that service at runtime for clip selection quality.
9. Runtime artifacts and media handling
- The pipeline produces temporary media files, transcripts, analysis results, and final exported clips on disk during execution.
- cleanup_disk.py exists specifically to reclaim local storage, which is important because video workflows generate large intermediate files.
- The architecture is therefore file-artifact driven inside the backend, even though the user-facing interface is HTTP-based.
10. Progress, results, and user experience
- Live progress is a first-class part of the design, not an afterthought.
- The frontend can show generation state while the backend pipeline is still running, and then present the generated clips when the job completes.
- frontend/components/ResultsSection.tsx, ClipCard.tsx, and SourceVideoEmbed.tsx suggest the final UI combines the original source context with the output clips.
11. Deployment and infrastructure
- The repo is containerized end-to-end with Dockerfiles for both frontend and backend, plus a docker-compose.yml for local orchestration.
- The README and .github/workflows/deploy.yaml show a production deployment path using GitHub Actions to push updates to an AWS EC2 host.
- This means the operational boundary is: build in CI, deploy containerized services, and run the full app on a single server rather than a distributed cloud platform.
12. How to draw the architecture diagram
- Draw three layers: browser frontend, FastAPI orchestration backend, and external services/tooling.
- Inside the backend, show a linear-but-graph-managed processing chain with separate nodes for download, transcription, LLM analysis, clip cutting, and cleanup.
- Annotate the backend with two supporting subsystems: job management and live progress streaming.
- Show external dependencies at the edges: YouTube source, LLM provider, and FFmpeg/yt-dlp-style media tooling.
Main entry points: useClipGeneration.ts Generation hook · app.py FastAPI app · graph_runner.py Graph runner · yt_download.py Download stage
Components 22
Frontend
Next.js user interface and API client layer
- App shell · Main app entry and screen composition
frontend/app/page.tsx - URL input · Collects the YouTube link and starts generation
frontend/components/UrlInputCard.tsx - Generation hook · Coordinates job creation, progress updates, and final clip state
frontend/hooks/useClipGeneration.ts - Backend client · Centralizes HTTP and streaming calls to the backend
frontend/lib/backend.ts - Results view · Renders generated clips and job outcome
frontend/components/ResultsSection.tsx - Source embed · Shows the original YouTube video context
frontend/components/SourceVideoEmbed.tsx - Clip card · Displays an individual generated short clip
frontend/components/ClipCard.tsx
Backend
FastAPI orchestration and LangGraph pipeline
- FastAPI app · Exposes job creation, status, and progress streaming endpoints
backend/app.py - Job manager · Tracks long-running jobs outside the request lifecycle
backend/job_manager.py - Workflow graph · Defines the media-processing pipeline stages and transitions
backend/graph.py - Graph runner · Executes the graph and advances shared job state
backend/graph_runner.py - Job state · Carries intermediate artifacts and metadata between stages
backend/state.py - Download stage · Fetches the source YouTube video
backend/nodes/yt_download.py - Transcript stage · Extracts speech-to-text captions from the source media
backend/nodes/transcript.py - Analysis stage · Scores transcript segments for clip-worthy moments
backend/nodes/llm_analysis.py - Cut stage · Builds the final vertical short clips
backend/nodes/cut_clips.py - Cleanup stage · Removes temporary media artifacts after processing
backend/nodes/cleanup_disk.py - LLM config · Holds model wiring and prompt templates for analysis
backend/model/llm.py
External
Outside services, media tools, and deployment targets
- YouTube · Input source videos referenced by user URLs
- LLM provider · Remote model service used to score and select viral moments
- Media tools · Command-line tooling used for download and clip extraction
- EC2 deploy · GitHub Actions deploys containerized services to an AWS EC2 host
Connections 24
- App shell → URL input: renders · The app shell presents the URL entry surface
- App shell → Results view: renders · The app shell switches into the results view after generation
- URL input → Generation hook: starts job · Submitting a URL triggers the generation hook
- Generation hook → Backend client: calls · The hook uses the backend client for request and stream handling
- Backend client → FastAPI app: HTTP + SSE · Frontend communicates with the FastAPI app for job lifecycle and progress
- Generation hook → Source embed: updates · The hook provides source video metadata for preview
- Generation hook → Results view: feeds · Job completion state populates the results section
- Results view → Clip card: contains · Results are rendered as individual clip cards
- FastAPI app → Job manager: delegates · API requests are handed off to job orchestration
- FastAPI app → Graph runner: streams status · The API exposes live progress while execution continues
- Job manager → Workflow graph: selects workflow · Jobs are associated with the graph-based pipeline
- Workflow graph → Graph runner: drives · The runner executes the defined workflow stages
- Graph runner → Job state: reads/writes · Stages exchange artifacts through shared state
- Graph runner → Download stage: runs · Pipeline begins with source download
- Download stage → YouTube: fetches · Downloads the YouTube source video
- Download stage → Transcript stage: passes media · Downloaded media feeds transcription
- Transcript stage → Analysis stage: passes text · Transcript output becomes analysis input
- Analysis stage → LLM config: uses · Analysis stage relies on model configuration and prompts
- LLM config → LLM provider: calls · Model execution depends on an external LLM service
- Analysis stage → Cut stage: selects segments · Scored moments are turned into clip cut instructions
- Cut stage → Media tools: uses · Clip extraction depends on media tooling
- Cut stage → Cleanup stage: finalizes · Cleanup runs after clips are exported
- Cleanup stage → Media tools: clears artifacts · Temporary files created by media tooling are removed
- FastAPI app → EC2 deploy: deployed via · The containerized application is delivered through CI to EC2