# balajiraoiim/openshorts architecture

> Architecture diagram of the GitHub repository balajiraoiim/openshorts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/balajiraoiim/openshorts
- Repository: https://github.com/balajiraoiim/openshorts

## Overview

1. **What this repo is**
   - OpenShorts.app is a self-hosted, full-stack AI video platform.
   - The architecture centers on a **Python backend** that runs media/AI pipelines and a **React dashboard** that operators use to trigger jobs, inspect results, and publish content.

2. **Top-level split: backend vs. dashboard**
   - The `dashboard/` directory is a separate Vite + React app with its own Dockerfile and package lock, so the UI is independently built and served.
   - The Python files at the repo root (`main.py`, `app.py`, `editor.py`, `saasshorts.py`, etc.) form the backend/service layer and pipeline modules.

3. **User-facing subsystems**
   - The dashboard is organized around three major product areas reflected in the UI components:
     - **Clip Generator**
     - **AI Shorts / UGC videos**
     - **YouTube Studio**
   - There is also a **public gallery** experience for generated videos and avatars, with SEO-oriented pages.

4. **Frontend architecture**
   - The dashboard is a single-page app built with **React 18**, **Vite**, and **Tailwind CSS**.
   - Key component boundaries in `dashboard/src/components/` suggest feature-specific UI modules rather than a monolithic screen:
     - media ingestion: `MediaInput`, `KeyInput`
     - generation flows: `SaaShortsTab`, `ThumbnailStudio`
     - content editing and overlays: `HookModal`, `SubtitleModal`, `TranslateModal`
     - results and browsing: `ResultCard`, `Gallery`, `UGCGallery`, `GalleryCard`
     - processing feedback: `ProcessingAnimation`
   - This UI is likely a thin orchestration layer over backend jobs rather than doing heavy processing in-browser.

5. **Backend orchestration layer**
   - The backend appears to be the system of record for job creation, processing, and asset management.
   - `main.py` and `app.py` are the likely server entrypoints; the README says the backend serves both APIs and SEO/public pages.
   - The backend is responsible for coordinating pipeline steps, storing results, and exposing them to the dashboard and gallery pages.

6. **Clip Generator pipeline**
   - The README describes a pipeline that transforms long videos or uploads into shorts:
     - ingest from YouTube or local file
     - transcribe with Whisper
     - detect scenes with PySceneDetect
     - analyze moments with Gemini
     - cut clips with FFmpeg
     - reframe vertically using face/subject tracking
     - add subtitles, hook text, and effects
     - back up and publish outputs
   - This subsystem is media-processing heavy and likely runs as async jobs rather than request/response work.

7. **AI Shorts / UGC video pipeline**
   - `saasshorts.py` and related modules likely implement the second major generation flow: marketing-style UGC videos from a product URL or description.
   - The pipeline spans analysis, script generation, avatar creation, voiceover, talking-head generation, b-roll generation, compositing, and final publishing.
   - This is a separate domain from clip extraction: it synthesizes a new video from text, product context, and AI-generated assets.

8. **YouTube Studio subsystem**
   - The “studio” feature is a third pipeline focused on creator utilities:
     - title generation
     - thumbnail generation
     - description/chapter generation
     - direct publishing to YouTube
   - The presence of `thumbnail.py`, `translate.py`, and `editor.py` suggests specialized post-processing modules that support this studio workflow.

9. **Shared media and storage boundary**
   - The architecture distinguishes between transient local processing and durable storage.
   - `s3_uploader.py` plus the README’s S3 setup indicate two buckets/boundaries:
     - a **private bucket** for generated clips and working outputs
     - a **public bucket** for gallery assets and avatars
   - The gallery pages depend on these public assets being directly reachable, while job outputs can remain private until published.

10. **External AI/service integrations**
   - The core system depends on multiple external services that define major architectural boundaries:
     - **Google Gemini** for analysis, script generation, and viral-moment detection
     - **fal.ai** for image/video/actor generation
     - **ElevenLabs** for voiceover and dubbing
     - **Upload-Post** for social publishing to TikTok, Instagram, and YouTube
   - These integrations are not optional implementation details; they materially shape the end-to-end workflow and cost model.

11. **Publishing and distribution**
   - After generation, content can be distributed to social platforms through the Upload-Post integration.
   - The platform supports direct publishing and scheduled publishing, so the backend likely manages a publish queue or deferred jobs separate from the media creation pipeline.

12. **Job control and operational concerns**
   - The README mentions semaphore-based concurrency (`MAX_CONCURRENT_JOBS`) and automatic cleanup, implying background job management is a first-class part of the backend design.
   - This suggests the backend is built to queue and throttle expensive media/AI jobs, rather than executing them synchronously in API handlers.

13. **Validation and quality gates**
   - Files like `verify_aesthetic.py`, `verify_custom_hook.py`, and `verify_hooks.py` indicate post-generation validation checks.
   - These likely act as quality gates for generated assets before they are surfaced in the UI, stored publicly, or published externally.

14. **Deployment shape**
   - The repo is containerized with a root `Dockerfile`, a `docker-compose.yml`, and a separate `dashboard/Dockerfile`.
   - That implies at least two deployable services: the backend service and the frontend dashboard service, with shared environment configuration via `.env`.

15. **Diagram-friendly mental model**
   - Draw this as: **Dashboard UI** → **Backend API/orchestrator** → **AI/media pipelines** → **storage/public gallery/publishing services**.
   - The important boundaries are:
     - browser vs. server
     - request handling vs. async job execution
     - transient processing vs. durable S3/public assets
     - internal generation vs. external AI and social APIs

Main entry points: [`dashboard/src/App.jsx`](https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/App.jsx) (Dashboard App), [`main.py`](https://github.com/balajiraoiim/openshorts/blob/main/main.py) (Main Service), [`saasshorts.py`](https://github.com/balajiraoiim/openshorts/blob/main/saasshorts.py) (UGC Pipeline), [`s3_uploader.py`](https://github.com/balajiraoiim/openshorts/blob/main/s3_uploader.py) (S3 Uploader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_dashboard["Dashboard UI"]
  node_dashboard_app["Dashboard App<br/>react spa<br/>[App.jsx]"]
  node_dashboard_media["Media Input<br/>ui module<br/>[MediaInput.jsx]"]
  node_dashboard_shorts["Shorts Tab<br/>ui module<br/>[SaaShortsTab.jsx]"]
  node_dashboard_studio["Thumbnail Studio<br/>ui module"]
  node_dashboard_gallery["Gallery View<br/>ui module<br/>[Gallery.jsx]"]
  node_dashboard_ugc["UGC Gallery<br/>ui module<br/>[UGCGallery.jsx]"]
  node_dashboard_feedback["Processing UI<br/>ui module"]
end

subgraph group_backend["Python Backend"]
  node_server["Backend API<br/>python service<br/>[app.py]"]
  node_server_main["Main Service<br/>python entrypoint<br/>[main.py]"]
  node_clip_pipeline["Clip Pipeline<br/>media workflow"]
  node_ugc_pipeline["UGC Pipeline<br/>ai video workflow<br/>[saasshorts.py]"]
  node_studio_tools["Studio Tools<br/>post-process modules<br/>[thumbnail.py]"]
  node_validation["Quality Gates<br/>verification<br/>[verify_hooks.py]"]
  node_storage["S3 Uploader<br/>storage boundary<br/>[s3_uploader.py]"]
end

subgraph group_integrations["External Services"]
  node_gemini{{"Gemini<br/>ai analysis api"}}
  node_fal{{"fal.ai<br/>generation api"}}
  node_elevenlabs{{"ElevenLabs<br/>voice api"}}
  node_uploadpost{{"Upload-Post<br/>publishing api"}}
end

node_dashboard_app --> node_dashboard_media
node_dashboard_app --> node_dashboard_shorts
node_dashboard_app --> node_dashboard_studio
node_dashboard_app --> node_dashboard_gallery
node_dashboard_app --> node_dashboard_ugc
node_dashboard_app --> node_dashboard_feedback
node_dashboard_shorts --> node_server
node_dashboard_studio --> node_server
node_dashboard_gallery --> node_server_main
node_dashboard_ugc --> node_storage
node_server --> node_server_main
node_server_main --> node_clip_pipeline
node_server_main --> node_ugc_pipeline
node_server_main --> node_studio_tools
node_server_main --> node_validation
node_clip_pipeline --> node_gemini
node_clip_pipeline --> node_storage
node_ugc_pipeline --> node_gemini
node_ugc_pipeline --> node_fal
node_ugc_pipeline --> node_elevenlabs
node_ugc_pipeline --> node_storage
node_studio_tools --> node_uploadpost
node_validation --> node_storage
node_storage --> node_uploadpost

click node_dashboard_app "https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/App.jsx"
click node_dashboard_media "https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/MediaInput.jsx"
click node_dashboard_shorts "https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/SaaShortsTab.jsx"
click node_dashboard_studio "https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/ThumbnailStudio.jsx"
click node_dashboard_gallery "https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/Gallery.jsx"
click node_dashboard_ugc "https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/UGCGallery.jsx"
click node_dashboard_feedback "https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/ProcessingAnimation.jsx"
click node_server "https://github.com/balajiraoiim/openshorts/blob/main/app.py"
click node_server_main "https://github.com/balajiraoiim/openshorts/blob/main/main.py"
click node_ugc_pipeline "https://github.com/balajiraoiim/openshorts/blob/main/saasshorts.py"
click node_studio_tools "https://github.com/balajiraoiim/openshorts/blob/main/thumbnail.py"
click node_validation "https://github.com/balajiraoiim/openshorts/blob/main/verify_hooks.py"
click node_storage "https://github.com/balajiraoiim/openshorts/blob/main/s3_uploader.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard_app,node_dashboard_media,node_dashboard_shorts,node_dashboard_studio,node_dashboard_gallery,node_dashboard_ugc,node_dashboard_feedback toneBlue
class node_server,node_server_main,node_clip_pipeline,node_ugc_pipeline,node_studio_tools,node_validation,node_storage toneAmber
class node_gemini,node_fal,node_elevenlabs,node_uploadpost toneMint
```

## Components

### Dashboard UI

Separate React/Vite frontend for operators and browsing

- [Dashboard App](https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/App.jsx): react spa · Operator UI for generation, inspection, and publishing · `dashboard/src/App.jsx`
- [Media Input](https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/MediaInput.jsx): ui module · Collects source video or URL inputs · `dashboard/src/components/MediaInput.jsx`
- [Shorts Tab](https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/SaaShortsTab.jsx): ui module · Main clip-generation workflow panel · `dashboard/src/components/SaaShortsTab.jsx`
- [Thumbnail Studio](https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/ThumbnailStudio.jsx): ui module · YouTube studio tools for titles and thumbnails · `dashboard/src/components/ThumbnailStudio.jsx`
- [Gallery View](https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/Gallery.jsx): ui module · Browsing surface for generated videos · `dashboard/src/components/Gallery.jsx`
- [UGC Gallery](https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/UGCGallery.jsx): ui module · Public gallery for avatars and generated marketing videos · `dashboard/src/components/UGCGallery.jsx`
- [Processing UI](https://github.com/balajiraoiim/openshorts/blob/main/dashboard/src/components/ProcessingAnimation.jsx): ui module · Generation progress and result presentation · `dashboard/src/components/ProcessingAnimation.jsx`

### Python Backend

Server, orchestration, and media pipeline modules

- [Backend API](https://github.com/balajiraoiim/openshorts/blob/main/app.py): python service · HTTP entrypoint and request orchestration · `app.py`
- [Main Service](https://github.com/balajiraoiim/openshorts/blob/main/main.py): python entrypoint · Primary backend runtime and job control · `main.py`
- **Clip Pipeline**: media workflow · Transforms long-form video into shorts
- [UGC Pipeline](https://github.com/balajiraoiim/openshorts/blob/main/saasshorts.py): ai video workflow · Generates marketing-style videos from product context · `saasshorts.py`
- [Studio Tools](https://github.com/balajiraoiim/openshorts/blob/main/thumbnail.py): post-process modules · Titles, thumbnails, translation, and editor support · `thumbnail.py`
- [Quality Gates](https://github.com/balajiraoiim/openshorts/blob/main/verify_hooks.py): verification · Post-generation checks before publish or gallery exposure · `verify_hooks.py`
- [S3 Uploader](https://github.com/balajiraoiim/openshorts/blob/main/s3_uploader.py): storage boundary · Moves working outputs and public assets into buckets · `s3_uploader.py`

### External Services

Third-party AI, storage, and publishing boundaries

- **Gemini**: ai analysis api · Analysis, scripting, and moment detection
- **fal.ai**: generation api · Image, avatar, and video generation
- **ElevenLabs**: voice api · Voiceover and dubbing generation
- **Upload-Post**: publishing api · Social distribution to YouTube, TikTok, and Instagram

## Connections

- Dashboard App → Media Input: Collects source inputs
- Dashboard App → Shorts Tab: Launches clip generation
- Dashboard App → Thumbnail Studio: Opens creator tools
- Dashboard App → Gallery View: Browses generated clips
- Dashboard App → UGC Gallery: Browses public UGC assets
- Dashboard App → Processing UI: Shows job progress and results
- Shorts Tab → Backend API: Submits generation jobs
- Thumbnail Studio → Backend API: Requests titles, thumbnails, and edits
- Gallery View → Main Service: Fetches gallery data
- UGC Gallery → S3 Uploader: Loads public assets
- Backend API → Main Service: Delegates request handling
- Main Service → Clip Pipeline: Runs shorts workflow
- Main Service → UGC Pipeline: Runs UGC generation workflow
- Main Service → Studio Tools: Invokes studio utilities
- Main Service → Quality Gates: Checks generated outputs
- Clip Pipeline → Gemini: Analyzes content and moments
- Clip Pipeline → S3 Uploader: Stores clips and artifacts
- UGC Pipeline → Gemini: Generates scripts and analysis
- UGC Pipeline → fal.ai: Creates avatars and video assets
- UGC Pipeline → ElevenLabs: Generates narration
- UGC Pipeline → S3 Uploader: Uploads final outputs
- Studio Tools → Upload-Post: Publishes or schedules content
- Quality Gates → S3 Uploader: Approves assets for durable storage
- S3 Uploader → Upload-Post: Supplies publishable media
