# gyoridavid/short-video-maker architecture

> Architecture diagram of the GitHub repository gyoridavid/short-video-maker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gyoridavid/short-video-maker
- Repository: https://github.com/gyoridavid/short-video-maker

## Overview

##### Project Type and Purpose
- This is a full-stack, open-source automated video creation tool (“Short Video Maker”) that exposes both REST and MCP endpoints for generating short videos from text.
- Key responsibilities:
- Accepts scene definitions via API or MCP
- Converts text → speech (Kokoro.js)
- Generates captions (Whisper.cpp)
- Downloads background clips (Pexels API)
- Composes and renders video (Remotion + FFmpeg)
- Serves a minimal React-based Web UI

##### High-Level File Structure Analysis
- Top-level:
- Dockerfiles & docker-compose for containerized deployment
- .env.example, config files (tsconfig, vite, tailwind)
- package.json / pnpm-lock
- src/
- server/ – Node.js server entrypoint (server.ts) and routers for REST (rest.ts) and MCP (mcp.ts)
- short-creator/ – orchestration module + per-service adapters (FFmpeg, Kokoro TTS, Whisper, Pexels, Remotion)
- ui/ – React application (pages / components / styles) served by Remotion and Vite
- components/ – shared utilities & base React components for video composition (Remotion scene components)
- static/music – local MP3 assets for background music
- scripts/, config.ts, logger.ts, tests, mocks

##### README Insights
- Tech stack: Node.js + TypeScript, Express-style HTTP server, Remotion, FFmpeg, Whisper.cpp, Kokoro.js, Pexels API, Docker
- Two server modes: REST (HTTP JSON) and MCP (Model Context Protocol SSE/messages)
- Web UI served on port 3123 for manual video creation
- Configurable via environment variables and Docker images (tiny, normal, cuda)

a. Main Components:
1. Client Layer
– Web UI (React/Vite)
– REST HTTP client or MCP client (AI agent, e.g., n8n)
2. API Layer (Node.js Server)
– REST Router
– MCP Router
– Orchestration Controller
3. Short Creator Service (Orchestration Module)
– TTS Adapter (Kokoro.js)
– STT Adapter (Whisper.cpp)
– Video Search Adapter (Pexels API)
– Composition Engine (Remotion + OffthreadVideo)
– FFmpeg Post-processing
4. Storage & Cache
– Local filesystem (generated videos under /app/data/videos)
– In-memory or disk cache for video frames
– Static music assets
5. External Services
– Pexels API
– (Optional) n8n or other AI agents via MCP
b. Relationships & Data Flow:
1. User/browser or AI agent → HTTP POST /api/short-video or MCP SSE → Server
2. Server orchestration controller invokes Short Creator:
- Calls Kokoro TTS → returns audio stream
- Calls Whisper → returns caption timestamps
- Calls Pexels → returns background clip URLs
- Passes assets into Remotion scenes → renders frames
- Offloads to Chrome tabs (Remotion concurrency)
- Pipes result through FFmpeg (mux music + audio + video)
3. Final video persisted to disk → Server responds with videoId
4. Clients poll GET /status or subscribe to MCP messages
5. GET /api/short-video/{id} streams binary video
c. Architectural Patterns & Principles:
– Modular adapters for each external dependency (SOLID: single-responsibility)
– Clear separation between API layer, orchestration logic, and asset-processing libraries
– Concurrency control for Remotion (configurable)
– Dockerized builds targeting different resource profiles
d. Key Technologies & Frameworks:
– Node.js + TypeScript
– Express-style routing (server/routers/*.ts)
– Remotion (programmatic video composition)
– whisper.cpp (speech-to-text captions)
– kokoro-js (text-to-speech)
– FFmpeg (media manipulation)
– Pexels API (background videos)
– React + Vite + Tailwind CSS (UI)
– Docker & Docker Compose

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    subgraph "Client Layer"
        direction TB
        WebUI["Web UI (React/Vite)"]:::client
        RESTClient["REST HTTP Client"]:::client
        MCPClient["MCP Client"]:::client
    end

    %% API Server Layer
    subgraph "API Server" 
        direction TB
        APIEntry["Server Entry\n(src/index.ts)"]:::api
        Server["Node.js Server\n(src/server/server.ts)"]:::api
        Config["Config (src/config.ts)"]:::api
        Logger["Logger (src/logger.ts)"]:::api
        Validator["Request Validation\n(src/server/validator.ts)"]:::api
        RestRouter["REST Router\n(/api)\n(src/server/routers/rest.ts)"]:::api
        MCPRouter["MCP Router\n(MCP)\n(src/server/routers/mcp.ts)"]:::api
        OrchController["ShortCreator Controller\n(src/short-creator/ShortCreator.ts)"]:::core
    end

    %% Orchestration Layer
    subgraph "Short Creator Service" 
        direction TB
        TTS["TTS Adapter\n(Kokoro.js)"]:::core
        STT["STT Adapter\n(Whisper.cpp)"]:::core
        Search["Video Search Adapter\n(Pexels API)"]:::core
        Compose["Composition Engine\n(Remotion)"]:::core
        FFmpeg["FFmpeg Post-processing"]:::core
        MusicHelper["Music Helper"]:::core
    end

    %% Storage & Cache
    subgraph "Storage & Cache"
        direction TB
        Videos["Video Storage\n(/app/data/videos)"]:::storage
        Cache["Frame Cache"]:::storage
        StaticMusic["Static Music Assets"]:::storage
    end

    %% External Services
    subgraph "External Services"
        direction TB
        PexelsAPI["Pexels API"]:::external
        AIAgent["AI Agent\n(e.g., n8n)"]:::external
    end

    %% Flows
    WebUI -->|"POST /api/short-video"| RestRouter
    RESTClient -->|POST| RestRouter
    MCPClient -->|SSE → MCP| MCPRouter

    RestRouter -->|invoke| OrchController
    MCPRouter -->|invoke| OrchController

    OrchController -->|1 TTS Request| TTS
    OrchController -->|2 STT Request| STT
    OrchController -->|3 Search Request| Search
    OrchController -->|4 Compose Frames| Compose
    Compose --> Cache
    OrchController -->|5 FFmpeg Mux| FFmpeg
    MusicHelper --> FFmpeg

    TTS -->|audio stream| OrchController
    STT -->|captions| OrchController
    Search -->|clip URLs| OrchController
    Compose -->|frames| OrchController
    FFmpeg -->|final video| OrchController

    OrchController -->|persist| Videos
    StaticMusic -.-> MusicHelper
    PexelsAPI --> Search
    AIAgent -.-> MCPRouter

    WebUI -->|"GET /api/status"| RestRouter
    WebUI -->|"GET /api/short-video/{id} stream"| RestRouter

    %% Click Events
    click WebUI "https://github.com/gyoridavid/short-video-maker/tree/main/src/ui"
    click APIEntry "https://github.com/gyoridavid/short-video-maker/blob/main/src/index.ts"
    click Server "https://github.com/gyoridavid/short-video-maker/blob/main/src/server/server.ts"
    click Config "https://github.com/gyoridavid/short-video-maker/blob/main/src/config.ts"
    click Logger "https://github.com/gyoridavid/short-video-maker/blob/main/src/logger.ts"
    click Validator "https://github.com/gyoridavid/short-video-maker/blob/main/src/server/validator.ts"
    click RestRouter "https://github.com/gyoridavid/short-video-maker/blob/main/src/server/routers/rest.ts"
    click MCPRouter "https://github.com/gyoridavid/short-video-maker/blob/main/src/server/routers/mcp.ts"
    click OrchController "https://github.com/gyoridavid/short-video-maker/blob/main/src/short-creator/ShortCreator.ts"
    click TTS "https://github.com/gyoridavid/short-video-maker/blob/main/src/short-creator/libraries/Kokoro.ts"
    click STT "https://github.com/gyoridavid/short-video-maker/blob/main/src/short-creator/libraries/Whisper.ts"
    click Search "https://github.com/gyoridavid/short-video-maker/blob/main/src/short-creator/libraries/Pexels.ts"
    click Compose "https://github.com/gyoridavid/short-video-maker/blob/main/src/short-creator/libraries/Remotion.ts"
    click FFmpeg "https://github.com/gyoridavid/short-video-maker/blob/main/src/short-creator/libraries/FFmpeg.ts"
    click MusicHelper "https://github.com/gyoridavid/short-video-maker/blob/main/src/short-creator/music.ts"
    click StaticMusic "https://github.com/gyoridavid/short-video-maker/tree/main/static/music"

    %% Styles
    classDef client fill:#D4C0F9,stroke:#805AD5,color:#000
    classDef api fill:#BEE3F8,stroke:#3182CE,color:#000
    classDef core fill:#C6F6D5,stroke:#2F855A,color:#000
    classDef storage fill:#F0F0F0,stroke:#A0AEC0,color:#000
    classDef external fill:#FEEBC8,stroke:#DD6B20,color:#000
```
