How gyoridavid/short-video-maker works
Project Type and Purpose
• This is a full-stack, open-source automated video creation tool (“Short Video Maker”) that exposes both REST and MCP endpoints for generating short videos from text.
• Key responsibilities:
• Accepts scene definitions via API or MCP
• Converts text → speech (Kokoro.js)
• Generates captions (Whisper.cpp)
• Downloads background clips (Pexels API)
• Composes and renders video (Remotion + FFmpeg)
• Serves a minimal React-based Web UI
High-Level File Structure Analysis
• Top-level:
• Dockerfiles & docker-compose for containerized deployment
• .env.example, config files (tsconfig, vite, tailwind)
• package.json / pnpm-lock
• src/
• server/ – Node.js server entrypoint (server.ts) and routers for REST (rest.ts) and MCP (mcp.ts)
• short-creator/ – orchestration module + per-service adapters (FFmpeg, Kokoro TTS, Whisper, Pexels, Remotion)
• ui/ – React application (pages / components / styles) served by Remotion and Vite
• components/ – shared utilities & base React components for video composition (Remotion scene components)
• static/music – local MP3 assets for background music
• scripts/, config.ts, logger.ts, tests, mocks
README Insights
• Tech stack: Node.js + TypeScript, Express-style HTTP server, Remotion, FFmpeg, Whisper.cpp, Kokoro.js, Pexels API, Docker
• Two server modes: REST (HTTP JSON) and MCP (Model Context Protocol SSE/messages)
• Web UI served on port 3123 for manual video creation
• Configurable via environment variables and Docker images (tiny, normal, cuda)
a. Main Components:
1. Client Layer
– Web UI (React/Vite)
– REST HTTP client or MCP client (AI agent, e.g., n8n)
2. API Layer (Node.js Server)
– REST Router
– MCP Router
– Orchestration Controller
3. Short Creator Service (Orchestration Module)
– TTS Adapter (Kokoro.js)
– STT Adapter (Whisper.cpp)
– Video Search Adapter (Pexels API)
– Composition Engine (Remotion + OffthreadVideo)
– FFmpeg Post-processing
4. Storage & Cache
– Local filesystem (generated videos under /app/data/videos)
– In-memory or disk cache for video frames
– Static music assets
5. External Services
– Pexels API
– (Optional) n8n or other AI agents via MCP
b. Relationships & Data Flow:
1. User/browser or AI agent → HTTP POST /api/short-video or MCP SSE → Server
2. Server orchestration controller invokes Short Creator:
• Calls Kokoro TTS → returns audio stream
• Calls Whisper → returns caption timestamps
• Calls Pexels → returns background clip URLs
• Passes assets into Remotion scenes → renders frames
• Offloads to Chrome tabs (Remotion concurrency)
• Pipes result through FFmpeg (mux music + audio + video)
3. Final video persisted to disk → Server responds with videoId
4. Clients poll GET /status or subscribe to MCP messages
5. GET /api/short-video/{id} streams binary video
c. Architectural Patterns & Principles:
– Modular adapters for each external dependency (SOLID: single-responsibility)
– Clear separation between API layer, orchestration logic, and asset-processing libraries
– Concurrency control for Remotion (configurable)
– Dockerized builds targeting different resource profiles
d. Key Technologies & Frameworks:
– Node.js + TypeScript
– Express-style routing (server/routers/*.ts)
– Remotion (programmatic video composition)
– whisper.cpp (speech-to-text captions)
– kokoro-js (text-to-speech)
– FFmpeg (media manipulation)
– Pexels API (background videos)
– React + Vite + Tailwind CSS (UI)
– Docker & Docker Compose