How tejas0019/video-caption-generator works
1. What this repository is
- This is a small, single-purpose video captioning app.
- It has two main parts: a browser-based UI in frontend/ and a Python backend in backend/ that does the actual media processing.
2. Primary request flow
- A user uploads a video or audio file in the frontend.
- The frontend sends the file plus caption/style options to the backend API.
- The backend transcribes the media, generates subtitle files, optionally burns subtitles into the video, and returns the result for download.
3. Frontend boundary
- frontend/index.html is a static client, not a separate SPA framework.
- Its job is limited to collecting input, showing progress, and calling backend endpoints.
- The README suggests the frontend can be opened directly or served from the same backend on http://localhost:8000/.
4. Backend application boundary
- backend/main.py is the server entrypoint and likely hosts the FastAPI app.
- This is the central orchestration layer: it receives uploads, coordinates transcription and subtitle generation, and manages output files.
- The README confirms FastAPI is the backend runtime and exposes auto-generated docs at /docs.
5. Transcription subsystem
- backend/whisper_service.py is the AI transcription layer.
- It abstracts how speech-to-text is produced, with the README indicating support for OpenAI Whisper API and an optional local Whisper model.
- Environment variables such as OPENAI_API_KEY, WHISPER_LANGUAGE, and WHISPER_MODEL_PATH control this behavior.
6. Subtitle generation subsystem
- backend/subtitle_service.py turns transcript data into subtitle artifacts.
- It supports at least two output modes described in the README:
- Standard mode: classic SRT subtitles.
- Kinetic mode: centered, word-synced subtitles intended for social video.
- This is the formatting boundary between raw transcription data and user-facing caption assets.
7. Media processing subsystem
- backend/utils.py contains FFmpeg-related helpers and likely handles video/audio extraction, subtitle burning, and file manipulation.
- FFmpeg is a hard dependency of the architecture, because it is the component that actually modifies media files.
- The presence of backend/ffmpeg.exe suggests a Windows-friendly local binary option in addition to system-installed FFmpeg.
8. Storage and intermediate artifacts
- Processed files live under backend/uploads/ according to the README.
- This folder is the working area for uploaded media, temporary transcodes, subtitle files, and final rendered outputs.
- The architecture appears file-based rather than database-driven.
9. Configuration and runtime inputs
- The app is driven mostly by environment variables in backend/.env.
- Key configuration includes the OpenAI API key, Whisper language selection, and whether to use a local model.
- This means deployment requires secret management plus a working FFmpeg runtime.
10. Deployment and packaging
- Dockerfile indicates the backend can be containerized as a self-contained service.
- vercel.json suggests there is also deployment configuration for a Vercel-style host, likely for static/frontend or lightweight server deployment.
- The root and backend requirements.txt files define Python dependencies for the server side.
11. End-to-end architecture in one sentence
- Browser UI → FastAPI backend → transcription service (OpenAI/local Whisper) → subtitle formatter → FFmpeg media processor → downloadable captioned video and/or subtitle file.
12. What matters most for an architecture diagram
- Show the browser and backend as separate boundaries.
- Show Whisper/OpenAI as an external service boundary.
- Show FFmpeg as a local processing dependency inside the backend runtime.
- Show uploads/ as the working storage area connecting upload, processing, and download stages.
Main entry points: main.py API Server · whisper_service.py Transcription · utils.py Media Tools · index.html Browser UI
Components 15
Frontend
Static browser client for upload, options, progress, and download.
- Browser UI · Single-page browser interface for choosing media and caption options, then calling the backend.
frontend/index.html
Backend
FastAPI app plus transcription, subtitle, and media-processing services.
- API Server · Server entrypoint and orchestration layer for uploads, processing, and downloads.
backend/main.py - Transcription · Speech-to-text abstraction supporting OpenAI Whisper API and optional local model use.
backend/whisper_service.py - Captions · Turns transcript data into SRT and kinetic subtitle outputs.
backend/subtitle_service.py - Media Tools · FFmpeg-oriented utilities for extraction, burning subtitles, and file manipulation.
backend/utils.py - Working Files · File-based workspace for uploaded media, intermediate files, subtitles, and rendered outputs.
- Config · Environment-driven settings such as API keys, language, and local model path.
backend/.env.example - FFmpeg Binary · Bundled Windows-friendly FFmpeg binary used when a system install is unavailable.
backend/ffmpeg.exe - Backend Deps · Backend-specific Python dependency specification.
backend/requirements.txt
External
Outside services and runtimes the app depends on.
- Whisper API · Remote transcription service used when the app is configured for hosted Whisper.
- FFmpeg · External media-processing engine used to transcode and burn subtitles.
Other components
- Docker Build · Container image definition for running the backend as a self-contained service.
Dockerfile - Deploy Config · Deployment configuration for Vercel-style hosting.
vercel.json - Python Deps · Top-level Python dependency specification.
requirements.txt - Docs · Repository documentation describing setup, runtime, and processing modes.
README.md
Connections 16
- Browser UI → API Server: uploads/options · The browser client sends media and caption settings to the API.
- API Server → Transcription: transcribe · The server delegates speech-to-text generation to the transcription layer.
- API Server → Captions: format captions · Transcript results are converted into subtitle artifacts.
- API Server → Media Tools: process media · The server uses media helpers for extraction, transcoding, and burning.
- API Server → Working Files: read/write · Uploads and generated outputs are stored in the working files area.
- API Server → Config: load config · Runtime settings are read from environment variables.
- API Server → FFmpeg Binary: invoke · The server can use the bundled FFmpeg executable during processing.
- Transcription → Whisper API: API mode · Hosted Whisper is the remote transcription path.
- Transcription → Config: model settings · Language and model-path settings influence transcription behavior.
- Media Tools → FFmpeg: executes · Media helpers depend on FFmpeg for core file processing.
- Media Tools → Working Files: temp files · Processing steps operate on uploaded and intermediate artifacts.
- Captions → Working Files: emit subtitles · Generated SRT or kinetic subtitle files are written to the workspace.
- Browser UI → Docs: usage cues · The docs explain how the static client and backend are meant to be used.
- Docker Build → API Server: packages app · Containerization targets the backend service.
- Deploy Config → Browser UI: deploys · Hosting config likely serves the static client or lightweight app entry.
- Python Deps → Backend Deps: python deps · Both manifests describe the backend runtime dependencies.