How smartclipai/faceless-video-api works
Project Type and Purpose
• This is a backend microservice (FastAPI) focused on AI-driven faceless video creation.
• It exposes REST endpoints for authentication, story generation, image creation, and video production.
• It uses asynchronous background processing, a PostgreSQL database, and integrates with external AI/TTS/storage services.
File Structure
• app/main.py – FastAPI application entrypoint.
• app/api/endpoints/ – HTTP router layer (auth.py, image.py, video.py).
• app/core/ – configuration, logging, security (JWT).
• app/db/ – SQLAlchemy setup, session management, DB initialization script.
• app/models, app/schemas – ORM models and Pydantic schemas.
• app/services/ – business logic for story, image, audio, video generation, storage, and background task processing.
• app/utils, app/resources – helper functions, fonts, transitions.
• config.json – feature toggles and AI/TTS parameters.
• scripts/run_init_db.py – database bootstrap.
• requirements.txt, .env.example – dependencies and environment variables.
README for Insights
• RESTful API with token auth (OAuth2PasswordBearer).
• Async background tasks for long-running video builds.
• Uses OpenAI (or Azure) for story, FAL or Replicate for image AI, TTS voices for narration, and Cloudflare R2 for storage.
• PostgreSQL for persistence; uvicorn as the server.
Main Components
A. API Layer
• Uvicorn + FastAPI (app/main.py)
• Endpoint Routers (/v1/auth, /v1/image, /v1/video)
• Security middleware (JWT auth)
B. Service Layer
• StoryGenerator (calls OpenAI)
• ImageGenerator / ImageAPI (calls FAL/Replicate)
• AudioGenerator (TTS engine)
• VideoGenerator (stitch images+audio, add subtitles)
• VideoTaskProcessor (async job orchestration)
• StorageService (uploads to Cloudflare R2)
C. Data Layer
• PostgreSQL (user, image, video_task tables) via SQLAlchemy
• Pydantic schemas for request/response validation
D. External Dependencies
• OpenAI / Azure OpenAI endpoint
• FAL API or Replicate API
• Text-to-Speech engine or API
• Cloudflare R2 storage
E. Configuration
• config.json feature toggles
• .env (API keys, DB URL, JWT secret)
Relationships and Data Flows
1. Client → FastAPI Endpoints (HTTP POST)
2. Auth endpoint ↔ PostgreSQL (validate credentials, issue JWT)
3. Video/Image endpoints → VideoTaskProcessor (enqueue background job)
4. VideoTaskProcessor orchestrates:
a. StoryGenerator → OpenAI → returns text scenes
b. For each scene: ImageGenerator → FAL/Replicate → returns images
c. AudioGenerator → TTS API → returns audio files
d. VideoGenerator → merges images+audio → produces final video file
5. StorageService → Cloudflare R2 → returns public URL
6. VideoTaskProcessor updates PostgreSQL with progress & result URL
7. Client polls status endpoint → reads from PostgreSQL → retrieves final URL
Architectural Patterns & Principles
• Layered architecture: Router → Service → Data
• Dependency injection using FastAPI’s Depends
• Asynchronous background processing (non-blocking API)
• Externalized configuration (config.json + env)
• Separation of concerns: authentication, business logic, persistence, external integration
Essential Elements
• Component names & boundaries
• Annotations for key technologies (FastAPI, SQLAlchemy, Pydantic, Uvicorn, PostgreSQL, Cloudflare R2, OpenAI, FAL/Replicate, TTS)