How mblanc/storycraft works
Project Type and Purpose
• StoryCraft is a full-stack web application built with Next.js (App Router) that provides an AI-powered video storyboard generation workflow.
• It transforms user-entered text into cinematic scenes, voiceovers, music, and final video exports by orchestrating Google’s Vertex AI models (Imagen 4.0, Veo 3.0, Chirp 3, Lyria 2), FFmpeg processing, and Google Cloud services.
File Structure Highlights
• app/ – Next.js App Router:
• app/actions/ – Server actions for AI generation, media processing, GCS interactions
• app/api/ – REST endpoints for scenarios, scenes, users, videos
• app/components/ – React UI components organized by feature (create, editor, storyboard, video, UI primitives)
• app/client-layout.tsx, layout.tsx, page.tsx – Top-level layouts and pages
• lib/ – Core utilities and service wrappers: FFmpeg logic, Firestore client, Vertex AI model calls, Cloud Storage, TTS
• public/ – Static assets: sample videos, audio tracks, generated TTS files, style references
• scripts/ & terraform/ – Deployment automation: container build & deploy, Terraform infra for Cloud Run & associated resources
• Configuration – Dockerfile, next.config.mjs, tailwind.config.ts, ESLint, CORS, PostCSS, TSConfig
Key Architectural Components
a. Frontend Layer (Next.js + React)
• Pages & layouts serve UI; TanStack Query manages server state & caching
• Components: timeline editor (audio waveform via Web Audio API), creation wizards, story scenario manager
• UI libraries: Radix UI primitives, Framer Motion animations, Tailwind CSS styling
b. API & Server Actions Layer
• API Routes (app/api/*/route.ts) for CRUD on scenarios, scenes, users, videos
• Server Actions (app/actions/*) triggered from client for AI calls, upload/download, media processing
c. Business Logic & Processing
• lib/ffmpeg.ts orchestrates video concatenation, audio mixing, overlays with fluent-ffmpeg
• Prompt utilities and retry logic in lib/prompt-utils.ts for robust Vertex AI calls
• Storage signed URLs in lib/storage.ts for secure GCS access
d. Data & Storage
• Firestore (lib/firestore.ts) holds metadata: scenario definitions, scene settings, user profiles, video records
• Google Cloud Storage (via signed URLs) stores raw/generated images, audio, video assets
e. External AI Services
• Imagen (image generation), Veo (video generation), Chirp (voice synthesis), Lyria (music composition)
• Google Cloud Text-to-Speech for neural voiceovers
f. Infrastructure & Deployment
• Docker container for Cloud Run
• Terraform scripts to provision Cloud Run service, Firestore, GCS buckets, IAM roles
Relationships and Data Flows
• Client UI → API Routes / Server Actions → Business Logic → External AI Services & FFmpeg → GCS (store assets) → Firestore (save metadata) → Client UI
• User request → Next.js frontend → server
• Server calls to Vertex AI / TTS → asset outputs
• FFmpeg processing → final video
• Asset storage in GCS & metadata in Firestore → retrieval by frontend
Architectural Patterns and Design Principles
• Next.js App Router with server actions for collocating UI and server code
• Separation of Concerns: UI components vs. API routes vs. processing logic in lib/
• Event-driven Workflow: each AI generation step as a discrete server action
• Retry with Exponential Backoff for stability on AI calls
• Infrastructure as Code: Terraform for reproducible Cloud Run & storage provisioning