How wagefolabessy/ai-agent-animation-studio works
AI-Agent Animation Studio turns audio and timestamped lyrics into deterministic, procedurally directed 2.5D music videos. The main workflow analyzes audio, develops a story specification, compiles it into staged shots and choreography, builds frame state, composites SVG frames, then rasterizes and streams video for encoding and audio muxing. Separate capabilities include genre-aware staging, reusable character and environment generation, chunked rendering with checkpoint/resume, and schema/invariant validation. The graph uses README statements and SOURCE INDEX references; unsampled subsystems are represented only at the directory level where supported.
Main entry points: production-pipeline.ts Production pipeline · orchestrator.ts Studio orchestration · audio Audio analysis · story-spec-agent.ts Story specification
Components 20
Audio Intelligence
- Audio analysis
src/engine/audio - Lyric parsing
src/engine/audio/lrc-parser.ts - Python audio analysis
python/analyze_audio.py
Story and Direction
- Story specification
src/agents/story-spec-agent.ts - LLM providers
src/agents/llm-provider.ts - Studio orchestration
src/agents/orchestrator.ts - Verb compiler
src/engine/director/verb-engine/verb-compiler.ts - Shot and choreography
src/engine/director
Procedural Scene Engine
- Cast and environments
src/engine/library - Character rigging
src/engine/rigging - 3D camera
src/engine/camera/camera-3d.ts - Scene compositor
src/engine/compositor/scene-compositor.ts
Rendering and Output
- Production pipeline
src/engine/director/production-pipeline.ts - Chunk rendering and resume
src/engine/director - Raster and video pipeline
src/renderer/pipeline.ts - FFmpeg streaming
src/renderer/ffmpeg-streamer.ts
Contracts and Validation
- Data contracts
src/schemas - Invariant validation
src/schemas/invariants.ts
Other components
- Audio and lyrics
- Rendered music video
Connections 22
- Audio and lyrics → Audio analysis: provides audio
README.md - Audio and lyrics → Lyric parsing: provides lyrics
README.md - Audio analysis → Story specification: informs story
src/agents/story-spec-agent.ts - Lyric parsing → Story specification: profiles lyrics
src/agents/story-spec-agent.ts - Story specification → LLM providers: requests drafts
src/agents/story-spec-agent.ts - Studio orchestration → Story specification: runs agent
src/agents/orchestrator.ts - Studio orchestration → Verb compiler: compiles story
src/agents/orchestrator.ts - Verb compiler → Cast and environments: uses archetypes
src/engine/director/verb-engine/verb-compiler.ts - Verb compiler → Shot and choreography: builds choreography
src/engine/director/verb-engine/verb-compiler.ts - Shot and choreography → 3D camera: evaluates shots
src/engine/director/shot-sequencer.ts - Cast and environments → Character rigging: generates motion
src/engine/library/archetype-factory.ts - Production pipeline → Studio orchestration: runs orchestration
src/engine/director/production-pipeline.ts - Production pipeline → Audio analysis: loads analysis
src/engine/director/production-pipeline.ts - Production pipeline → Chunk rendering and resume: renders chunks
src/engine/director/production-pipeline.ts - Production pipeline → Scene compositor: builds frames
src/engine/director/production-pipeline.ts - Scene compositor → 3D camera: projects scene
src/engine/compositor/scene-compositor.ts - Scene compositor → Raster and video pipeline: emits SVG frames
README.md - Raster and video pipeline → FFmpeg streaming: streams frames
src/renderer/pipeline.ts - FFmpeg streaming → Rendered music video: encodes and muxes
README.md - Studio orchestration → Invariant validation: validates output
src/agents/orchestrator.ts - Invariant validation → Data contracts: checks contracts
src/schemas/invariants.ts - Python audio analysis → Audio analysis: alternate analysis
README.md