How met4citizen/talkinghead works
Project type and purpose
• This is primarily a front-end JavaScript library and demo application (“TalkingHead”) for real-time 3D avatars that speak and lip-sync in the browser.
• It provides a core ES-module (modules/talkinghead.mjs) plus auxiliary modules for lip-sync, dynamic bones, and audio playback. It loads 3D models (GLB), animations (FBX), audio assets and integrates with external TTS/AI services via HTTP/WebSocket proxies.
File structure
Top-level components:
• index.html + siteconfig.js – the test/demo web app UI and configuration
• modules/ – ES modules: core TalkingHead class, lipsync-xx, dynamicbones, playback worklet
• examples/ – standalone HTML demos showing different integrations (Azure, MP3, minimal)
• avatars/, animations/, poses/, audio/, images/, fonts/ – static assets (models, FBX, audio, textures, fonts)
• README.md – detailed project overview, API reference, usage patterns
• siteconfig.js – demo app resource URLs, API proxy endpoints
Insights from README
• Core features: real-time TTS + word-level timestamps for lip-sync in multiple languages, dynamic bones physics, external TTS/A.I. service integrations
• Architecture: module-based, dynamic imports via importmap, uses ThreeJS/WebGL for rendering, Web Audio API + AudioWorklet for low-latency audio
• Extensibility: pluggable lipsync modules, dynamic bones config, custom poses/gestures, streaming interface