How rammeshgar/digital-twin-demo works
Sadeq’s Interactive 3D Digital Twin is a browser-based portfolio guide. Visitors ask questions by text or voice; the browser can recognize speech or send a recording for transcription, then requests a grounded AI response. The interface presents the answer as chat and speech, coordinating lip movement and blended gestures while rendering the character. The graph preserves the documented animation, transcription, responsive, and fallback capabilities. Some links rely on README documentation because the corresponding browser request-handling code was not included in the excerpts.
Main entry points: mascot.js Experience controller · chat.js Chat service · index.html Chat interface · movement.mjs Gesture blending
Components 22
Browser experience
- Chat interface
index.html - Experience controller
mascot.js - Speech recognition
mascot.js - Audio capture
mascot.js - Chat transcript
mascot.js - Speech playback
mascot.js - Responsive styling
style.css
Character animation
- Viseme lip sync
mascot.js - 3D rendering
mascot.js - Character refinement
character-refinement.mjs - Digital twin model
assets/sadeq-digital-twin.meshopt.glb - Gesture blending
movement.mjs - Gesture library
gestures.json - Idle presence
presence.mjs
Serverless services
- Audio transcription
netlify/functions/transcribe.js - Chat service
netlify/functions/chat.js
Grounded knowledge
- Knowledge context
netlify/functions/chat.js - Persona guidance
netlify/functions/persona.md - Professional source
netlify/functions/sadeq-source-of-truth.md
Other components
- Visitor
- Gemini
- ElevenLabs
Connections 24
- Visitor → Chat interface: asks
README.md - Chat interface → Experience controller: controls
mascot.js - Experience controller → Speech recognition: starts recognition
README.md - Experience controller → Audio capture: records audio
README.md - Audio capture → Audio transcription: submits recording
README.md - Speech recognition → Experience controller: provides text
README.md - Audio transcription → Experience controller: returns transcript
README.md - Experience controller → Chat service: sends question
README.md - Chat service → Knowledge context: builds context
netlify/functions/chat.js - Knowledge context → Persona guidance: reads
netlify/functions/chat.js - Knowledge context → Professional source: reads
netlify/functions/chat.js - Chat service → Gemini: requests response
netlify/functions/chat.js - Chat service → ElevenLabs: requests speech
netlify/functions/chat.js - Chat service → Experience controller: returns answer
README.md - Experience controller → Chat transcript: updates transcript
README.md - Experience controller → Speech playback: plays response
mascot.js - Speech playback → Viseme lip sync: drives mouth
README.md - Experience controller → 3D rendering: updates scene
mascot.js - 3D rendering → Digital twin model: loads
mascot.js - 3D rendering → Character refinement: refines model
mascot.js - Experience controller → Gesture blending: creates movement
mascot.js - Gesture blending → Gesture library: loads clips
movement.mjs - Gesture blending → Idle presence: creates presence
movement.mjs - Chat interface → Responsive styling: uses styles