How saekisui/binaural-voice works
This is a small command-line tool for turning mono speech into headphone-oriented binaural audio with KU100 near-field HRIRs and scripted or randomized movement. The main workflow is either WAV input to position scheduling and rendering, or tagged text through ElevenLabs timestamped speech to cue conversion and the same renderer. The README documents tag behavior, random choreography, and application integration; the shown source establishes the core calls. The HRIR asset is documented as converted measurement data, but its loading and use are visible in the renderer.
Main entry points: binaural_voice.py Cue scheduling · speak.py TTS orchestration · ku100_nearfield_circ360.npz KU100 HRIR data
Components 14
Speech input
- Mono WAV
- Tagged script
- TTS orchestration
speak.py - ElevenLabs TTS
Positioning
- Timestamp cues
speak.py - Cue scheduling
binaural_voice.py - Speech activity
binaural_voice.py - Random choreography
binaural_voice.py - Position tracks
binaural_voice.py
Binaural rendering
- KU100 HRIR data
hrir/ku100_nearfield_circ360.npz - HRIR lookup
binaural_voice.py - Frame convolution
binaural_voice.py - Stereo WAV
Other components
- User
Connections 14
- User → Mono WAV: provides audio
README.md - User → Tagged script: provides script
README.md - Tagged script → TTS orchestration: submits text
README.md - TTS orchestration → ElevenLabs TTS: requests timestamped speech
speak.py - ElevenLabs TTS → Timestamp cues: returns audio and alignment
speak.py - Timestamp cues → Cue scheduling: passes cues and WAV
speak.py - Mono WAV → Cue scheduling: reads WAV
binaural_voice.py - Cue scheduling → Speech activity: derives voiced time
binaural_voice.py - Cue scheduling → Random choreography: chooses when cues absent
binaural_voice.py - Cue scheduling → Position tracks: builds movement segments
binaural_voice.py - Position tracks → Frame convolution: provides frame positions
binaural_voice.py - Frame convolution → HRIR lookup: requests filters
binaural_voice.py - HRIR lookup → KU100 HRIR data: loads measurements
binaural_voice.py - Frame convolution → Stereo WAV: writes stereo audio
binaural_voice.py