# saekisui/binaural-voice architecture

> Architecture diagram of the GitHub repository saekisui/binaural-voice, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/saekisui/binaural-voice
- Repository: https://github.com/saekisui/binaural-voice

## Overview

This is a small command-line tool for turning mono speech into headphone-oriented binaural audio with KU100 near-field HRIRs and scripted or randomized movement. The main workflow is either WAV input to position scheduling and rendering, or tagged text through ElevenLabs timestamped speech to cue conversion and the same renderer. The README documents tag behavior, random choreography, and application integration; the shown source establishes the core calls. The HRIR asset is documented as converted measurement data, but its loading and use are visible in the renderer.

Main entry points: [`binaural_voice.py`](https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py) (Cue scheduling), [`speak.py`](https://github.com/saekisui/binaural-voice/blob/main/speak.py) (TTS orchestration), [`hrir/ku100_nearfield_circ360.npz`](https://github.com/saekisui/binaural-voice/blob/main/hrir/ku100_nearfield_circ360.npz) (KU100 HRIR data)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_speech["Speech input"]
  node_mono["Mono WAV"]
  node_tagged_text["Tagged script"]
  node_speak["TTS orchestration<br/>[speak.py]"]
  node_elevenlabs["ElevenLabs TTS"]
end

subgraph group_motion["Positioning"]
  node_cue_conversion["Timestamp cues<br/>[speak.py]"]
  node_cue_schedule["Cue scheduling<br/>[binaural_voice.py]"]
  node_speech_activity["Speech activity<br/>[binaural_voice.py]"]
  node_random_cues["Random choreography<br/>[binaural_voice.py]"]
  node_position_tracks["Position tracks<br/>[binaural_voice.py]"]
end

subgraph group_audio["Binaural rendering"]
  node_hrir_asset["KU100 HRIR data"]
  node_hrir_lookup["HRIR lookup<br/>[binaural_voice.py]"]
  node_convolution["Frame convolution<br/>[binaural_voice.py]"]
  node_wav_output["Stereo WAV"]
end

node_user(("User"))

node_user -->|"provides audio"| node_mono
node_user -->|"provides script"| node_tagged_text
node_tagged_text -->|"submits text"| node_speak
node_speak -->|"requests timestamped speech"| node_elevenlabs
node_elevenlabs -->|"returns audio and alignment"| node_cue_conversion
node_cue_conversion -->|"passes cues and WAV"| node_cue_schedule
node_mono -->|"reads WAV"| node_cue_schedule
node_cue_schedule -->|"derives voiced time"| node_speech_activity
node_cue_schedule -->|"chooses when cues absent"| node_random_cues
node_cue_schedule -->|"builds movement segments"| node_position_tracks
node_position_tracks -->|"provides frame positions"| node_convolution
node_convolution -->|"requests filters"| node_hrir_lookup
node_hrir_lookup -->|"loads measurements"| node_hrir_asset
node_convolution -->|"writes stereo audio"| node_wav_output

click node_speak "https://github.com/saekisui/binaural-voice/blob/main/speak.py"
click node_cue_conversion "https://github.com/saekisui/binaural-voice/blob/main/speak.py"
click node_cue_schedule "https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py"
click node_speech_activity "https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py"
click node_random_cues "https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py"
click node_position_tracks "https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py"
click node_hrir_asset "https://github.com/saekisui/binaural-voice/blob/main/hrir/ku100_nearfield_circ360.npz"
click node_hrir_lookup "https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py"
click node_convolution "https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_mono,node_tagged_text,node_speak,node_elevenlabs,node_user toneBlue
class node_cue_conversion,node_cue_schedule,node_speech_activity,node_random_cues,node_position_tracks toneAmber
class node_hrir_asset,node_hrir_lookup,node_convolution,node_wav_output toneMint
```

## Components

### Speech input

- **Mono WAV**
- **Tagged script**
- [TTS orchestration](https://github.com/saekisui/binaural-voice/blob/main/speak.py): `speak.py`
- **ElevenLabs TTS**

### Positioning

- [Timestamp cues](https://github.com/saekisui/binaural-voice/blob/main/speak.py): `speak.py`
- [Cue scheduling](https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py): `binaural_voice.py`
- [Speech activity](https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py): `binaural_voice.py`
- [Random choreography](https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py): `binaural_voice.py`
- [Position tracks](https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py): `binaural_voice.py`

### Binaural rendering

- [KU100 HRIR data](https://github.com/saekisui/binaural-voice/blob/main/hrir/ku100_nearfield_circ360.npz): `hrir/ku100_nearfield_circ360.npz`
- [HRIR lookup](https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py): `binaural_voice.py`
- [Frame convolution](https://github.com/saekisui/binaural-voice/blob/main/binaural_voice.py): `binaural_voice.py`
- **Stereo WAV**

### Other components

- **User**

## Connections

- User → Mono WAV: provides audio (evidence: `README.md`)
- User → Tagged script: provides script (evidence: `README.md`)
- Tagged script → TTS orchestration: submits text (evidence: `README.md`)
- TTS orchestration → ElevenLabs TTS: requests timestamped speech (evidence: `speak.py`)
- ElevenLabs TTS → Timestamp cues: returns audio and alignment (evidence: `speak.py`)
- Timestamp cues → Cue scheduling: passes cues and WAV (evidence: `speak.py`)
- Mono WAV → Cue scheduling: reads WAV (evidence: `binaural_voice.py`)
- Cue scheduling → Speech activity: derives voiced time (evidence: `binaural_voice.py`)
- Cue scheduling → Random choreography: chooses when cues absent (evidence: `binaural_voice.py`)
- Cue scheduling → Position tracks: builds movement segments (evidence: `binaural_voice.py`)
- Position tracks → Frame convolution: provides frame positions (evidence: `binaural_voice.py`)
- Frame convolution → HRIR lookup: requests filters (evidence: `binaural_voice.py`)
- HRIR lookup → KU100 HRIR data: loads measurements (evidence: `binaural_voice.py`)
- Frame convolution → Stereo WAV: writes stereo audio (evidence: `binaural_voice.py`)