How mayankpratapsingh022/turn-detection-model-whisper-tiny-dual-scale-attention-classification-head-for-low-latency-voice-ai works
Turn Detection Model is an audio-only endpoint classifier and its supporting data, training, evaluation, and release workflow. In the user path, a candidate pause’s audio is scored by the deployed model and policy as COMPLETE or HOLD; streaming continues listening on HOLD until speech resumes or timeout. The README documents preparation, causal-pause examples, hard-negative mining, calibration, evaluation, and ONNX export. Some subsystem links are based on documented capabilities and tree paths; unsampled internal wiring is intentionally not asserted.
Main entry points: inference.py Turn detector · prepare.py Audio preparation · evaluator.py Evaluation · demo.py Inference demo
Components 24
Inference and demo
- Inference demo
src/turn_detector/demo.py - Turn detector
src/turn_detector/inference.py - Audio handling
src/turn_detector/audio.py - Feature extraction
src/turn_detector/features.py - ONNX model
- COMPLETE / HOLD
src/turn_detector/types.py
Data preparation
- Audio preparation
src/turn_detector/data/prepare.py - Audio records
src/turn_detector/data/records.py - Duplicate grouping
src/turn_detector/data/duplicates.py - Dataset splits
src/turn_detector/data/splits.py - Speech mix tags
src/turn_detector/data/hinglish.py
Training and model
- Hard negatives
src/turn_detector/training/hard_negatives.py - Training dataset
src/turn_detector/training/dataset.py - Trainer
src/turn_detector/training/trainer.py - Model architecture
src/turn_detector/modeling.py - Model checkpoint
Evaluation and release
- Evaluation
src/turn_detector/evaluation/evaluator.py - Metrics and policy
src/turn_detector/evaluation/metrics.py - Baseline comparison
src/turn_detector/evaluation/baselines.py - ONNX export
src/turn_detector/export.py - Model publishing
src/turn_detector/publishing.py
Other components
- Audio user
- Source dataset
- Hugging Face Hub
Connections 27
- Audio user → Inference demo: submits audio
- Inference demo → Audio handling: loads audio
- Inference demo → Turn detector: requests score
- Turn detector → Audio handling: standardizes audio
- Turn detector → Feature extraction: extracts features
- Turn detector → ONNX model: runs inference
- Turn detector → COMPLETE / HOLD: returns prediction
- Source dataset → Audio preparation: provides rows
- Audio preparation → Audio records: writes manifests
- Audio preparation → Duplicate grouping: groups duplicates
- Audio preparation → Dataset splits: assigns splits
- Audio preparation → Speech mix tags: tags speech mix
- Audio records → Training dataset: loads examples
- Hard negatives → Training dataset: adds mined examples
- Training dataset → Trainer: supplies batches
- Trainer → Model architecture: trains model
- Trainer → Model checkpoint: saves checkpoint
- Model checkpoint → Hard negatives: scores examples
- Model checkpoint → ONNX export: exports weights
- ONNX export → ONNX model: writes ONNX
- ONNX model → Evaluation: is evaluated
- Evaluation → Metrics and policy: computes policy metrics
- Evaluation → Baseline comparison: compares predictions
- Evaluation → Model publishing: provides report
- ONNX export → Model publishing: provides export
- Model publishing → Hugging Face Hub: publishes artifacts
- Hugging Face Hub → Turn detector: downloads model