How jobicari/noisy-speech-asr-benchmarking-pipeline works
Noisy-Speech ASR Benchmark measures recognition accuracy across clean and synthetically damaged LibriSpeech audio. The main run loads speech and an impulse response, prepares seeded corruption variants and audio-health metrics in CPU workers, batches them through configured ASR models, then evaluates transcripts and checkpoints results. Configuration and run metadata support reproducibility; analysis and visualization present benchmark findings. The README documents cloud execution and result upload, but sampled source does not show that orchestration, so those links are omitted.
Main entry points: app.py Run entry point · models.py Transcriber engines · prep.py Worker preparation · storage.py Checkpoint storage
Components 18
Benchmark run
- Run entry point
app.py - Run configuration
src/asr_pipeline_benchmark/configuration.py - Speech dataset
src/asr_pipeline_benchmark/data.py - Run metadata and batching
src/asr_pipeline_benchmark/utility.py
Audio preparation
- Impulse dataset
src/asr_pipeline_benchmark/data.py - Corruptions and cleanup
src/asr_pipeline_benchmark/preprocessing.py - Worker preparation
src/asr_pipeline_benchmark/prep.py - Audio health metrics
src/asr_pipeline_benchmark/audio_analysis.py
Recognition and metrics
- Transcriber engines
src/asr_pipeline_benchmark/models.py - Error-rate evaluation
src/asr_pipeline_benchmark/evaluation.py
Results and analysis
- Checkpoint storage
src/asr_pipeline_benchmark/storage.py - Benchmark analysis
analysis/v2 - Feature visualization
src/asr_pipeline_benchmark/visualization.py
Other components
- Benchmark operator
- Hugging Face datasets
- ASR model runtimes
- Local files
- Google Cloud Storage
Connections 22
- Benchmark operator → Run entry point: starts run
README.md - Run entry point → Run configuration: reads settings
app.py - Run entry point → Speech dataset: loads speech
app.py - Run entry point → Corruptions and cleanup: loads impulse response
app.py - Speech dataset → Hugging Face datasets: streams dataset
src/asr_pipeline_benchmark/data.py - Impulse dataset → Hugging Face datasets: loads dataset
src/asr_pipeline_benchmark/data.py - Run entry point → Worker preparation: initializes workers
app.py - Run entry point → Transcriber engines: runs checkpoint loop
app.py - Run entry point → Checkpoint storage: selects backend
app.py - Run entry point → Run metadata and batching: records run
app.py - Worker preparation → Corruptions and cleanup: applies corruptions
src/asr_pipeline_benchmark/prep.py - Worker preparation → Audio health metrics: analyzes audio
src/asr_pipeline_benchmark/prep.py - Worker preparation → Run metadata and batching: seeds and times
src/asr_pipeline_benchmark/prep.py - Transcriber engines → Worker preparation: dispatches preparation
src/asr_pipeline_benchmark/models.py - Transcriber engines → ASR model runtimes: transcribes batches
README.md - Transcriber engines → Checkpoint storage: checkpoints results
src/asr_pipeline_benchmark/models.py - Transcriber engines → Run metadata and batching: batches samples
src/asr_pipeline_benchmark/models.py - Error-rate evaluation → Transcriber engines: uses transcripts
src/asr_pipeline_benchmark/evaluation.py - Checkpoint storage → Local files: reads and writes
src/asr_pipeline_benchmark/storage.py - Checkpoint storage → Google Cloud Storage: reads and writes
src/asr_pipeline_benchmark/storage.py - Benchmark analysis → Checkpoint storage: analyzes run results
- Feature visualization → Benchmark analysis: plots features