How isruihu/slideasr-bench works
VAPO is a GPU-parallel benchmark runner for slide-enhanced speech recognition. Its primary workflow loads SlideASR examples, selects an audio or audio-visual model, transcribes each item, and appends per-item results. Model implementations include direct multimodal slide input, OCR-derived slide context, and VAPO’s visually anchored <think>/<answer> prompting. A separate evaluation workflow prepares text and entity matches, calculates WER and entity metrics, then aggregates results. The graph reflects sampled runtime code; README-documented benchmark/model options are included, but unsampled evaluation wiring is not inferred.
Main entry points: run.py GPU runner · base_model.py Audio inference base · prepare_asr_result_for_metrics.py Normalize and match entities · slide_asr_evaluation Prepared metric data
Components 22
Benchmark execution
- GPU runner
run.py - Model and dataset registry
config.py - Worker queues
run.py
Recognition models
- Audio inference base
model/base_model.py - Audio-visual base
model/base_model.py - OCR context
model/base_model.py - Qwen2.5 Omni
model/qwen2_5_omni.py - VAPO prompting
model/qwen2_5_omni.py - Qwen3 Omni
model/qwen3_omni.py - Qwen2 Audio
model/qwen2_audio.py - MiDasheng
model/midashenglm.py
Metric evaluation
- Normalize and match entities
evaluation/slide_asr_evaluation/prepare_asr_result_for_metrics.py - WER and entity metrics
evaluation/slide_asr_evaluation/calculate_metrics.py - Text and metric utilities
evaluation/slide_asr_evaluation/utils.py - Metric aggregation
evaluation/slide_asr_evaluation/view_metric_result.py
Benchmark data
- SlideASR datasets
dataset/slide_asr_bench.py - SlideASR files
resource - Inference JSONL
run.py - Item processing
dataset/slide_asr_bench.py - Prepared metric data
evaluation/slide_asr_evaluation
Other components
- Benchmark operator
- PaddleOCR models
Connections 29
- Benchmark operator → GPU runner: starts run
- GPU runner → Model and dataset registry: selects entries
- GPU runner → SlideASR datasets: loads samples
- SlideASR datasets → SlideASR files: reads examples
- SlideASR datasets → Inference JSONL: skips completed IDs
- GPU runner → Worker queues: dispatches tasks
- GPU runner → Model and dataset registry: starts workers
- GPU runner → Worker queues: collects results
- GPU runner → Inference JSONL: appends results
- Worker queues → Item processing: delivers items
- Item processing → Audio inference base: runs audio inference
- Item processing → Audio-visual base: runs visual inference
- Audio inference base → OCR context: gets slide text
- OCR context → PaddleOCR models: calls OCR
- Audio inference base → Qwen2.5 Omni: dispatches atom
- Audio-visual base → Qwen2.5 Omni: dispatches atom
- Audio inference base → Qwen3 Omni: dispatches atom
- Audio-visual base → Qwen3 Omni: dispatches atom
- Audio inference base → Qwen2 Audio: dispatches atom
- Audio inference base → MiDasheng: dispatches atom
- VAPO prompting → Qwen2.5 Omni: uses generation
- Benchmark operator → Normalize and match entities: evaluates results
- Normalize and match entities → Inference JSONL: reads transcripts
- Normalize and match entities → Text and metric utilities: normalizes and matches
- Normalize and match entities → Prepared metric data: writes prepared data
- WER and entity metrics → Prepared metric data: reads prepared data
- WER and entity metrics → Text and metric utilities: calculates WER
- WER and entity metrics → Prepared metric data: writes metric data
- Metric aggregation → Prepared metric data: reads metric data