How soundsgoodai/fast-gpu-asr works
Fast GPU ASR is a Python toolkit for exporting supported Zipformer and Parakeet checkpoints into TensorRT bundles, then running batched offline GPU speech recognition through a shared API. The runtime prepares audio, encodes it, decodes tokens with CTC or transducer/TDT search, and returns text with word timestamps. Exported model implementations and native TensorRT plugins support the two model families. The excerpts show the runtime entry point and selected components; some export and postprocessing connections are based only on documented behavior or remain unverified.
Main entry points: asr.py ASR pipeline · export_utils.py Engine export utilities · export_zipformer.py Zipformer exporter · export_parakeet.py Parakeet exporter
Components 18
Model export
- Zipformer exporter
src/fast_gpu_asr/export/export_zipformer.py - Parakeet exporter
src/fast_gpu_asr/export/export_parakeet.py - Engine export utilities
src/fast_gpu_asr/export/export_utils.py
Model architectures
- Zipformer model
src/fast_gpu_asr/export/model/zipformer - Parakeet model
src/fast_gpu_asr/export/model/parakeet
Inference runtime
- Python API
src/fast_gpu_asr/__init__.py - ASR pipeline
src/fast_gpu_asr/asr.py - Model bundle validation
src/fast_gpu_asr/utils.py - Audio encoder
src/fast_gpu_asr/encoder/encoder.py - Text postprocessing
src/fast_gpu_asr/decoder/postprocessor.py
GPU decoding
- Zipformer transducer
src/fast_gpu_asr/decoder/zipformer_rnnt_decoder.py - Parakeet TDT search
src/fast_gpu_asr/decoder/parakeet_tdt_decoder.py - CTC greedy search
src/fast_gpu_asr/decoder/ctc_decoder.py - CUDA search kernels
src/fast_gpu_asr/decoder/gpu_kernels.py
TensorRT acceleration
- Native TensorRT plugins
src/fast_gpu_asr/tensorrt_plugins/__init__.py - TensorRT runtime
Other components
- ASR user
- Text and timestamps
Connections 23
- ASR user → Python API: calls
README.md - Python API → ASR pipeline: exposes
src/fast_gpu_asr/__init__.py - ASR user → Zipformer exporter: exports checkpoint
README.md - ASR user → Parakeet exporter: exports checkpoint
README.md - Zipformer exporter → Zipformer model: builds from
src/fast_gpu_asr/export/export_zipformer.py - Parakeet exporter → Parakeet model: builds from
src/fast_gpu_asr/export/export_parakeet.py - Zipformer exporter → Engine export utilities: uses
src/fast_gpu_asr/export/export_zipformer.py - Parakeet exporter → Engine export utilities: uses
src/fast_gpu_asr/export/export_parakeet.py - Engine export utilities → Native TensorRT plugins: loads plugins
src/fast_gpu_asr/export/export_utils.py - Engine export utilities → TensorRT runtime: builds engines
README.md - ASR pipeline → Model bundle validation: validates bundle
src/fast_gpu_asr/asr.py - ASR pipeline → Audio encoder: runs audio
src/fast_gpu_asr/asr.py - ASR pipeline → Zipformer transducer: dispatches
src/fast_gpu_asr/asr.py - ASR pipeline → Parakeet TDT search: dispatches
src/fast_gpu_asr/asr.py - ASR pipeline → CTC greedy search: dispatches
README.md - Zipformer transducer → CUDA search kernels: launches
src/fast_gpu_asr/decoder/zipformer_rnnt_decoder.py - Parakeet TDT search → CUDA search kernels: launches
src/fast_gpu_asr/decoder/parakeet_tdt_decoder.py - CTC greedy search → CUDA search kernels: launches
src/fast_gpu_asr/decoder/ctc_decoder.py - ASR pipeline → Text postprocessing: postprocesses
src/fast_gpu_asr/asr.py - Audio encoder → TensorRT runtime: executes engine
src/fast_gpu_asr/encoder/encoder.py - Zipformer transducer → TensorRT runtime: executes engine
src/fast_gpu_asr/decoder/zipformer_rnnt_decoder.py - Parakeet TDT search → TensorRT runtime: executes engine
src/fast_gpu_asr/decoder/parakeet_tdt_decoder.py - ASR pipeline → Text and timestamps: returns transcript
README.md