# soundsgoodai/fast-gpu-asr architecture

> Architecture diagram of the GitHub repository soundsgoodai/fast-gpu-asr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/soundsgoodai/fast-gpu-asr
- Repository: https://github.com/soundsgoodai/fast-gpu-asr

## Overview

Fast GPU ASR is a Python toolkit for exporting supported Zipformer and Parakeet checkpoints into TensorRT bundles, then running batched offline GPU speech recognition through a shared API. The runtime prepares audio, encodes it, decodes tokens with CTC or transducer/TDT search, and returns text with word timestamps. Exported model implementations and native TensorRT plugins support the two model families. The excerpts show the runtime entry point and selected components; some export and postprocessing connections are based only on documented behavior or remain unverified.

Main entry points: [`src/fast_gpu_asr/asr.py`](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/asr.py) (ASR pipeline), [`src/fast_gpu_asr/export/export_utils.py`](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/export/export_utils.py) (Engine export utilities), [`src/fast_gpu_asr/export/export_zipformer.py`](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/export/export_zipformer.py) (Zipformer exporter), [`src/fast_gpu_asr/export/export_parakeet.py`](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/export/export_parakeet.py) (Parakeet exporter)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_export["Model export"]
  node_zipexport["Zipformer exporter"]
  node_paraexport["Parakeet exporter<br/>[export_parakeet.py]"]
  node_exportutils["Engine export utilities<br/>[export_utils.py]"]
end

subgraph group_models["Model architectures"]
  node_zipmodel["Zipformer model"]
  node_paramodel["Parakeet model"]
end

subgraph group_runtime["Inference runtime"]
  node_api["Python API<br/>[__init__.py]"]
  node_pipeline["ASR pipeline<br/>[asr.py]"]
  node_bundle["Model bundle validation<br/>[utils.py]"]
  node_encoder["Audio encoder<br/>[encoder.py]"]
  node_text["Text postprocessing<br/>[postprocessor.py]"]
end

subgraph group_decode["GPU decoding"]
  node_zipdecode["Zipformer transducer"]
  node_tdtdecode["Parakeet TDT search"]
  node_ctcdecode["CTC greedy search<br/>[ctc_decoder.py]"]
  node_kernels["CUDA search kernels<br/>[gpu_kernels.py]"]
end

subgraph group_accel["TensorRT acceleration"]
  node_plugins["Native TensorRT plugins<br/>[__init__.py]"]
  node_tensorrt["TensorRT runtime"]
end

node_user(("ASR user"))
node_result["Text and timestamps"]

node_user -->|"calls"| node_api
node_api -->|"exposes"| node_pipeline
node_user -->|"exports checkpoint"| node_zipexport
node_user -->|"exports checkpoint"| node_paraexport
node_zipexport -->|"builds from"| node_zipmodel
node_paraexport -->|"builds from"| node_paramodel
node_zipexport -->|"uses"| node_exportutils
node_paraexport -->|"uses"| node_exportutils
node_exportutils -->|"loads plugins"| node_plugins
node_exportutils -->|"builds engines"| node_tensorrt
node_pipeline -->|"validates bundle"| node_bundle
node_pipeline -->|"runs audio"| node_encoder
node_pipeline -->|"dispatches"| node_zipdecode
node_pipeline -->|"dispatches"| node_tdtdecode
node_pipeline -.->|"dispatches"| node_ctcdecode
node_zipdecode -->|"launches"| node_kernels
node_tdtdecode -->|"launches"| node_kernels
node_ctcdecode -->|"launches"| node_kernels
node_pipeline -.->|"postprocesses"| node_text
node_encoder -->|"executes engine"| node_tensorrt
node_zipdecode -->|"executes engine"| node_tensorrt
node_tdtdecode -->|"executes engine"| node_tensorrt
node_pipeline -->|"returns transcript"| node_result

click node_api "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/__init__.py"
click node_pipeline "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/asr.py"
click node_bundle "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/utils.py"
click node_encoder "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/encoder/encoder.py"
click node_text "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/postprocessor.py"
click node_zipdecode "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/zipformer_rnnt_decoder.py"
click node_tdtdecode "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/parakeet_tdt_decoder.py"
click node_ctcdecode "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/ctc_decoder.py"
click node_kernels "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/gpu_kernels.py"
click node_zipexport "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/export/export_zipformer.py"
click node_paraexport "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/export/export_parakeet.py"
click node_exportutils "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/export/export_utils.py"
click node_zipmodel "https://github.com/soundsgoodai/fast-gpu-asr/tree/main/src/fast_gpu_asr/export/model/zipformer"
click node_paramodel "https://github.com/soundsgoodai/fast-gpu-asr/tree/main/src/fast_gpu_asr/export/model/parakeet"
click node_plugins "https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/tensorrt_plugins/__init__.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_zipexport,node_paraexport,node_exportutils,node_user toneBlue
class node_zipmodel,node_paramodel toneAmber
class node_api,node_pipeline,node_bundle,node_encoder,node_text toneMint
class node_zipdecode,node_tdtdecode,node_ctcdecode,node_kernels toneRose
class node_plugins,node_tensorrt,node_result toneIndigo
```

## Components

### Model export

- [Zipformer exporter](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/export/export_zipformer.py): `src/fast_gpu_asr/export/export_zipformer.py`
- [Parakeet exporter](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/export/export_parakeet.py): `src/fast_gpu_asr/export/export_parakeet.py`
- [Engine export utilities](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/export/export_utils.py): `src/fast_gpu_asr/export/export_utils.py`

### Model architectures

- [Zipformer model](https://github.com/soundsgoodai/fast-gpu-asr/tree/main/src/fast_gpu_asr/export/model/zipformer): `src/fast_gpu_asr/export/model/zipformer`
- [Parakeet model](https://github.com/soundsgoodai/fast-gpu-asr/tree/main/src/fast_gpu_asr/export/model/parakeet): `src/fast_gpu_asr/export/model/parakeet`

### Inference runtime

- [Python API](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/__init__.py): `src/fast_gpu_asr/__init__.py`
- [ASR pipeline](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/asr.py): `src/fast_gpu_asr/asr.py`
- [Model bundle validation](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/utils.py): `src/fast_gpu_asr/utils.py`
- [Audio encoder](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/encoder/encoder.py): `src/fast_gpu_asr/encoder/encoder.py`
- [Text postprocessing](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/postprocessor.py): `src/fast_gpu_asr/decoder/postprocessor.py`

### GPU decoding

- [Zipformer transducer](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/zipformer_rnnt_decoder.py): `src/fast_gpu_asr/decoder/zipformer_rnnt_decoder.py`
- [Parakeet TDT search](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/parakeet_tdt_decoder.py): `src/fast_gpu_asr/decoder/parakeet_tdt_decoder.py`
- [CTC greedy search](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/ctc_decoder.py): `src/fast_gpu_asr/decoder/ctc_decoder.py`
- [CUDA search kernels](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/decoder/gpu_kernels.py): `src/fast_gpu_asr/decoder/gpu_kernels.py`

### TensorRT acceleration

- [Native TensorRT plugins](https://github.com/soundsgoodai/fast-gpu-asr/blob/main/src/fast_gpu_asr/tensorrt_plugins/__init__.py): `src/fast_gpu_asr/tensorrt_plugins/__init__.py`
- **TensorRT runtime**

### Other components

- **ASR user**
- **Text and timestamps**

## Connections

- ASR user → Python API: calls (evidence: `README.md`)
- Python API → ASR pipeline: exposes (evidence: `src/fast_gpu_asr/__init__.py`)
- ASR user → Zipformer exporter: exports checkpoint (evidence: `README.md`)
- ASR user → Parakeet exporter: exports checkpoint (evidence: `README.md`)
- Zipformer exporter → Zipformer model: builds from (evidence: `src/fast_gpu_asr/export/export_zipformer.py`)
- Parakeet exporter → Parakeet model: builds from (evidence: `src/fast_gpu_asr/export/export_parakeet.py`)
- Zipformer exporter → Engine export utilities: uses (evidence: `src/fast_gpu_asr/export/export_zipformer.py`)
- Parakeet exporter → Engine export utilities: uses (evidence: `src/fast_gpu_asr/export/export_parakeet.py`)
- Engine export utilities → Native TensorRT plugins: loads plugins (evidence: `src/fast_gpu_asr/export/export_utils.py`)
- Engine export utilities → TensorRT runtime: builds engines (evidence: `README.md`)
- ASR pipeline → Model bundle validation: validates bundle (evidence: `src/fast_gpu_asr/asr.py`)
- ASR pipeline → Audio encoder: runs audio (evidence: `src/fast_gpu_asr/asr.py`)
- ASR pipeline → Zipformer transducer: dispatches (evidence: `src/fast_gpu_asr/asr.py`)
- ASR pipeline → Parakeet TDT search: dispatches (evidence: `src/fast_gpu_asr/asr.py`)
- ASR pipeline → CTC greedy search: dispatches (evidence: `README.md`)
- Zipformer transducer → CUDA search kernels: launches (evidence: `src/fast_gpu_asr/decoder/zipformer_rnnt_decoder.py`)
- Parakeet TDT search → CUDA search kernels: launches (evidence: `src/fast_gpu_asr/decoder/parakeet_tdt_decoder.py`)
- CTC greedy search → CUDA search kernels: launches (evidence: `src/fast_gpu_asr/decoder/ctc_decoder.py`)
- ASR pipeline → Text postprocessing: postprocesses (evidence: `src/fast_gpu_asr/asr.py`)
- Audio encoder → TensorRT runtime: executes engine (evidence: `src/fast_gpu_asr/encoder/encoder.py`)
- Zipformer transducer → TensorRT runtime: executes engine (evidence: `src/fast_gpu_asr/decoder/zipformer_rnnt_decoder.py`)
- Parakeet TDT search → TensorRT runtime: executes engine (evidence: `src/fast_gpu_asr/decoder/parakeet_tdt_decoder.py`)
- ASR pipeline → Text and timestamps: returns transcript (evidence: `README.md`)
