# gradio-app/fastrtc architecture

> Architecture diagram of the GitHub repository gradio-app/fastrtc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gradio-app/fastrtc
- Repository: https://github.com/gradio-app/fastrtc

## Overview

FastRTC turns Python handlers into real-time audio/video sessions: users connect through a built-in Gradio UI or mounted WebRTC/WebSocket endpoints, media reaches a handler, and generated media or messages return to the client. Audio turn-taking, speech conversion, and phone access are documented capabilities. The graph covers the Python runtime and sampled frontend components; bundled Mermaid assets are included because they are substantial indexed modules. Some links rely on README or SOURCE INDEX references rather than inspected implementation details.

Main entry points: [`backend/fastrtc/stream.py`](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/stream.py) (Stream orchestration), [`backend/fastrtc/reply_on_pause.py`](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/reply_on_pause.py) (Pause / stopword handlers), [`backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js`](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js) (Radar diagram), [`frontend/Index.svelte`](https://github.com/gradio-app/fastrtc/blob/main/frontend/Index.svelte) (Gradio component UI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry points"]
  node_stream["Stream orchestration<br/>[stream.py]"]
end

subgraph group_media["Media transport"]
  node_webrtc["WebRTC component<br/>[webrtc.py]"]
  node_connection["Peer connections"]
  node_tracks["Media tracks<br/>[tracks.py]"]
  node_websocket["WebSocket transport<br/>[websocket.py]"]
  node_utils["Media utilities<br/>[utils.py]"]
end

subgraph group_audio["Audio processing"]
  node_turn["Pause / stopword handlers<br/>[reply_on_pause.py]"]
  node_pause["Silero pause detection<br/>[silero.py]"]
  node_stt["Speech recognition<br/>[stt_.py]"]
  node_tts["Speech synthesis<br/>[tts.py]"]
end

subgraph group_frontend["Browser UI"]
  node_frontend["Gradio component UI<br/>[Index.svelte]"]
  node_audio_ui["Interactive audio"]
  node_video_ui["Interactive video"]
  node_audio_wave["Audio visualization<br/>[AudioWave.svelte]"]
  node_pulse["Pulse indicator<br/>[PulsingIcon.svelte]"]
end

subgraph group_diagrams["Diagram assets"]
  node_mermaid["Mermaid core"]
  node_cytoscape["Cytoscape engine"]
  node_diagram_index["Diagram component index<br/>[index-Bte5FCpu.js]"]
  node_radar["Radar diagram"]
end

node_user(("Caller / user"))

node_user -->|"starts stream"| node_stream
node_stream -->|"configures UI"| node_webrtc
node_stream -->|"routes sessions"| node_websocket
node_webrtc -->|"uses connection mixin"| node_connection
node_connection -->|"manages tracks"| node_tracks
node_websocket -->|"uses track handlers"| node_tracks
node_stream -->|"uses media tracks"| node_tracks
node_stream -->|"uses utilities"| node_utils
node_stream -->|"accepts handler"| node_turn
node_turn -.->|"uses VAD"| node_pause
node_turn -.->|"may transcribe"| node_stt
node_turn -.->|"may synthesize"| node_tts
node_user -->|"uses built-in UI"| node_frontend
node_frontend -->|"renders audio"| node_audio_ui
node_frontend -->|"renders video"| node_video_ui
node_audio_ui -->|"renders visualization"| node_audio_wave
node_audio_wave -->|"renders icon"| node_pulse
node_radar -->|"imports core"| node_mermaid
node_radar -->|"uses engine"| node_cytoscape
node_radar -.->|"references component"| node_diagram_index

click node_stream "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/stream.py"
click node_turn "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/reply_on_pause.py"
click node_pause "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/pause_detection/silero.py"
click node_stt "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/speech_to_text/stt_.py"
click node_tts "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/text_to_speech/tts.py"
click node_webrtc "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/webrtc.py"
click node_connection "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/webrtc_connection_mixin.py"
click node_tracks "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/tracks.py"
click node_websocket "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/websocket.py"
click node_utils "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/utils.py"
click node_frontend "https://github.com/gradio-app/fastrtc/blob/main/frontend/Index.svelte"
click node_audio_ui "https://github.com/gradio-app/fastrtc/blob/main/frontend/shared/InteractiveAudio.svelte"
click node_video_ui "https://github.com/gradio-app/fastrtc/blob/main/frontend/shared/InteractiveVideo.svelte"
click node_audio_wave "https://github.com/gradio-app/fastrtc/blob/main/frontend/shared/AudioWave.svelte"
click node_pulse "https://github.com/gradio-app/fastrtc/blob/main/frontend/shared/PulsingIcon.svelte"
click node_mermaid "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/templates/component/mermaid.core-DBM7LDcW.js"
click node_cytoscape "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/templates/component/cytoscape.esm-C2cgT2B2.js"
click node_diagram_index "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/templates/component/index-Bte5FCpu.js"
click node_radar "https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_stream,node_user toneBlue
class node_webrtc,node_connection,node_tracks,node_websocket,node_utils toneAmber
class node_turn,node_pause,node_stt,node_tts toneMint
class node_frontend,node_audio_ui,node_video_ui,node_audio_wave,node_pulse toneRose
class node_mermaid,node_cytoscape,node_diagram_index,node_radar toneIndigo
```

## Components

### Entry points

- [Stream orchestration](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/stream.py): `backend/fastrtc/stream.py`

### Media transport

- [WebRTC component](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/webrtc.py): `backend/fastrtc/webrtc.py`
- [Peer connections](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/webrtc_connection_mixin.py): `backend/fastrtc/webrtc_connection_mixin.py`
- [Media tracks](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/tracks.py): `backend/fastrtc/tracks.py`
- [WebSocket transport](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/websocket.py): `backend/fastrtc/websocket.py`
- [Media utilities](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/utils.py): `backend/fastrtc/utils.py`

### Audio processing

- [Pause / stopword handlers](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/reply_on_pause.py): `backend/fastrtc/reply_on_pause.py`
- [Silero pause detection](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/pause_detection/silero.py): `backend/fastrtc/pause_detection/silero.py`
- [Speech recognition](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/speech_to_text/stt_.py): `backend/fastrtc/speech_to_text/stt_.py`
- [Speech synthesis](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/text_to_speech/tts.py): `backend/fastrtc/text_to_speech/tts.py`

### Browser UI

- [Gradio component UI](https://github.com/gradio-app/fastrtc/blob/main/frontend/Index.svelte): `frontend/Index.svelte`
- [Interactive audio](https://github.com/gradio-app/fastrtc/blob/main/frontend/shared/InteractiveAudio.svelte): `frontend/shared/InteractiveAudio.svelte`
- [Interactive video](https://github.com/gradio-app/fastrtc/blob/main/frontend/shared/InteractiveVideo.svelte): `frontend/shared/InteractiveVideo.svelte`
- [Audio visualization](https://github.com/gradio-app/fastrtc/blob/main/frontend/shared/AudioWave.svelte): `frontend/shared/AudioWave.svelte`
- [Pulse indicator](https://github.com/gradio-app/fastrtc/blob/main/frontend/shared/PulsingIcon.svelte): `frontend/shared/PulsingIcon.svelte`

### Diagram assets

- [Mermaid core](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/templates/component/mermaid.core-DBM7LDcW.js): `backend/fastrtc/templates/component/mermaid.core-DBM7LDcW.js`
- [Cytoscape engine](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/templates/component/cytoscape.esm-C2cgT2B2.js): `backend/fastrtc/templates/component/cytoscape.esm-C2cgT2B2.js`
- [Diagram component index](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/templates/component/index-Bte5FCpu.js): `backend/fastrtc/templates/component/index-Bte5FCpu.js`
- [Radar diagram](https://github.com/gradio-app/fastrtc/blob/main/backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js): `backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js`

### Other components

- **Caller / user**

## Connections

- Caller / user → Stream orchestration: starts stream (evidence: `README.md`)
- Stream orchestration → WebRTC component: configures UI (evidence: `backend/fastrtc/stream.py`)
- Stream orchestration → WebSocket transport: routes sessions (evidence: `backend/fastrtc/stream.py`)
- WebRTC component → Peer connections: uses connection mixin (evidence: `backend/fastrtc/webrtc.py`)
- Peer connections → Media tracks: manages tracks (evidence: `backend/fastrtc/webrtc_connection_mixin.py`)
- WebSocket transport → Media tracks: uses track handlers (evidence: `backend/fastrtc/websocket.py`)
- Stream orchestration → Media tracks: uses media tracks (evidence: `backend/fastrtc/stream.py`)
- Stream orchestration → Media utilities: uses utilities (evidence: `backend/fastrtc/stream.py`)
- Stream orchestration → Pause / stopword handlers: accepts handler (evidence: `README.md`)
- Pause / stopword handlers → Silero pause detection: uses VAD
- Pause / stopword handlers → Speech recognition: may transcribe
- Pause / stopword handlers → Speech synthesis: may synthesize
- Caller / user → Gradio component UI: uses built-in UI (evidence: `README.md`)
- Gradio component UI → Interactive audio: renders audio (evidence: `frontend/Index.svelte`)
- Gradio component UI → Interactive video: renders video (evidence: `frontend/Index.svelte`)
- Interactive audio → Audio visualization: renders visualization (evidence: `frontend/shared/InteractiveAudio.svelte`)
- Audio visualization → Pulse indicator: renders icon (evidence: `frontend/shared/AudioWave.svelte`)
- Radar diagram → Mermaid core: imports core (evidence: `backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js`)
- Radar diagram → Cytoscape engine: uses engine (evidence: `backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js`)
- Radar diagram → Diagram component index: references component
