How gradio-app/fastrtc works
FastRTC turns Python handlers into real-time audio/video sessions: users connect through a built-in Gradio UI or mounted WebRTC/WebSocket endpoints, media reaches a handler, and generated media or messages return to the client. Audio turn-taking, speech conversion, and phone access are documented capabilities. The graph covers the Python runtime and sampled frontend components; bundled Mermaid assets are included because they are substantial indexed modules. Some links rely on README or SOURCE INDEX references rather than inspected implementation details.
Main entry points: stream.py Stream orchestration · reply_on_pause.py Pause / stopword handlers · radar-MK3ICKWK-CdUl77BF.js Radar diagram · Index.svelte Gradio component UI
Components 20
Entry points
- Stream orchestration
backend/fastrtc/stream.py
Media transport
- WebRTC component
backend/fastrtc/webrtc.py - Peer connections
backend/fastrtc/webrtc_connection_mixin.py - Media tracks
backend/fastrtc/tracks.py - WebSocket transport
backend/fastrtc/websocket.py - Media utilities
backend/fastrtc/utils.py
Audio processing
- Pause / stopword handlers
backend/fastrtc/reply_on_pause.py - Silero pause detection
backend/fastrtc/pause_detection/silero.py - Speech recognition
backend/fastrtc/speech_to_text/stt_.py - Speech synthesis
backend/fastrtc/text_to_speech/tts.py
Browser UI
- Gradio component UI
frontend/Index.svelte - Interactive audio
frontend/shared/InteractiveAudio.svelte - Interactive video
frontend/shared/InteractiveVideo.svelte - Audio visualization
frontend/shared/AudioWave.svelte - Pulse indicator
frontend/shared/PulsingIcon.svelte
Diagram assets
- Mermaid core
backend/fastrtc/templates/component/mermaid.core-DBM7LDcW.js - Cytoscape engine
backend/fastrtc/templates/component/cytoscape.esm-C2cgT2B2.js - Diagram component index
backend/fastrtc/templates/component/index-Bte5FCpu.js - Radar diagram
backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js
Other components
- Caller / user
Connections 20
- Caller / user → Stream orchestration: starts stream
README.md - Stream orchestration → WebRTC component: configures UI
backend/fastrtc/stream.py - Stream orchestration → WebSocket transport: routes sessions
backend/fastrtc/stream.py - WebRTC component → Peer connections: uses connection mixin
backend/fastrtc/webrtc.py - Peer connections → Media tracks: manages tracks
backend/fastrtc/webrtc_connection_mixin.py - WebSocket transport → Media tracks: uses track handlers
backend/fastrtc/websocket.py - Stream orchestration → Media tracks: uses media tracks
backend/fastrtc/stream.py - Stream orchestration → Media utilities: uses utilities
backend/fastrtc/stream.py - Stream orchestration → Pause / stopword handlers: accepts handler
README.md - Pause / stopword handlers → Silero pause detection: uses VAD
- Pause / stopword handlers → Speech recognition: may transcribe
- Pause / stopword handlers → Speech synthesis: may synthesize
- Caller / user → Gradio component UI: uses built-in UI
README.md - Gradio component UI → Interactive audio: renders audio
frontend/Index.svelte - Gradio component UI → Interactive video: renders video
frontend/Index.svelte - Interactive audio → Audio visualization: renders visualization
frontend/shared/InteractiveAudio.svelte - Audio visualization → Pulse indicator: renders icon
frontend/shared/AudioWave.svelte - Radar diagram → Mermaid core: imports core
backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js - Radar diagram → Cytoscape engine: uses engine
backend/fastrtc/templates/component/radar-MK3ICKWK-CdUl77BF.js - Radar diagram → Diagram component index: references component