How enricollen/fastrtc-voice-agent works
FastRTC Voice Agent turns microphone audio into a transcript, runs a tool-capable conversational agent, then synthesizes and plays an audio reply. The sampled Gradio entry point shows that orchestration directly; README documents a separate custom web UI and provider options. Speech providers are selected through a shared service, while the agent uses LlamaIndex with LiteLLM and a weather tool. The custom UI wiring and several provider implementations were not sampled, so their internal connections are omitted or marked unverified.
Main entry points: speech_service.py Speech service · agent.py Conversation agent · provider.py STT provider contract · main_custom_ui.py Custom web UI
Components 20
Voice interfaces
- Gradio / phone UI
main.py - Custom web UI
main_custom_ui.py - UI assets
static
Conversation and tools
- Conversation agent
src/agent.py - LLM service
src/llm_service.py - Chat history
src/chat_history.py - Weather tool
src/tools/weather_tool.py
Speech processing
- Speech service
src/speech/speech_service.py - STT provider contract
src/speech/stt/provider.py - TTS provider contract
src/speech/tts/provider.py
Speech providers
- ElevenLabs STT
src/speech/stt/elevenlabs_stt.py - Groq STT
src/speech/stt/groq_stt.py - OpenAI STT
src/speech/stt/openai_stt.py - Local Whisper STT
src/speech/stt/whisper_stt.py - ElevenLabs TTS
src/speech/tts/elevenlabs_tts.py - Local Kokoro TTS
src/speech/tts/kokoro_tts.py
Other components
- Voice user
- Browser client
- Meteosource API
- LLM providers
Connections 23
- Voice user → Gradio / phone UI: sends audio
main.py - Gradio / phone UI → Speech service: transcribes / synthesizes
main.py - Gradio / phone UI → Conversation agent: invokes
main.py - Conversation agent → LLM service: initializes
src/agent.py - Conversation agent → Weather tool: registers tool
src/agent.py - Weather tool → Meteosource API: requests weather
src/tools/weather_tool.py - LLM service → LLM providers: queries models
README.md - Custom web UI → Browser client: serves interface
README.md - Browser client → UI assets: uses assets
- Custom web UI → Speech service: handles voice
- Custom web UI → Conversation agent: submits messages
main_custom_ui.py - Speech service → ElevenLabs STT: dispatches STT
src/speech/speech_service.py - Speech service → Groq STT: dispatches STT
src/speech/speech_service.py - Speech service → OpenAI STT: dispatches STT
src/speech/speech_service.py - Speech service → Local Whisper STT: dispatches STT
src/speech/speech_service.py - Speech service → ElevenLabs TTS: dispatches TTS
src/speech/speech_service.py - Speech service → Local Kokoro TTS: dispatches TTS
src/speech/speech_service.py - ElevenLabs STT → STT provider contract: implements
src/speech/stt/elevenlabs_stt.py - Groq STT → STT provider contract: implements
src/speech/stt/groq_stt.py - OpenAI STT → STT provider contract: implements
src/speech/stt/openai_stt.py - Local Whisper STT → STT provider contract: implements
src/speech/stt/whisper_stt.py - ElevenLabs TTS → TTS provider contract: implements
src/speech/tts/elevenlabs_tts.py - Local Kokoro TTS → TTS provider contract: implements
src/speech/tts/kokoro_tts.py