How twilio-labs/call-gpt works
Main Components
a) Client Layer:
• Phone/PSTN connection
• Twilio Phone Numbers
b) Communication Layer:
• Twilio Media Streams (WebSocket connections)
• Voice streams (bidirectional audio)
c) Core Application Components:
• Express.js Server (app.js)
• Service Layer:
• GPT Service (gpt-service.js)
• Recording Service (recording-service.js)
• Stream Service (stream-service.js)
• Transcription Service (transcription-service.js)
• TTS Service (tts-service.js)
d) Function Layer:
• Function Manifest
• Individual Functions:
• checkInventory
• checkPrice
• placeOrder
• transferCall
e) External Services:
• Deepgram (Speech-to-Text & Text-to-Speech)
• OpenAI (GPT)
• Optional: Eleven Labs (Alternative TTS)
[User/Phone] → [Twilio] → [Application] → [External Services]
Key Flows to Highlight
a) Inbound Call Flow:
Phone → Twilio Number → Media Streams → Stream Service →
Transcription Service → GPT Service → TTS Service →
Stream Service → Phone
b) Function Call Flow:
GPT Service → Function Manifest → Specific Function →
External API/Data → GPT Service
Important Details
• WebSocket connections for real-time audio streaming
• Bidirectional audio flows
• Chat history maintenance in GPT service
• Function calling mechanism
• Service interaction patterns
Architectural Patterns to Emphasize
• Event-driven architecture (evident in the service implementations)
• Microservices-like separation of concerns
• Streaming architecture for real-time processing
• Plugin architecture for functions
Deployment Context
A deployment section showing:
• Fly.io hosting
• Region selection (us-east-1 recommended)
• ngrok for local development
Remember to include notes about scalability considerations and potential bottlenecks, especially around the external service dependencies (Deepgram, OpenAI, etc.).
This system design should help developers understand both the high-level architecture and the detailed component interactions that make this voice-based AI system work effectively.