How openai/openai-realtime-twilio-demo works
Project Type and Purpose
• This is a full-stack, real-time communications application (“OpenAI Realtime API with Twilio Quickstart”).
• Purpose: Provide a browser-based call configuration UI (Next.js “webapp”) and a backend service (Express/WebSocket “websocket-server”) that bridges Twilio media streams with OpenAI’s Realtime API, enabling an AI calling assistant.
File-Structure Analysis
• Top-level:
• webapp/ – Next.js frontend
• websocket-server/ – Express + WebSocket backend
• Frontend folders of note:
• app/api/twilio – REST endpoints for Twilio integration and webhooks
• components/ – React UI panels for call setup & transcript display
• lib/ – helpers for Twilio, Realtime API calls, real-time event handling
• Backend folders:
• src/server.ts – Express app that exposes /twiml and WebSocket endpoints (/call, /logs)
• src/sessionManager.ts + functionHandlers.ts – orchestrate Twilio streams ↔ OpenAI streams
• Config files: .env.example in both services, ngrok assumption in README.
Key Insights from README
• Two processes:
1. Frontend dev server (npm run dev in webapp)
2. WebSocket server (npm run dev in websocket-server)
3. ngrok to expose websocket server port (8081) for Twilio callbacks
• Call flow: incoming call → Twilio fetches TwiML (/twiml) → Twilio opens WS stream (/call) → backend ↔ OpenAI Realtime API ↔ Twilio → backend pushes logs/transcripts to frontend WS (/logs).
Detail Orientation
• Every hop, even tunnel layers (ngrok) if running locally.
• Where authentication/env vars are required (Twilio creds, OpenAI key).