How infinitocloud/asterisk_to_openai_rt works
Step 1. Identify Project Type and Purpose
• This is a real‐time voice integration middleware: a Node.js application (“asterisk_to_openai_rt.js”) that sits between an Asterisk PBX and OpenAI’s real‐time API.
• Purpose: answer incoming SIP calls through Asterisk, capture and process RTP audio, send audio chunks to OpenAI’s WebSocket API, receive back synthesized audio + transcripts, and stream responses back to the caller.
Step 2. Analyze File Structure
• Top level:
• asterisk_to_openai_rt.js – main application
• package.json / package-lock.json – Node.js dependencies (e.g., ‘ari-client’, ‘ws’, ‘audio-processing’, ‘winston’)
• .env.example – environment config
• sample Asterisk configs (extensions.conf.sample, pjsip.conf.sample)
• test-tools/ – collection of test scripts for individual features (TTS, audio validation, chat RT, ARI integration tests).
• No separate frontend/backend; single service with internal sub‐modules.
Step 3. Key Insights from README
• Connects to Asterisk via ARI (HTTP+WebSocket events) at port 8088
• Sets up mixing bridge & ExternalMedia channels on RTP port 12000
• Audio pipeline:
μ-law RTP → strip headers → upsample to 24 kHz PCM → normalize → buffer
• WebSocket client to OpenAI’s RT API → send base64 PCM chunks every 200 ms → receive PCM + transcripts
• Convert back to μ-law → RTP send back with 10 ms packet timing
• Manages VAD via server config to avoid overlapping streams
• Logging via Winston, optional file‐recording, and cleanup on call end / errors.
Step 4. Main Components & Interactions
a. Components
1. Asterisk PBX
– ARI REST/WebSocket endpoint
– RTP engine for SIP media
2. Node.js Application
– ARI Client module
– RTP Receiver module
– Audio Processing module (interpolation, normalization, μ-law⇄PCM)
– OpenAI WebSocket Client
– RTP Sender module
– VAD Controller
– Logger & Recorder
– Configuration Loader
3. OpenAI Real-Time API (External Service)
4. Test-Tools Suite (Optional; separate scripts to validate each piece)
b. Relationships & Data Flow
1. SIP Client dials → Asterisk receives call → Stasis event via ARI → Node.js ARI Client
2. Node.js configures bridge & ExternalMedia → instructs Asterisk to send RTP on port 12000
3. RTP Receiver grabs μ-law packets → strips RTP headers → sends raw audio to Audio Processing pipeline
4. Audio Processing upsamples → normalizes → pushes PCM chunks to OpenAI WebSocket Client
5. OpenAI streams back synthesized PCM + transcripts → Node.js converts to μ-law
6. RTP Sender streams μ-law back to Asterisk on a new RTP port → caller hears response
7. VAD signals pause/resume of RTP streaming based on speech detection
8. Logger records events; optional Recorder writes raw/.wav files to disk
9. On call end or error → Node.js tears down WebSocket, RTP channels, and cleaning resources
c. Architectural Patterns / Principles
• Single‐process microkernel: discrete internal modules communicate via in‐memory events or function calls.
• Event‐driven (ARI events, WebSocket data, RTP packet callbacks).
• Pipeline pattern for audio processing.
• Clear separation: Signaling (ARI) vs. Media (RTP) vs. External API.
d. Key Technologies / Frameworks
• ari-client (ARI REST & WebSocket)
• ws (WebSocket client for OpenAI)
• Node.js built-in dgram/tls/http
• Audio libs (μ-law conversion, sample rate conversion)
• Winston (logging)
• dotenv (config)
• Asterisk 20 (PBX), OpenAI realtime model gpt-4o-realtime-preview
• Separation of:
Front‐end (Asterisk PBX & SIP Client) ↔ Middleware (Node.js app) ↔ Back‐end (OpenAI WebSocket API)
• Bidirectional RTP media flows and control-plane ARI interactions.
• Audio processing pipeline within the middleware.
• SIP Client → Asterisk (SIP signaling)
• Asterisk ↔ Node.js (ARI events HTTP/WebSocket)
• Asterisk ←→ Node.js (RTP μ-law media)
• Node.js ↔ OpenAI (WebSocket audio/text)
• Green for “External Systems” (SIP Client, Asterisk, OpenAI)
• Blue for “Core Service” sub‐modules (inside Node.js)
• Gray for “Test Tools”
Final Tips
• Buffering & VAD logic in the audio pipeline.
• Keep it tidy: group related sub‐modules visually, but don’t omit any essential component or data flow.