# infinitocloud/asterisk_to_openai_rt architecture

> Architecture diagram of the GitHub repository infinitocloud/asterisk_to_openai_rt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/infinitocloud/asterisk_to_openai_rt
- Repository: https://github.com/infinitocloud/asterisk_to_openai_rt

## Overview

Step 1. Identify Project Type and Purpose
- This is a real‐time voice integration middleware: a Node.js application (“asterisk_to_openai_rt.js”) that sits between an Asterisk PBX and OpenAI’s real‐time API.
- Purpose: answer incoming SIP calls through Asterisk, capture and process RTP audio, send audio chunks to OpenAI’s WebSocket API, receive back synthesized audio + transcripts, and stream responses back to the caller.

Step 2. Analyze File Structure
- Top level:
- asterisk_to_openai_rt.js – main application
- package.json / package-lock.json – Node.js dependencies (e.g., ‘ari-client’, ‘ws’, ‘audio-processing’, ‘winston’)
- .env.example – environment config
- sample Asterisk configs (extensions.conf.sample, pjsip.conf.sample)
- test-tools/ – collection of test scripts for individual features (TTS, audio validation, chat RT, ARI integration tests).
- No separate frontend/backend; single service with internal sub‐modules.

Step 3. Key Insights from README
- Connects to Asterisk via ARI (HTTP+WebSocket events) at port 8088
- Sets up mixing bridge & ExternalMedia channels on RTP port 12000
- Audio pipeline:
μ-law RTP → strip headers → upsample to 24 kHz PCM → normalize → buffer
- WebSocket client to OpenAI’s RT API → send base64 PCM chunks every 200 ms → receive PCM + transcripts
- Convert back to μ-law → RTP send back with 10 ms packet timing
- Manages VAD via server config to avoid overlapping streams
- Logging via Winston, optional file‐recording, and cleanup on call end / errors.

Step 4. Main Components & Interactions
a. Components
1. Asterisk PBX
– ARI REST/WebSocket endpoint
– RTP engine for SIP media
2. Node.js Application
– ARI Client module
– RTP Receiver module
– Audio Processing module (interpolation, normalization, μ-law⇄PCM)
– OpenAI WebSocket Client
– RTP Sender module
– VAD Controller
– Logger & Recorder
– Configuration Loader
3. OpenAI Real-Time API (External Service)
4. Test-Tools Suite (Optional; separate scripts to validate each piece)

b. Relationships & Data Flow
1. SIP Client dials → Asterisk receives call → Stasis event via ARI → Node.js ARI Client
2. Node.js configures bridge & ExternalMedia → instructs Asterisk to send RTP on port 12000
3. RTP Receiver grabs μ-law packets → strips RTP headers → sends raw audio to Audio Processing pipeline
4. Audio Processing upsamples → normalizes → pushes PCM chunks to OpenAI WebSocket Client
5. OpenAI streams back synthesized PCM + transcripts → Node.js converts to μ-law
6. RTP Sender streams μ-law back to Asterisk on a new RTP port → caller hears response
7. VAD signals pause/resume of RTP streaming based on speech detection
8. Logger records events; optional Recorder writes raw/.wav files to disk
9. On call end or error → Node.js tears down WebSocket, RTP channels, and cleaning resources

c. Architectural Patterns / Principles
- Single‐process microkernel: discrete internal modules communicate via in‐memory events or function calls.
- Event‐driven (ARI events, WebSocket data, RTP packet callbacks).
- Pipeline pattern for audio processing.
- Clear separation: Signaling (ARI) vs. Media (RTP) vs. External API.

d. Key Technologies / Frameworks
- ari-client (ARI REST & WebSocket)
- ws (WebSocket client for OpenAI)
- Node.js built-in dgram/tls/http
- Audio libs (μ-law conversion, sample rate conversion)
- Winston (logging)
- dotenv (config)
- Asterisk 20 (PBX), OpenAI realtime model gpt-4o-realtime-preview

- Separation of:
Front‐end (Asterisk PBX & SIP Client) ↔ Middleware (Node.js app) ↔ Back‐end (OpenAI WebSocket API)
- Bidirectional RTP media flows and control-plane ARI interactions.
- Audio processing pipeline within the middleware.

- SIP Client → Asterisk (SIP signaling)
- Asterisk ↔ Node.js (ARI events HTTP/WebSocket)
- Asterisk ←→ Node.js (RTP μ-law media)
- Node.js ↔ OpenAI (WebSocket audio/text)
- Green for “External Systems” (SIP Client, Asterisk, OpenAI)
- Blue for “Core Service” sub‐modules (inside Node.js)
- Gray for “Test Tools”

Final Tips
- Buffering & VAD logic in the audio pipeline.
- Keep it tidy: group related sub‐modules visually, but don’t omit any essential component or data flow.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Systems
    SIPClient["SIP Client"]:::external
    Asterisk["Asterisk PBX\n(ARI + RTP)"]:::external
    OpenAI["OpenAI RT API"]:::external

    %% Middleware Service
    subgraph "Node.js Middleware" 
        direction TB
        Middleware["Main App\n(asterisk_to_openai_rt.js)"]:::core
        ConfigLoader["Config Loader\n(.env)"]:::core
        Dependencies["Dependencies\n(package.json)"]:::core
        ARIClient["ARI Client Module"]:::core
        RTPReceiver["RTP Receiver Module"]:::core
        AudioPipeline["Audio Processing Pipeline"]:::core
        OpenAIClient["OpenAI WS Client"]:::core
        RTPSender["RTP Sender Module"]:::core
        VADController["VAD Controller"]:::core
        LoggerRecorder["Logger & Recorder"]:::core
        Cleanup["Cleanup Handlers"]:::core
    end

    %% Test Tools
    subgraph "Test-Tools Suite" 
        direction TB
        TestTools["test-tools/"]:::test
        TTC1["0_app_check_key.js"]:::test
        TTC2["0_app_check_key_realtime.js"]:::test
        TTC3["1_app_tts.js"]:::test
        TTC4["2_app_validate_audio.js"]:::test
        TTC5["3_app_textchat_rt.js"]:::test
        TTC6["4_app_voicechat_rt.js"]:::test
        ARITest["ari/"]:::test
        ARIApp["ari_app_with_rtp.js"]:::test
        RTPServer["rtpServer.js"]:::test
    end

    %% Connections
    SIPClient -->|"SIP signaling"| Asterisk
    Asterisk -->|"ARI REST/WebSocket"| ARIClient
    ARIClient -->|"events/config"| Middleware
    Middleware -->|"ARI commands"| Asterisk
    Asterisk -->|"RTP μ-law audio"| RTPReceiver
    RTPReceiver -->|"raw audio"| AudioPipeline
    AudioPipeline -->|"base64 PCM"| OpenAIClient
    OpenAIClient -->|"PCM + transcripts"| AudioPipeline
    AudioPipeline -->|"μ-law RTP packets"| RTPSender
    RTPSender -->|"RTP μ-law audio"| Asterisk
    AudioPipeline -->|VAD signals| VADController
    VADController -->|control streams| RTPSender
    Middleware -->|"logs/events"| LoggerRecorder
    Middleware -->|cleanup| Cleanup
    OpenAIClient -->|"WebSocket"| OpenAI
    TestTools -->|offline tests| Middleware

    %% Click Events
    click Middleware "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/asterisk_to_openai_rt.js"
    click ConfigLoader "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/.env.example"
    click Dependencies "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/package.json"
    click ARIClient "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/asterisk_to_openai_rt.js"
    click RTPReceiver "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/asterisk_to_openai_rt.js"
    click AudioPipeline "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/asterisk_to_openai_rt.js"
    click OpenAIClient "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/asterisk_to_openai_rt.js"
    click RTPSender "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/asterisk_to_openai_rt.js"
    click VADController "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/asterisk_to_openai_rt.js"
    click LoggerRecorder "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/asterisk_to_openai_rt.js"
    click TestTools "https://github.com/infinitocloud/asterisk_to_openai_rt/tree/main/test-tools/"
    click TTC1 "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/test-tools/0_app_check_key.js"
    click TTC2 "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/test-tools/0_app_check_key_realtime.js"
    click TTC3 "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/test-tools/1_app_tts.js"
    click TTC4 "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/test-tools/2_app_validate_audio.js"
    click TTC5 "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/test-tools/3_app_textchat_rt.js"
    click TTC6 "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/test-tools/4_app_voicechat_rt.js"
    click ARITest "https://github.com/infinitocloud/asterisk_to_openai_rt/tree/main/test-tools/ari/"
    click ARIApp "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/test-tools/ari/ari_app_with_rtp.js"
    click RTPServer "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/test-tools/ari/rtpServer.js"
    click ExtConf "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/extensions.conf.sample"
    click PJSIPConf "https://github.com/infinitocloud/asterisk_to_openai_rt/blob/main/pjsip.conf.sample"

    %% Asterisk Sample Configs
    Asterisk --> ExtConf
    Asterisk --> PJSIPConf

    %% Styles
    classDef external fill:#c2f0c2,stroke:#333,stroke-width:1px
    classDef core fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef test fill:#dddddd,stroke:#333,stroke-width:1px
```
