# openai/openai-realtime-twilio-demo architecture

> Architecture diagram of the GitHub repository openai/openai-realtime-twilio-demo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openai/openai-realtime-twilio-demo
- Repository: https://github.com/openai/openai-realtime-twilio-demo

## Overview

##### Project Type and Purpose
- This is a full-stack, real-time communications application (“OpenAI Realtime API with Twilio Quickstart”).
- Purpose: Provide a browser-based call configuration UI (Next.js “webapp”) and a backend service (Express/WebSocket “websocket-server”) that bridges Twilio media streams with OpenAI’s Realtime API, enabling an AI calling assistant.

##### File-Structure Analysis
- Top-level:
- webapp/ – Next.js frontend
- websocket-server/ – Express + WebSocket backend
- Frontend folders of note:
- app/api/twilio – REST endpoints for Twilio integration and webhooks
- components/ – React UI panels for call setup & transcript display
- lib/ – helpers for Twilio, Realtime API calls, real-time event handling
- Backend folders:
- src/server.ts – Express app that exposes /twiml and WebSocket endpoints (/call, /logs)
- src/sessionManager.ts + functionHandlers.ts – orchestrate Twilio streams ↔ OpenAI streams
- Config files: .env.example in both services, ngrok assumption in README.

##### Key Insights from README
- Two processes:
1. Frontend dev server (`npm run dev` in webapp)
2. WebSocket server (`npm run dev` in websocket-server)
3. ngrok to expose websocket server port (8081) for Twilio callbacks
- Call flow: incoming call → Twilio fetches TwiML (/twiml) → Twilio opens WS stream (/call) → backend ↔ OpenAI Realtime API ↔ Twilio → backend pushes logs/transcripts to frontend WS (/logs).

##### Detail Orientation
- Every hop, even tunnel layers (ngrok) if running locally.
- Where authentication/env vars are required (Twilio creds, OpenAI key).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Agent
    Browser["Browser / User Agent"]:::frontend

    %% Frontend Layer
    subgraph "Next.js Webapp" 
        direction TB
        Webapp_Page["page.tsx"]:::frontend
        Webapp_Layout["layout.tsx"]:::frontend
        Webapp_CSS["globals.css"]:::frontend

        subgraph "API Routes (app/api/twilio)" 
            direction TB
            API_Twilio["route.ts"]:::control
            API_Numbers["numbers/route.ts"]:::control
            API_Webhook["webhook-local/route.ts"]:::control
        end

        subgraph "React UI Panels" 
            direction TB
            Comp_Call["call-interface.tsx"]:::frontend
            Comp_Transcript["transcript.tsx"]:::frontend
            Comp_SessionConfig["session-configuration-panel.tsx"]:::frontend
            Comp_PhoneChecklist["phone-number-checklist.tsx"]:::frontend
            Comp_FunctionCalls["function-calls-panel.tsx"]:::frontend
            Comp_ToolConfig["tool-configuration-dialog.tsx"]:::frontend
            Comp_TopBar["top-bar.tsx"]:::frontend
            Comp_ChecklistSummary["checklist-and-config.tsx"]:::frontend
        end

        subgraph "Helpers & SDK Wrappers (lib)" 
            direction TB
            Lib_Twilio["twilio.ts"]:::control
            Lib_RealtimeEvent["handle-realtime-event.ts"]:::control
            Lib_BackendTools["use-backend-tools.ts"]:::control
            Lib_Utils["utils.ts"]:::control
            Lib_Templates["tool-templates.ts"]:::control
        end
    end

    %% Backend Layer
    subgraph "WebSocket-Server (Express + WS)" 
        direction TB
        WS_Server["server.ts\n(/twiml, /call, /logs)"]:::backend
        WS_SessionMgr["sessionManager.ts"]:::backend
        WS_Functions["functionHandlers.ts"]:::backend
        WS_Twiml["twiml.xml"]:::backend
        WS_Types["types.ts"]:::backend
    end

    %% Infrastructure
    ngrok["ngrok Tunnel"]:::infra

    %% External Services
    Twilio["Twilio Platform\nREST & Media Stream"]:::external
    OpenAI["OpenAI Realtime API"]:::external

    %% Data Flows & Protocols
    Browser -->|"HTTPS"| Webapp_Page
    Browser -->|"WebSocket /logs"| WS_Server

    Webapp_Page -->|"REST API"| API_Twilio
    API_Twilio -->|"REST API"| Twilio

    Twilio -->|"HTTP Webhook"| ngrok
    ngrok -->|"HTTP /twiml"| WS_Server

    Twilio -->|"WS /call"| ngrok
    ngrok -->|"WebSocket /call"| WS_Server

    WS_Server -->|"Forward audio/text frames"| WS_SessionMgr
    WS_SessionMgr -->|"Invoke handlers"| WS_Functions
    WS_SessionMgr -->|"Bidirectional stream"| OpenAI
    OpenAI -->|"Stream responses"| WS_SessionMgr

    WS_Server -->|"WebSocket /logs"| Browser

    %% Click Events Frontend
    click Webapp_Page "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/app/page.tsx"
    click Webapp_Layout "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/app/layout.tsx"
    click Webapp_CSS "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/app/globals.css"
    click API_Twilio "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/app/api/twilio/route.ts"
    click API_Numbers "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/app/api/twilio/numbers/route.ts"
    click API_Webhook "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/app/api/twilio/webhook-local/route.ts"
    click Comp_Call "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/components/call-interface.tsx"
    click Comp_Transcript "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/components/transcript.tsx"
    click Comp_SessionConfig "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/components/session-configuration-panel.tsx"
    click Comp_PhoneChecklist "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/components/phone-number-checklist.tsx"
    click Comp_FunctionCalls "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/components/function-calls-panel.tsx"
    click Comp_ToolConfig "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/components/tool-configuration-dialog.tsx"
    click Comp_TopBar "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/components/top-bar.tsx"
    click Comp_ChecklistSummary "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/components/checklist-and-config.tsx"
    click Lib_Twilio "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/lib/twilio.ts"
    click Lib_RealtimeEvent "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/lib/handle-realtime-event.ts"
    click Lib_BackendTools "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/lib/use-backend-tools.ts"
    click Lib_Utils "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/lib/utils.ts"
    click Lib_Templates "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/lib/tool-templates.ts"
    click FrontendEnv "https://github.com/openai/openai-realtime-twilio-demo/blob/main/webapp/.env.example"

    %% Click Events Backend
    click WS_Server "https://github.com/openai/openai-realtime-twilio-demo/blob/main/websocket-server/src/server.ts"
    click WS_SessionMgr "https://github.com/openai/openai-realtime-twilio-demo/blob/main/websocket-server/src/sessionManager.ts"
    click WS_Functions "https://github.com/openai/openai-realtime-twilio-demo/blob/main/websocket-server/src/functionHandlers.ts"
    click WS_Twiml "https://github.com/openai/openai-realtime-twilio-demo/blob/main/websocket-server/src/twiml.xml"
    click WS_Types "https://github.com/openai/openai-realtime-twilio-demo/blob/main/websocket-server/src/types.ts"
    click BackendEnv "https://github.com/openai/openai-realtime-twilio-demo/blob/main/websocket-server/.env.example"

    %% Styles
    classDef frontend fill:#E8F0FE,stroke:#4285F4,color:#000
    classDef backend fill:#FFF4E5,stroke:#FB8C00,color:#000
    classDef external fill:#E8F5E9,stroke:#43A047,color:#000
    classDef infra fill:#ECEFF1,stroke:#607D8B,color:#000
    classDef control fill:#F3E5F5,stroke:#8E24AA,color:#000
    classDef data fill:#FFF3E0,stroke:#FB8C00,color:#000
```
