# twilio-labs/call-gpt architecture

> Architecture diagram of the GitHub repository twilio-labs/call-gpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/twilio-labs/call-gpt
- Repository: https://github.com/twilio-labs/call-gpt

## Overview

##### Main Components

a) Client Layer:
- Phone/PSTN connection
- Twilio Phone Numbers

b) Communication Layer:
- Twilio Media Streams (WebSocket connections)
- Voice streams (bidirectional audio)

c) Core Application Components:
- Express.js Server (app.js)
- Service Layer:
- GPT Service (gpt-service.js)
- Recording Service (recording-service.js)
- Stream Service (stream-service.js)
- Transcription Service (transcription-service.js)
- TTS Service (tts-service.js)

d) Function Layer:
- Function Manifest
- Individual Functions:
- checkInventory
- checkPrice
- placeOrder
- transferCall

e) External Services:
- Deepgram (Speech-to-Text & Text-to-Speech)
- OpenAI (GPT)
- Optional: Eleven Labs (Alternative TTS)

**[User/Phone] → [Twilio] → [Application] → [External Services]**

##### Key Flows to Highlight

a) Inbound Call Flow:
**Phone → Twilio Number → Media Streams → Stream Service →**
**Transcription Service → GPT Service → TTS Service →**
**Stream Service → Phone**

b) Function Call Flow:
**GPT Service → Function Manifest → Specific Function →**
**External API/Data → GPT Service**

##### Important Details

- WebSocket connections for real-time audio streaming
- Bidirectional audio flows
- Chat history maintenance in GPT service
- Function calling mechanism
- Service interaction patterns

##### Architectural Patterns to Emphasize

- Event-driven architecture (evident in the service implementations)
- Microservices-like separation of concerns
- Streaming architecture for real-time processing
- Plugin architecture for functions

##### Deployment Context

A deployment section showing:
- Fly.io hosting
- Region selection (us-east-1 recommended)
- ngrok for local development

Remember to include notes about scalability considerations and potential bottlenecks, especially around the external service dependencies (Deepgram, OpenAI, etc.).

This system design should help developers understand both the high-level architecture and the detailed component interactions that make this voice-based AI system work effectively.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client
        Phone[Phone/PSTN]
        TwilioNum[Twilio Phone Numbers]
    end

    subgraph Communication
        TwilioMS[Twilio Media Streams]
        WebSocket[WebSocket Connection]
    end

    subgraph "Core Application"
        Express["Express.js Server"]:::server
        click Express "https://github.com/twilio-labs/call-gpt/blob/main/app.js"

        subgraph Services
            StreamS["Stream Service"]:::service
            click StreamS "https://github.com/twilio-labs/call-gpt/blob/main/services/stream-service.js"
            TranscriptS["Transcription Service"]:::service
            click TranscriptS "https://github.com/twilio-labs/call-gpt/blob/main/services/transcription-service.js"
            GPTS["GPT Service"]:::service
            click GPTS "https://github.com/twilio-labs/call-gpt/blob/main/services/gpt-service.js"
            TTSS["TTS Service"]:::service
            click TTSS "https://github.com/twilio-labs/call-gpt/blob/main/services/tts-service.js"
            RecordS["Recording Service"]:::service
            click RecordS "https://github.com/twilio-labs/call-gpt/blob/main/services/recording-service.js"
        end

        subgraph "Function Layer"
            FuncManifest["Function Manifest"]:::function
            click FuncManifest "https://github.com/twilio-labs/call-gpt/blob/main/functions/function-manifest.js"
            CheckInv["Check Inventory"]:::function
            click CheckInv "https://github.com/twilio-labs/call-gpt/blob/main/functions/checkInventory.js"
            CheckPrice["Check Price"]:::function
            click CheckPrice "https://github.com/twilio-labs/call-gpt/blob/main/functions/checkPrice.js"
            PlaceOrder["Place Order"]:::function
            click PlaceOrder "https://github.com/twilio-labs/call-gpt/blob/main/functions/placeOrder.js"
            TransferCall["Transfer Call"]:::function
            click TransferCall "https://github.com/twilio-labs/call-gpt/blob/main/functions/transferCall.js"
        end

        subgraph "Call Handlers"
            InboundCall["Inbound Call Handler"]:::handler
            click InboundCall "https://github.com/twilio-labs/call-gpt/blob/main/scripts/inbound-call.js"
            OutboundCall["Outbound Call Handler"]:::handler
            click OutboundCall "https://github.com/twilio-labs/call-gpt/blob/main/scripts/outbound-call.js"
        end
    end

    subgraph "External Services"
        Deepgram[(Deepgram)]:::external
        OpenAI[(OpenAI)]:::external
        ElevenLabs[(Eleven Labs)]:::external
    end

    %% Connections
    Phone --> TwilioNum
    TwilioNum --> TwilioMS
    TwilioMS --> WebSocket
    WebSocket --> StreamS
    StreamS --> TranscriptS
    TranscriptS --> GPTS
    GPTS --> TTSS
    TTSS --> StreamS
    StreamS --> TwilioMS

    GPTS --> FuncManifest
    FuncManifest --> CheckInv
    FuncManifest --> CheckPrice
    FuncManifest --> PlaceOrder
    FuncManifest --> TransferCall

    TranscriptS --> Deepgram
    GPTS --> OpenAI
    TTSS --> ElevenLabs

    Express --> InboundCall
    Express --> OutboundCall

    %% Styles
    classDef service fill:#a8d1ff,stroke:#333,stroke-width:2px
    classDef function fill:#ffb366,stroke:#333,stroke-width:2px
    classDef external fill:#d1ffbd,stroke:#333,stroke-width:2px
    classDef server fill:#ff9999,stroke:#333,stroke-width:2px
    classDef handler fill:#e6b3ff,stroke:#333,stroke-width:2px

    %% Legend
    subgraph Legend
        L1["Service Component"]:::service
        L2["Function Component"]:::function
        L3["External Service"]:::external
        L4["Server Component"]:::server
        L5["Call Handler"]:::handler
    end
```
