# opensips/opensips-ai-voice-connector-ce architecture

> Architecture diagram of the GitHub repository opensips/opensips-ai-voice-connector-ce, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opensips/opensips-ai-voice-connector-ce
- Repository: https://github.com/opensips/opensips-ai-voice-connector-ce

## Overview

##### Project type and purpose
- This is a middleware/microservice application (“AI Voice Connector”) that integrates a SIP gateway (OpenSIPS) with external AI engines for real-time voice interactions. It acts as a B2BUA media proxy, relaying SIP signaling and RTP media between user agents and AI engines.

##### File structure
- Top-level directories:
- docker/               → Dockerfile for building the connector image
- docs/                 → Architecture, AI-flavors, and deployment guides
- examples/             → Docker-compose examples showing how OpenSIPS and the connector run together
- src/                  → Core Python application
- Key Python modules in src/:
- main.py               → Entry point
- engine.py             → Core call/session management
- call.py, rtp.py       → RTP capture/injection logic
- *\_api.py             → Adapters for each AI provider (openai_api.py, deepgram_api.py, etc.)
- config.py             → Configuration loader (file + env)
- utils.py, codec.py    → Helpers and audio encoding
- examples/simple/ and examples/demo-summit/ demonstrate full Docker-Compose stacks: OpenSIPS container + AI Voice Connector container.

##### README for additional insights
- OpenSIPS handles SIP INVITEs, RTP relay to the AI Voice Connector.
- AI Voice Connector captures RTP, selects an “AI flavor” (Deepgram STT→OpenAI→Deepgram TTS, direct OpenAI speech-to-speech, Azure STT/TTS, or Deepgram native V2V).
- Media and control flows are fully asynchronous and modular.

- main.py / engine.py (orchestration)
- call.py / rtp.py (media hooks)
- ai adapters (openai_api.py, deepgram_api.py, etc.)
- codec.py (audio format conversion)
- Config passing into each subcomponent.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "SIP Layer"
        UA["Softphone (SIP UA)"]:::sip
    end

    subgraph "Connector Layer"
        OpenSIPS(("OpenSIPS Container")):::container
        AIVC(("AI Voice Connector\n(Core Python Microservice)")):::container
        UA -->|"SIP (INVITE/200 OK/ACK)"| OpenSIPS
        OpenSIPS -->|"RTP Media"| AIVC

        Engine["Engine Orchestration\n(src/engine.py)"]:::module
        Call["Call Logic\n(src/call.py)"]:::module
        RTP["RTP Module\n(src/rtp.py)"]:::module
        AIFwk["Generic AI Interface\n(src/ai.py)"]:::module
        Codec["Audio Codec Helpers\n(src/codec.py)"]:::module
        Config["Config Loader\n(src/config.py)"]:::infra
        Utils["Utility Functions\n(src/utils.py)"]:::module
        Version["Version Metadata\n(src/version.py)"]:::module

        AIVC -->|orchestrates| Engine
        Engine -->|manages| Call
        Engine -->|handles| RTP
        Engine -->|uses| AIFwk
        Engine -->|loads| Codec
        Engine -->|reads| Config
        Engine -->|calls| Utils
        Engine -->|reports version| Version

        AIFwk --> Azure["Azure API Adapter\n(src/azure_api.py)"]:::ai
        AIFwk --> Deepgram["Deepgram API Adapter\n(src/deepgram_api.py)"]:::ai
        AIFwk --> DeepgramNative["Deepgram Native V2V\n(src/deepgram_native_api.py)"]:::ai
        AIFwk --> OpenAI["OpenAI API Adapter\n(src/openai_api.py)"]:::ai
        AIFwk --> ChatGPT["ChatGPT API Adapter\n(src/chatgpt_api.py)"]:::ai
    end

    subgraph "AI Services"
        AzureSvc["Azure STT/TTS"]:::ai
        DGsvc["Deepgram STT/TTS"]:::ai
        DGv2v["Deepgram V2V"]:::ai
        OAI["OpenAI Speech"]:::ai
        ChatSvc["ChatGPT Text"]:::ai
    end

    Azure -->|REST/WebSocket| AzureSvc
    Deepgram -->|REST/WebSocket| DGsvc
    DeepgramNative -->|REST/WebSocket| DGv2v
    OpenAI -->|REST/WebSocket| OAI
    ChatGPT -->|REST API| ChatSvc

    subgraph "Infrastructure"
        ConfigFile["Configuration Store\n(env/file)"]:::infra
        Logging["Logging/Monitoring"]:::infra
        docker["Dockerfile\n(docker/Dockerfile)"]:::infra
        docs["Documentation\n(docs/ai-flavors.md & docs/ai/*.md)"]:::infra
    end

    ConfigFile -.->|provides| Config
    AIVC -.->|logs metrics| Logging
    AIVC -.->|built by| docker
    docs -->|guides| AIVC

    %% Click Events
    click AIVC "https://github.com/opensips/opensips-ai-voice-connector-ce/tree/main/src/"
    click Engine "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/engine.py"
    click Call "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/call.py"
    click RTP "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/rtp.py"
    click AIFwk "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/ai.py"
    click Azure "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/azure_api.py"
    click Deepgram "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/deepgram_api.py"
    click DeepgramNative "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/deepgram_native_api.py"
    click OpenAI "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/openai_api.py"
    click ChatGPT "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/chatgpt_api.py"
    click Codec "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/codec.py"
    click Config "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/config.py"
    click Utils "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/utils.py"
    click Version "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/src/version.py"
    click docker "https://github.com/opensips/opensips-ai-voice-connector-ce/tree/main/docker/Dockerfile"
    click docs "https://github.com/opensips/opensips-ai-voice-connector-ce/blob/main/docs/ai-flavors.md"

    %% Styles
    classDef sip fill:#DAE8FC,stroke:#6C8EBF,color:#274B7C
    classDef container fill:#FFF2CC,stroke:#D6A400,color:#000000
    classDef module fill:#E1D5E7,stroke:#9673A6,color:#000000
    classDef ai fill:#E2EFDA,stroke:#519839,color:#000000
    classDef infra fill:#F8CECC,stroke:#B85450,color:#000000
```
