# azure-samples/art-voice-agent-accelerator architecture

> Architecture diagram of the GitHub repository azure-samples/art-voice-agent-accelerator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/azure-samples/art-voice-agent-accelerator
- Repository: https://github.com/azure-samples/art-voice-agent-accelerator

## Overview

##### Project type and purpose
- This repo is a full-stack accelerator framework (“ARTVoice Accelerator Framework”) for building real-time voice agents on Azure.
- It includes:
- A backend voice pipeline (FastAPI + WebSockets) integrating ACS (Azure Communication Services), STT, LLM, TTS, orchestration, memory and tool stores.
- A React/Vite frontend demo client.
- Core libraries (speech, ACS, Redis, CosmosDB helpers, VAD, prompts…) under `src/`.
- Infrastructure-as-Code (Bicep + Terraform) under `infra/`.
- Samples, tests, docs, and dev-ops scripts.

##### File structure
- Top-level components:
- apps/rtagent/backend: FastAPI app exposing REST & WebSocket endpoints, event handlers, orchestrators, agent implementations, configs
- apps/rtagent/frontend: Vite+React demo client connecting over WebSockets & REST
- src/: reusable language-agnostic libraries (ACS helper, AOAI, VAD, pools, Redis/Cosmos managers, prompts)
- infra/: IaC definitions (Azure Bicep modules + Terraform) provisioning App Gateway, Container Apps, APIM, Redis, Cosmos DB, ACS, AI services, Key Vault, Event Grid, Log Analytics
- tests/: pytest suite and load-testing harness.
- docs/: architecture, deployment, API references, troubleshooting.
- Architectural patterns:
- Separation of concerns: transport/distribution (FastAPI & WebSockets), AI inference loop (STT→LLM→TTS), orchestration, storage, middleware
- Modular agent layers (Lvagent, artagent, foundryagents) with plugin-style factories
- Microservices containerized via Docker, orchestrated under Container Apps behind App Gateway & APIM
- Event-driven integration (Event Grid handlers, ACS media events)
- Shared state: Redis cache for sessions, Cosmos DB for persistence, Key Vault for secrets.

##### README for additional insights
- Three voice-inference modes: from-scratch, Managed (Azure Voice Live API + Foundry), BYO model.
- Emphasis on pluggable STT, TTS, LLM providers; low-latency design; multi-agent orchestration; telemetry hooks.

##### Detail without overthinking
- Break the system into as many logical sub-components as needed
- Transport, orchestration, AI loop, storage, infra, and CI/CD

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Side" 
        direction TB
        FE["Frontend Client (React/Vite)"]:::client
    end

    %% API Gateway Layer
    subgraph "Gateway Layer"
        direction TB
        APIGW["API Gateway / APIM"]:::gateway
    end

    %% Backend Layer
    subgraph "Backend Container Apps" 
        direction TB
        FG["FastAPI Service\nREST & WebSocket"]:::compute
        MAIN["Main Entry\nmain.py"]:::compute
        CFG["Configuration\nsettings"]:::compute
        WSH["WebSocket Helpers"]:::compute
        ACSINT["ACS Integration"]:::compute
        EVT_MEDIA["ACS Media Event Handlers"]:::compute
        EVT_CALL["ACS Call Lifecycle Handlers"]:::compute
        ORCH["Orchestration Core"]:::compute
        AG_ART["artagent Module"]:::compute
        AG_LIVE["liveapiagent Module"]:::compute
        AG_LV["Lvagent Implementation"]:::compute
        AG_FND["Foundry Agents"]:::compute
        SPEECH["Speech Services"]:::ai
        OPENAI["OpenAI / LLM Services"]:::ai
        REDIS["Redis Cache Manager"]:::store
        COSMOS["Cosmos DB Persistence"]:::store
    end

    %% AI Services Layer
    subgraph "AI Services" 
        direction TB
        AZ_SPEECH["Azure Speech (STT & TTS)"]:::ai
        AZ_OPENAI["Azure OpenAI / Foundry"]:::ai
        VOICE_LIVE["Azure Voice Live API"]:::ai
    end

    %% ACS PaaS
    ACS["Azure Communication Services"]:::paas

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        REDIS_DB["Redis Cache"]:::store
        COSMOS_DB["Cosmos DB"]:::store
        KV["Key Vault"]:::infra
    end

    %% Infra
    subgraph "Infrastructure-as-Code"
        direction TB
        BICEP["Bicep Definitions"]:::infra
        TF["Terraform Definitions"]:::infra
    end

    %% CI/CD
    subgraph "CI/CD & DevOps"
        direction TB
        CI["GitHub Actions Workflows"]:::infra
    end

    %% Connections
    FE -->|"HTTPS / WebSocket"| APIGW
    APIGW -->|"Forward API Calls"| FG
    FG -->|"uses"| MAIN
    MAIN --> CFG
    FG --> WSH
    FG --> ACSINT
    FG --> EVT_MEDIA
    FG --> EVT_CALL
    FG --> ORCH
    ORCH --> AG_ART
    ORCH --> AG_LIVE
    ORCH --> AG_LV
    ORCH --> AG_FND
    ORCH --> SPEECH
    ORCH --> OPENAI
    SPEECH --> AZ_SPEECH
    OPENAI --> AZ_OPENAI
    ORCH --> REDIS
    ORCH --> COSMOS
    ACSINT --> ACS
    EVT_MEDIA --> ACS
    EVT_CALL --> ACS
    REDIS --> REDIS_DB
    COSMOS --> COSMOS_DB
    FG --> KV
    BICEP --- TF
    BICEP --> FG
    TF --> FG
    CI --> FG

    %% Click Events
    click FE "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/apps/rtagent/frontend"
    click FG "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/apps/rtagent/backend/api/v1/endpoints"
    click MAIN "https://github.com/azure-samples/art-voice-agent-accelerator/blob/main/apps/rtagent/backend/main.py"
    click CFG "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/apps/rtagent/backend/config"
    click WSH "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/apps/rtagent/backend/src/ws_helpers"
    click ACSINT "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/apps/rtagent/backend/src/services/acs"
    click SPEECH "https://github.com/azure-samples/art-voice-agent-accelerator/blob/main/apps/rtagent/backend/src/services/speech_services.py"
    click OPENAI "https://github.com/azure-samples/art-voice-agent-accelerator/blob/main/apps/rtagent/backend/src/services/openai_services.py"
    click REDIS "https://github.com/azure-samples/art-voice-agent-accelerator/blob/main/apps/rtagent/backend/src/services/redis_services.py"
    click COSMOS "https://github.com/azure-samples/art-voice-agent-accelerator/blob/main/apps/rtagent/backend/src/services/cosmosdb_services.py"
    click ORCH "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/apps/rtagent/backend/src/orchestration"
    click AG_ART "https://github.com/azure-samples/art-voice-agent-accelerator/blob/main/apps/rtagent/backend/src/orchestration/artagent/orchestrator.py"
    click AG_LIVE "https://github.com/azure-samples/art-voice-agent-accelerator/blob/main/apps/rtagent/backend/src/orchestration/liveapiagent/orchestrator.py"
    click AG_LV "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/apps/rtagent/backend/src/agents/Lvagent"
    click AG_FND "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/apps/rtagent/backend/src/agents/foundryagents"
    click BICEP "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/infra/bicep"
    click TF "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/infra/terraform"
    click CI "https://github.com/azure-samples/art-voice-agent-accelerator/tree/main/.github/workflows"

    %% Styles
    classDef client fill:#D6E9FF,stroke:#4A90E2,color:#000
    classDef gateway fill:#D1F7D6,stroke:#3CB371,color:#000
    classDef compute fill:#E8F1FF,stroke:#6B8ECA,color:#000
    classDef ai fill:#FFF4E5,stroke:#FFA500,color:#000
    classDef store fill:#F3E8FF,stroke:#9370DB,color:#000
    classDef paas fill:#E8E8E8,stroke:#808080,color:#000
    classDef infra fill:#F5F5F5,stroke:#A9A9A9,color:#000
```
