How azure-samples/art-voice-agent-accelerator works
Project type and purpose
• This repo is a full-stack accelerator framework (“ARTVoice Accelerator Framework”) for building real-time voice agents on Azure.
• It includes:
• A backend voice pipeline (FastAPI + WebSockets) integrating ACS (Azure Communication Services), STT, LLM, TTS, orchestration, memory and tool stores.
• A React/Vite frontend demo client.
• Core libraries (speech, ACS, Redis, CosmosDB helpers, VAD, prompts…) under src/.
• Infrastructure-as-Code (Bicep + Terraform) under infra/.
• Samples, tests, docs, and dev-ops scripts.
File structure
• Top-level components:
• apps/rtagent/backend: FastAPI app exposing REST & WebSocket endpoints, event handlers, orchestrators, agent implementations, configs
• apps/rtagent/frontend: Vite+React demo client connecting over WebSockets & REST
• src/: reusable language-agnostic libraries (ACS helper, AOAI, VAD, pools, Redis/Cosmos managers, prompts)
• infra/: IaC definitions (Azure Bicep modules + Terraform) provisioning App Gateway, Container Apps, APIM, Redis, Cosmos DB, ACS, AI services, Key Vault, Event Grid, Log Analytics
• tests/: pytest suite and load-testing harness.
• docs/: architecture, deployment, API references, troubleshooting.
• Architectural patterns:
• Separation of concerns: transport/distribution (FastAPI & WebSockets), AI inference loop (STT→LLM→TTS), orchestration, storage, middleware
• Modular agent layers (Lvagent, artagent, foundryagents) with plugin-style factories
• Microservices containerized via Docker, orchestrated under Container Apps behind App Gateway & APIM
• Event-driven integration (Event Grid handlers, ACS media events)
• Shared state: Redis cache for sessions, Cosmos DB for persistence, Key Vault for secrets.
README for additional insights
• Three voice-inference modes: from-scratch, Managed (Azure Voice Live API + Foundry), BYO model.
• Emphasis on pluggable STT, TTS, LLM providers; low-latency design; multi-agent orchestration; telemetry hooks.
Detail without overthinking
• Break the system into as many logical sub-components as needed
• Transport, orchestration, AI loop, storage, infra, and CI/CD