deepl/deepl-voice-api-aws-connect-demo

Purpose and principal workflow

This repository is a demo web application for bidirectional, real-time voice translation during Amazon Connect calls. The initiating actor is the agent using the browser app: it embeds the Connect contact experience, captures agent microphone audio and customer audio from the call, and coordinates transcription, translation, speech synthesis, and playback in the opposite language. The README documents DeepL Voice2Voice as the primary translation path; the sampled code also shows an AWS transcription/translation/speech configuration path. DeepL proxy Lambdas are deployed manually, separately from the CDK stacks.

Components

Browser application and call audio

• Agent browser — *(external; no repository path)* The agent operates the demo, selects languages and audio options, and grants microphone access; the README describes the demo controls and local transcription.

• Amazon Connect — *(external; no repository path)* Supplies the customer call and contact experience integrated into the demo; the README identifies Connect Streams JS and RTC JS as the integration libraries.

• Browser app orchestration — webapp/main.js Coordinates UI settings, language setup, audio managers, and the DeepL and AWS flows; excerpts show it creating audio/session managers and invoking translation and speech-language adapters.

• Audio routing and playback — webapp/managers/AudioStreamManager.js Mixes and plays audio through Web Audio, captures microphone audio, and exposes an outgoing media stream.

• Connect session tracks — webapp/managers/SessionTrackManager.js Creates microphone, file, silent, and other audio tracks and replaces the audio sender track on the Connect peer connection.

• Audio context and input testing — webapp/managers/AudioContextManager.js Manages browser AudioContext activation and lifecycle; webapp/managers/InputTestManager.js measures microphone input for the UI.

• Latency and evaluation tracking — webapp/managers/AudioLatencyTrackManager.js Tracks timing and voice-activity information and, in evaluation mode, posts translation data to a local endpoint.

Speech and translation paths

• DeepL client adapters — webapp/adapters/voiceToVoiceAdapter.js Provides the DeepL voice-to-voice client boundary; the other adapter files in webapp/adapters provide transcription, translation, and Polly operations. Their detailed request flows were not sampled.

• DeepL session proxy — lambda-functions/request-session/index.mjs Accepts a session request, selects the DeepL development or production endpoint and key, and calls the DeepL realtime voice API.

• DeepL language proxy — lambda-functions/get-languages/index.mjs Fetches DeepL language options and returns them to its caller.

• AWS service adapters — webapp/adapters/transcribeAdapter.js and webapp/adapters/translateAdapter.js Form the browser-side AWS transcription and translation boundaries; webapp/adapters/pollyAdapter.js provides Polly language, voice, and engine operations. The README documents local transcription through Amazon Transcribe, and main.js imports these adapters.

• Amazon Transcribe, Translate, and Polly — *(external; no repository path)* AWS speech services used by the AWS configuration path; the README and adapter usage identify these services.

Authentication, deployment, and evaluation

• Cognito browser authentication — webapp/utils/authUtility.js Handles Cognito authorization-code redirects, token storage/refresh, and identity credential retrieval.

• CDK deployment stacks — cdk-stacks/lib/cdk-backend-stack.ts and cdk-stacks/lib/cdk-frontend-stack.ts Define backend and frontend deployment stacks; the infrastructure and frontend stack files under cdk-stacks/lib cover Cognito and frontend configuration/deployment.

• Cognito — *(external; no repository path)* Provides the managed login and identity services used by the browser app; the README documents Cognito authentication.

• Local evaluation endpoint — webapp/server/server.js Receives evaluation posts and publishes their payloads to a Redis channel.

• Redis — *(external; no repository path)* Receives evaluation messages from the local Express endpoint.

Relationships

• Agent browser -> Browser app orchestration: starts and configures the call workflow; webapp/main.js initializes UI behavior and audio/session managers.

• Browser app orchestration -> Amazon Connect: integrates the contact experience using the Connect Streams and RTC libraries; documented in the README. Exact initialization calls are not in the sampled excerpt.

• Amazon Connect -> Browser app orchestration: makes call audio available to the browser integration; the README describes customer phone audio and Connect media streams.

• Browser app orchestration -> DeepL client adapters: invokes the DeepL voice flow; webapp/main.js streams microphone audio to DeepLVoiceClientAgent.

• DeepL client adapters -> DeepL session proxy: requests a realtime session through the proxy; the proxy handler calls DeepL, but the adapter’s exact call is not sampled.

• DeepL client adapters -> DeepL language proxy: requests language data through the proxy; the handler calls the DeepL languages endpoint. The specific client call is not sampled.

• DeepL session proxy -> DeepL API: sends a realtime voice session request; lambda-functions/request-session/index.mjs makes the HTTP request.

• DeepL language proxy -> DeepL API: fetches language options; lambda-functions/get-languages/index.mjs makes the HTTP request.

• Browser app orchestration -> AWS service adapters: loads AWS language and voice options and invokes AWS translation; shown by main.js imports and calls.

• AWS service adapters -> Amazon Transcribe, Translate, and Polly: provide the browser-side service boundary; individual network calls are not established by sampled adapter source.

• Browser app orchestration -> Audio routing and playback: manages audio streams; main.js creates and replaces stream managers.

• Browser app orchestration -> Connect session tracks: replaces the peer connection’s outgoing audio track; main.js creates a SessionTrackManager, whose implementation replaces the sender track.

• Connect session tracks -> Amazon Connect: updates the peer connection audio sender; webapp/managers/SessionTrackManager.js calls replaceTrack or addTrack.

• Browser app orchestration -> Cognito browser authentication: uses the authentication utility to manage login state; exact startup ordering is not established by the excerpts.

• Cognito browser authentication -> Cognito: exchanges authorization codes and refreshes tokens; webapp/utils/authUtility.js makes the token requests.

• CDK deployment stacks -> Cognito: define Cognito infrastructure; the Cognito stack is part of the CDK infrastructure tree.

• Audio and translation tracking -> Local evaluation endpoint: posts evaluation payloads to the localhost route; AudioLatencyTrackManager.js shows the fetch call.

• Local evaluation endpoint -> Redis: publishes the received data to evaluator-dev; webapp/server/server.js performs the Redis publish.

Coverage limits

The README and tree document more components than the sampled source excerpts expose. In particular, the Connect bootstrap, adapter internals, deployment wiring, and exact audio-to-translation-to-synthesis sequencing could not be fully verified. The README states the DeepL proxy Lambdas are not provisioned by the CDK stacks; their manual deployment and runtime connectivity are not shown here.