# deepl/deepl-voice-api-aws-connect-demo architecture

> Architecture diagram of the GitHub repository deepl/deepl-voice-api-aws-connect-demo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deepl/deepl-voice-api-aws-connect-demo
- Repository: https://github.com/deepl/deepl-voice-api-aws-connect-demo

## Overview

#### Purpose and principal workflow

This repository is a demo web application for bidirectional, real-time voice translation during Amazon Connect calls. The initiating actor is the agent using the browser app: it embeds the Connect contact experience, captures agent microphone audio and customer audio from the call, and coordinates transcription, translation, speech synthesis, and playback in the opposite language. The README documents DeepL Voice2Voice as the primary translation path; the sampled code also shows an AWS transcription/translation/speech configuration path. DeepL proxy Lambdas are deployed manually, separately from the CDK stacks.

#### Components

##### Browser application and call audio
- **Agent browser** — *(external; no repository path)* The agent operates the demo, selects languages and audio options, and grants microphone access; the README describes the demo controls and local transcription.
- **Amazon Connect** — *(external; no repository path)* Supplies the customer call and contact experience integrated into the demo; the README identifies Connect Streams JS and RTC JS as the integration libraries.
- **Browser app orchestration** — `webapp/main.js` Coordinates UI settings, language setup, audio managers, and the DeepL and AWS flows; excerpts show it creating audio/session managers and invoking translation and speech-language adapters.
- **Audio routing and playback** — `webapp/managers/AudioStreamManager.js` Mixes and plays audio through Web Audio, captures microphone audio, and exposes an outgoing media stream.
- **Connect session tracks** — `webapp/managers/SessionTrackManager.js` Creates microphone, file, silent, and other audio tracks and replaces the audio sender track on the Connect peer connection.
- **Audio context and input testing** — `webapp/managers/AudioContextManager.js` Manages browser AudioContext activation and lifecycle; `webapp/managers/InputTestManager.js` measures microphone input for the UI.
- **Latency and evaluation tracking** — `webapp/managers/AudioLatencyTrackManager.js` Tracks timing and voice-activity information and, in evaluation mode, posts translation data to a local endpoint.

##### Speech and translation paths
- **DeepL client adapters** — `webapp/adapters/voiceToVoiceAdapter.js` Provides the DeepL voice-to-voice client boundary; the other adapter files in `webapp/adapters` provide transcription, translation, and Polly operations. Their detailed request flows were not sampled.
- **DeepL session proxy** — `lambda-functions/request-session/index.mjs` Accepts a session request, selects the DeepL development or production endpoint and key, and calls the DeepL realtime voice API.
- **DeepL language proxy** — `lambda-functions/get-languages/index.mjs` Fetches DeepL language options and returns them to its caller.
- **AWS service adapters** — `webapp/adapters/transcribeAdapter.js` and `webapp/adapters/translateAdapter.js` Form the browser-side AWS transcription and translation boundaries; `webapp/adapters/pollyAdapter.js` provides Polly language, voice, and engine operations. The README documents local transcription through Amazon Transcribe, and `main.js` imports these adapters.
- **Amazon Transcribe, Translate, and Polly** — *(external; no repository path)* AWS speech services used by the AWS configuration path; the README and adapter usage identify these services.

##### Authentication, deployment, and evaluation
- **Cognito browser authentication** — `webapp/utils/authUtility.js` Handles Cognito authorization-code redirects, token storage/refresh, and identity credential retrieval.
- **CDK deployment stacks** — `cdk-stacks/lib/cdk-backend-stack.ts` and `cdk-stacks/lib/cdk-frontend-stack.ts` Define backend and frontend deployment stacks; the infrastructure and frontend stack files under `cdk-stacks/lib` cover Cognito and frontend configuration/deployment.
- **Cognito** — *(external; no repository path)* Provides the managed login and identity services used by the browser app; the README documents Cognito authentication.
- **Local evaluation endpoint** — `webapp/server/server.js` Receives evaluation posts and publishes their payloads to a Redis channel.
- **Redis** — *(external; no repository path)* Receives evaluation messages from the local Express endpoint.

#### Relationships

- **Agent browser -> Browser app orchestration**: starts and configures the call workflow; `webapp/main.js` initializes UI behavior and audio/session managers.
- **Browser app orchestration -> Amazon Connect**: integrates the contact experience using the Connect Streams and RTC libraries; documented in the README. Exact initialization calls are not in the sampled excerpt.
- **Amazon Connect -> Browser app orchestration**: makes call audio available to the browser integration; the README describes customer phone audio and Connect media streams.
- **Browser app orchestration -> DeepL client adapters**: invokes the DeepL voice flow; `webapp/main.js` streams microphone audio to `DeepLVoiceClientAgent`.
- **DeepL client adapters -> DeepL session proxy**: requests a realtime session through the proxy; the proxy handler calls DeepL, but the adapter’s exact call is not sampled.
- **DeepL client adapters -> DeepL language proxy**: requests language data through the proxy; the handler calls the DeepL languages endpoint. The specific client call is not sampled.
- **DeepL session proxy -> DeepL API**: sends a realtime voice session request; `lambda-functions/request-session/index.mjs` makes the HTTP request.
- **DeepL language proxy -> DeepL API**: fetches language options; `lambda-functions/get-languages/index.mjs` makes the HTTP request.
- **Browser app orchestration -> AWS service adapters**: loads AWS language and voice options and invokes AWS translation; shown by `main.js` imports and calls.
- **AWS service adapters -> Amazon Transcribe, Translate, and Polly**: provide the browser-side service boundary; individual network calls are not established by sampled adapter source.
- **Browser app orchestration -> Audio routing and playback**: manages audio streams; `main.js` creates and replaces stream managers.
- **Browser app orchestration -> Connect session tracks**: replaces the peer connection’s outgoing audio track; `main.js` creates a `SessionTrackManager`, whose implementation replaces the sender track.
- **Connect session tracks -> Amazon Connect**: updates the peer connection audio sender; `webapp/managers/SessionTrackManager.js` calls `replaceTrack` or `addTrack`.
- **Browser app orchestration -> Cognito browser authentication**: uses the authentication utility to manage login state; exact startup ordering is not established by the excerpts.
- **Cognito browser authentication -> Cognito**: exchanges authorization codes and refreshes tokens; `webapp/utils/authUtility.js` makes the token requests.
- **CDK deployment stacks -> Cognito**: define Cognito infrastructure; the Cognito stack is part of the CDK infrastructure tree.
- **Audio and translation tracking -> Local evaluation endpoint**: posts evaluation payloads to the localhost route; `AudioLatencyTrackManager.js` shows the fetch call.
- **Local evaluation endpoint -> Redis**: publishes the received data to `evaluator-dev`; `webapp/server/server.js` performs the Redis publish.

#### Coverage limits

The README and tree document more components than the sampled source excerpts expose. In particular, the Connect bootstrap, adapter internals, deployment wiring, and exact audio-to-translation-to-synthesis sequencing could not be fully verified. The README states the DeepL proxy Lambdas are not provisioned by the CDK stacks; their manual deployment and runtime connectivity are not shown here.

Main entry points: [`webapp/main.js`](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/main.js) (Browser app), [`webapp/adapters/voiceToVoiceAdapter.js`](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/adapters/voiceToVoiceAdapter.js) (DeepL client adapters), [`webapp/managers/SessionTrackManager.js`](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/managers/SessionTrackManager.js) (Connect session tracks), [`lambda-functions/request-session/index.mjs`](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/lambda-functions/request-session/index.mjs) (DeepL session proxy)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_browser_call["Browser and Call Audio"]
  node_agent["Agent browser<br/>External actor"]
  node_connect["Amazon Connect<br/>External service"]
  node_browser_app["Browser app<br/>Web application"]
  node_audio_routing["Audio routing<br/>Audio manager"]
  node_session_tracks["Connect session tracks<br/>Audio manager"]
  node_latency_tracking["Audio latency tracking<br/>Evaluation manager"]
end

subgraph group_deepl["DeepL Translation"]
  node_deepl_adapters["DeepL client adapters"]
  node_session_proxy["DeepL session proxy<br/>Lambda"]
  node_language_proxy["DeepL language proxy<br/>Lambda"]
  node_deepl_api["DeepL API<br/>External service"]
end

subgraph group_aws_speech["AWS Speech Services"]
  node_aws_adapters["AWS service adapters<br/>Client adapters"]
  node_aws_services["Transcribe, Translate, Polly<br/>External services"]
end

subgraph group_platform["Authentication and Evaluation"]
  node_auth_utility["Cognito browser auth<br/>Authentication utility"]
  node_cognito["Cognito<br/>External service"]
  node_cdk_stacks["CDK deployment stacks<br/>Infrastructure"]
  node_evaluation_endpoint["Local evaluation endpoint<br/>Express server"]
  node_redis[("Redis<br/>External data store")]
end

node_agent -->|"starts and configures"| node_browser_app
node_browser_app -->|"integrates contact experience"| node_connect
node_connect -->|"provides call audio"| node_browser_app
node_browser_app -->|"invokes voice flow"| node_deepl_adapters
node_deepl_adapters -->|"requests session"| node_session_proxy
node_deepl_adapters -->|"requests languages"| node_language_proxy
node_session_proxy -->|"requests voice session"| node_deepl_api
node_language_proxy -->|"fetches languages"| node_deepl_api
node_browser_app -->|"loads options and invokes translation"| node_aws_adapters
node_aws_adapters -->|"service boundary"| node_aws_services
node_browser_app -->|"manages audio streams"| node_audio_routing
node_browser_app -->|"replaces outgoing track"| node_session_tracks
node_session_tracks -->|"updates audio sender"| node_connect
node_browser_app -->|"uses authentication"| node_auth_utility
node_auth_utility -->|"exchanges and refreshes tokens"| node_cognito
node_cdk_stacks -->|"defines infrastructure"| node_cognito
node_latency_tracking -->|"posts evaluation data"| node_evaluation_endpoint
node_evaluation_endpoint -->|"publishes messages"| node_redis

click node_browser_app "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/main.js"
click node_audio_routing "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/managers/AudioStreamManager.js"
click node_session_tracks "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/managers/SessionTrackManager.js"
click node_latency_tracking "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/managers/AudioLatencyTrackManager.js"
click node_deepl_adapters "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/adapters/voiceToVoiceAdapter.js"
click node_session_proxy "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/lambda-functions/request-session/index.mjs"
click node_language_proxy "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/lambda-functions/get-languages/index.mjs"
click node_aws_adapters "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/adapters/transcribeAdapter.js"
click node_auth_utility "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/utils/authUtility.js"
click node_cdk_stacks "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/cdk-stacks/lib/cdk-backend-stack.ts"
click node_evaluation_endpoint "https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/server/server.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_agent,node_connect,node_browser_app,node_audio_routing,node_session_tracks,node_latency_tracking toneBlue
class node_deepl_adapters,node_session_proxy,node_language_proxy,node_deepl_api toneAmber
class node_aws_adapters,node_aws_services toneMint
class node_auth_utility,node_cognito,node_cdk_stacks,node_evaluation_endpoint,node_redis toneRose
```

## Components

### Browser and Call Audio

- **Agent browser**: External actor
- **Amazon Connect**: External service
- [Browser app](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/main.js): Web application · `webapp/main.js`
- [Audio routing](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/managers/AudioStreamManager.js): Audio manager · `webapp/managers/AudioStreamManager.js`
- [Connect session tracks](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/managers/SessionTrackManager.js): Audio manager · `webapp/managers/SessionTrackManager.js`
- [Audio latency tracking](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/managers/AudioLatencyTrackManager.js): Evaluation manager · `webapp/managers/AudioLatencyTrackManager.js`

### DeepL Translation

- [DeepL client adapters](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/adapters/voiceToVoiceAdapter.js): Client adapters · `webapp/adapters/voiceToVoiceAdapter.js`
- [DeepL session proxy](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/lambda-functions/request-session/index.mjs): Lambda · `lambda-functions/request-session/index.mjs`
- [DeepL language proxy](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/lambda-functions/get-languages/index.mjs): Lambda · `lambda-functions/get-languages/index.mjs`
- **DeepL API**: External service

### AWS Speech Services

- [AWS service adapters](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/adapters/transcribeAdapter.js): Client adapters · Also includes webapp/adapters/translateAdapter.js and webapp/adapters/pollyAdapter.js. · `webapp/adapters/transcribeAdapter.js`
- **Transcribe, Translate, Polly**: External services

### Authentication and Evaluation

- [Cognito browser auth](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/utils/authUtility.js): Authentication utility · `webapp/utils/authUtility.js`
- **Cognito**: External service
- [CDK deployment stacks](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/cdk-stacks/lib/cdk-backend-stack.ts): Infrastructure · Also includes cdk-stacks/lib/cdk-frontend-stack.ts. · `cdk-stacks/lib/cdk-backend-stack.ts`
- [Local evaluation endpoint](https://github.com/deepl/deepl-voice-api-aws-connect-demo/blob/main/webapp/server/server.js): Express server · `webapp/server/server.js`
- **Redis**: External data store

## Connections

- Agent browser → Browser app: starts and configures
- Browser app → Amazon Connect: integrates contact experience
- Amazon Connect → Browser app: provides call audio
- Browser app → DeepL client adapters: invokes voice flow
- DeepL client adapters → DeepL session proxy: requests session
- DeepL client adapters → DeepL language proxy: requests languages
- DeepL session proxy → DeepL API: requests voice session
- DeepL language proxy → DeepL API: fetches languages
- Browser app → AWS service adapters: loads options and invokes translation
- AWS service adapters → Transcribe, Translate, Polly: service boundary
- Browser app → Audio routing: manages audio streams
- Browser app → Connect session tracks: replaces outgoing track
- Connect session tracks → Amazon Connect: updates audio sender
- Browser app → Cognito browser auth: uses authentication
- Cognito browser auth → Cognito: exchanges and refreshes tokens
- CDK deployment stacks → Cognito: defines infrastructure
- Audio latency tracking → Local evaluation endpoint: posts evaluation data
- Local evaluation endpoint → Redis: publishes messages