# aws-samples/amazon-ivs-auto-captions-web-demo architecture

> Architecture diagram of the GitHub repository aws-samples/amazon-ivs-auto-captions-web-demo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aws-samples/amazon-ivs-auto-captions-web-demo
- Repository: https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo

## Overview

##### Main Components
- Frontend/User Interface: Represented by the “web-ui/player-app” directory, this is a single-page React application using the IVS Player to display video, captions, overlays, and various player components.
- API Gateway and WebSocket Infrastructure: The live caption delivery system is built on API Gateway WebSockets to push caption data to connected clients. This layer manages real-time bidirectional communication.
- AWS Lambda Functions: In the “serverless” directory, you have multiple Lambda functions to manage key events:
– Connection management functions (lambda-on-connect, lambda-on-disconnect, lambda-delete-stale-connection).
– Caption processing functions (lambda-send-transcription and lambda-send-transcription-chunks).
– Translation functions (lambda for translate-server).
- Transcription & Translation Servers: The “serverless/transcribe-server” and “serverless/translate-server” directories contain Dockerfiles and source code for containers that interface with Amazon Transcribe and Translate.
- Streaming Components: Within “serverless/stream-server” there is a Docker-based server (with Nginx, stunnel, and a validation server) that likely handles stream input or the secure routing of content.
- Configuration and Deployment: The “configuration” and “deployment” directories include various shell scripts, CloudFormation templates, and other tooling for setting up the environment, configuring custom vocabulary and overlays, and deploying the infrastructure.
- Supporting Materials: Documentation files and images (inside docs and deployment/img) provide additional context and configuration support.

##### Relationships and Data Flow
- The frontend (React web UI) connects via WebSockets (API Gateway) to the backend services.
- When a user starts a stream, captions are generated via a pipeline:
– Incoming video from IVS may get routed through an ECS container (or Lambda) that interfaces with Amazon Transcribe.
– Transcribed text is processed by Lambda functions to send caption chunks (and optionally invoking translation via another Lambda that calls Amazon Translate).
– Processed captions are then pushed through the API Gateway WebSocket layer to connected clients.
- The configuration scripts may also show a separate workflow to modify or update application settings (e.g., keywords, overlays, vocabulary filters).

##### Architectural Patterns & Technologies
- This architecture follows a microservices-serverless pattern: discrete Lambda functions handle individual tasks and are orchestrated via API Gateway.
- Containerized components (found under “serverless/transcribe-server”, “serverless/translate-server”, and “serverless/stream-server”) indicate the use of Docker/ECS for some parts of the processing pipeline.
- Frontend is a decoupled single page application (SPA) that interfaces with backend services via APIs.
- Emphasis is on real-time communication via WebSockets, making it essential to capture asynchronous interactions and event-driven flows.

##### Extra Details to Consider
- That some components (like the ECS container for IVS streaming) are intermediate points and might be depicted as “Optional/Experimental” given the outlined constraints and limitations in scaling.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Source
    IVS["IVS Stream (Amazon IVS)"]:::external

    %% Containerized Services
    subgraph "Containerized Services"
        Stream_Server["Stream Server (Nginx/stunnel/Validation)"]:::container
        Transcribe_Server["Transcribe Server"]:::container
        Translate_Server["Translate Server"]:::container
    end

    %% AWS Lambda Functions
    subgraph "AWS Lambda Functions"
        subgraph "Connection Management"
            Lambda_Connect["Lambda On Connect"]:::lambda
            Lambda_Disconnect["Lambda On Disconnect"]:::lambda
            Lambda_Delete["Lambda Delete Stale Connection"]:::lambda
        end
        subgraph "Caption Processing"
            Lambda_Transcription["Lambda Send Transcription"]:::lambda
            Lambda_TranscriptionChunks["Lambda Send Transcription Chunks"]:::lambda
            Caption_Utils["Caption Processing Utils"]:::lambda
        end
    end

    %% API Gateway & WebSocket Layer
    APIGateway["API Gateway (WebSockets)"]:::api

    %% Frontend
    Frontend["Frontend (React Player App)"]:::frontend

    %% Infrastructure & Deployment
    subgraph "Infrastructure & Deployment"
        Config_Tools["Configuration Tools"]:::infra
        Deploy_Tools["Deployment Tools"]:::infra
    end

    %% Supporting Materials
    Supporting_Materials["Supporting Materials & Docs"]:::docs

    %% Data Flow Connections
    IVS -->|"secure stream"| Stream_Server
    Stream_Server -->|"routes video"| Transcribe_Server
    Transcribe_Server -->|"transcribes audio"| Lambda_Transcription
    Lambda_Transcription -->|"processes transcription"| Lambda_TranscriptionChunks
    Lambda_TranscriptionChunks -->|"optional translation"| Translate_Server
    %% Caption Processing output flow to API Gateway
    Lambda_Transcription ---|"push caption"| APIGateway
    Lambda_TranscriptionChunks ---|"push caption"| APIGateway
    Caption_Utils ---|"supporting ops"| APIGateway

    %% Connection Management flow (for WebSocket connections)
    Lambda_Connect ---|"manage connect"| APIGateway
    Lambda_Disconnect ---|"manage disconnect"| APIGateway
    Lambda_Delete ---|"manage stale"| APIGateway

    %% API Gateway to Frontend
    APIGateway -->|"deliver captions"| Frontend

    %% Infrastructure and Deployment influence
    Config_Tools ---|"configure system"| APIGateway
    Deploy_Tools ---|"deploy infrastructure"| APIGateway

    %% Supporting Materials association
    Supporting_Materials ---|"documentation"| Frontend

    %% Click Events
    click Frontend "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/tree/main/web-ui/player-app"
    click APIGateway "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/blob/main/serverless/utils/webSocketManager.js"
    click Lambda_Connect "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/blob/main/serverless/lambda-on-connect/app.js"
    click Lambda_Disconnect "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/blob/main/serverless/lambda-on-disconnect/app.js"
    click Lambda_Delete "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/blob/main/serverless/lambda-delete-stale-connection/app.js"
    click Lambda_Transcription "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/blob/main/serverless/lambda-send-transcription/app.js"
    click Lambda_TranscriptionChunks "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/blob/main/serverless/lambda-send-transcription-chunks/app.js"
    click Caption_Utils "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/blob/main/serverless/lambda-send-transcription/utils.js"
    click Transcribe_Server "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/tree/main/serverless/transcribe-server"
    click Translate_Server "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/tree/main/serverless/translate-server"
    click Stream_Server "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/tree/main/serverless/stream-server"
    click Config_Tools "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/tree/main/configuration"
    click Deploy_Tools "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/tree/main/deployment"
    click Supporting_Materials "https://github.com/aws-samples/amazon-ivs-auto-captions-web-demo/tree/main/docs"

    %% Styles
    classDef frontend fill:#B3E5FC,stroke:#0277BD,stroke-width:2px;
    classDef api fill:#FFE082,stroke:#FFB300,stroke-width:2px;
    classDef lambda fill:#FFCC80,stroke:#FB8C00,stroke-width:2px;
    classDef container fill:#C8E6C9,stroke:#388E3C,stroke-width:2px;
    classDef infra fill:#D1C4E9,stroke:#7E57C2,stroke-width:2px;
    classDef docs fill:#F8BBD0,stroke:#AD1457,stroke-width:2px;
    classDef external fill:#FFCDD2,stroke:#E53935,stroke-width:2px;
```
