# aws/amazon-chime-sdk-js architecture

> Architecture diagram of the GitHub repository aws/amazon-chime-sdk-js, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aws/amazon-chime-sdk-js
- Repository: https://github.com/aws/amazon-chime-sdk-js

## Overview

##### Project type and purpose
- This is a TypeScript/JavaScript SDK (library) for real-time communications: audio, video, messaging, screen sharing, and meeting readiness checks.
- The core deliverable is the `amazon-chime-sdk-js` library (in `src/`), with demo applications under `demos/browser` and `demos/serverless`, extensive documentation in `docs/` and guides in `guides/`, plus integration tests under `integration/` and unit tests under `test/`.

##### File structure
- Top-level components:
- `src/` – the SDK implementation, broken into modules (audioVideoController, deviceController, signalingclient, messagingSession, tasks, videoframeprocessor, voicefocus, etc.).
- `demos/` – sample client applications illustrating usage.
- `integration/` and `test/` – automated integration tests and unit tests.
- `docs/`, `guides/` – generated API references and usage guides.
- `script/` and `.github/` – build, CI/CD, release, and demo-deploy automation.
- Architectural hints:
- The SDK uses a facade/controller pattern (`AudioVideoFacade` → `AudioVideoController`), task pipelines (`task/`), observer patterns for events (`AudioVideoObserver`, `ContentShareObserver`), and strategy/policy modules (simulcast, bandwidth policies).
- Signaling and metrics go through `DefaultSignalingClient` over WebSocket.
- Media processing pipelines (e.g., background blur, video FX, voice focus) plug into the video frames via `VideoFrameProcessorPipeline`.
- Messaging uses `DefaultMessagingSession` built atop AWS SDK’s `ChimeSDKMessagingClient`.

##### README for additional insights
- The README confirms it’s a client-side SDK that connects to AWS Chime infrastructure for sessions (meetings, messaging).
- External dependencies: WebRTC (browser API), AWS Chime Signaling service, AWS Chime SDK Meetings REST/control plane, optional AWS Transcribe for live transcription, TensorFlow.js models for background effects, Amazon Voice Focus library.

##### Main components
a. SDK Core Modules (in `src/`):
- Audio/Video Facade & Controller
- Device Controller & Media Device Factory
- Signaling Client (WebSocket)
- Task Pipeline (ICE, SDP, peer connections)
- Video Frame Processor Pipeline (background blur, video FX)
- Voice Focus module
- Messaging Session
- Meeting Readiness Checker
- Event Reporter & Metrics Collector
b. Demo Applications (under `demos/`):
- Browser meeting demo (Node/Express server + Web UI)
- Serverless demo (AWS CloudFormation template + Lambda handlers)
c. External Services:
- AWS Chime Signaling Endpoint (WebSocket)
- AWS Chime REST API (CreateMeeting/CreateAttendee)
- AWS Chime Messaging API
- AWS Transcribe (optional)
d. Build & CI/CD:
- GitHub Actions workflows (`.github/workflows/`)
- Build/test scripts (`script/`)
- Documentation generator (`typedoc`)

##### Relationships and interactions
- User’s Web App → invokes SDK Facade →
- DeviceController → browser MediaDevices → MediaStream →
- AudioVideoController → WebRTC RTCPeerConnection → SignalingClient (WebSocket) → AWS Chime Signaling Service → other attendees
- VideoFrameProcessorPipeline intercepts outgoing video frames → applies background blur/voice focus via TensorFlow.js or WebWorker → encoded and sent
- MessagingSession → AWS SDK client → AWS Chime Messaging WebSocket/HTTP endpoint
- SDK emits events (observer pattern) to the UI, and logs/metrics to CloudWatch or custom POST endpoints

##### Important architectural patterns and design principles
- Facade pattern: `AudioVideoFacade` hides complex subsystems
- Strategy/Policy pattern: bandwidth policies, simulcast, active speaker detection
- Observer pattern: event callbacks for session lifecycle, volume indicators, tile updates
- Task pipeline: discrete `Task` entities coordinate asynchronous operations (ICE, SDP, gathering)
- Dependency injection: pluggable `Logger`, `BrowserBehavior`, `MediaDeviceFactory`, `WebSocketAdapter`
- Modular, layered design separating concerns (UI demos vs. transport vs. media processing vs. business logic)

##### Relevant technologies, frameworks, and libraries
- TypeScript, Node.js, npm
- WebRTC APIs (RTCPeerConnection, getUserMedia)
- WebSocket (via `DefaultWebSocketAdapter`)
- AWS SDK v3 (`@aws-sdk/client-chime-sdk-messaging`)
- TensorFlow.js models for video segmentation (in demos)
- GitHub Actions, Mocha/Chai for testing
- CloudFormation/SAM for serverless demo

##### Elements
- Clear labels for each component (e.g., AudioVideoController, SignalingClient, VideoFrameProcessorPipeline, DefaultMessagingSession)
- UI → SDK Facade → Controllers → Browser APIs/AWS Services
- SDK → Observers → UI

## Diagram (Mermaid)

```mermaid
graph TD
    %% Client Applications
    subgraph "Client Applications"
        direction TB
        BrowserDemo["Browser Meeting Demo"]:::frontend
        ServerlessDemo["Serverless Demo"]:::frontend
    end

    %% SDK Core Modules
    subgraph "SDK Core" 
        direction TB

        subgraph "Facades"
            AudioVideoFacade["AudioVideoFacade"]:::core
            click AudioVideoFacade "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/audiovideofacade/AudioVideoFacade.ts"
            DefaultAudioVideoFacade["DefaultAudioVideoFacade"]:::core
            click DefaultAudioVideoFacade "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/audiovideofacade/DefaultAudioVideoFacade.ts"
        end

        subgraph "Controllers"
            AudioVideoController["AudioVideoController"]:::core
            click AudioVideoController "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/audiovideocontroller/AudioVideoController.ts"
            DefaultAudioVideoController["DefaultAudioVideoController"]:::core
            click DefaultAudioVideoController "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/audiovideocontroller/DefaultAudioVideoController.ts"
            AudioVideoControllerFacade["AudioVideoControllerFacade"]:::core
            click AudioVideoControllerFacade "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/audiovideocontroller/AudioVideoControllerFacade.ts"
        end

        subgraph "Device Management"
            DeviceController["DeviceController"]:::core
            click DeviceController "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/devicecontroller/DeviceController.ts"
            DefaultDeviceController["DefaultDeviceController"]:::core
            click DefaultDeviceController "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/devicecontroller/DefaultDeviceController.ts"
            MediaDeviceFactory["MediaDeviceFactory"]:::core
            click MediaDeviceFactory "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/mediadevicefactory/MediaDeviceFactory.ts"
            DefaultMediaDeviceFactory["DefaultMediaDeviceFactory"]:::core
            click DefaultMediaDeviceFactory "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/mediadevicefactory/DefaultMediaDeviceFactory.ts"
        end

        subgraph "Signaling & Tasks"
            SignalingClient["DefaultSignalingClient"]:::core
            click SignalingClient "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/signalingclient/DefaultSignalingClient.ts"
            SignalingClientInterface["SignalingClient"]:::core
            click SignalingClientInterface "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/signalingclient/SignalingClient.ts"
            SignalingClientEvent["SignalingClientEvent"]:::core
            click SignalingClientEvent "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/signalingclient/SignalingClientEvent.ts"
            TaskPipeline["Task Pipeline"]:::core
            click TaskPipeline "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/task/Task.ts"
            BaseTask["BaseTask"]:::core
            click BaseTask "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/task/BaseTask.ts"
            AttachMediaTask["AttachMediaInputTask"]:::core
            click AttachMediaTask "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/task/AttachMediaInputTask.ts"
            OpenSignalingTask["OpenSignalingConnectionTask"]:::core
            click OpenSignalingTask "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/task/OpenSignalingConnectionTask.ts"
            SetLocalDescTask["SetLocalDescriptionTask"]:::core
            click SetLocalDescTask "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/task/SetLocalDescriptionTask.ts"
            FinishICE["FinishGatheringICECandidatesTask"]:::core
            click FinishICE "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/task/FinishGatheringICECandidatesTask.ts"
        end

        subgraph "Media Processing"
            VideoPipeline["VideoFrameProcessorPipeline"]:::core
            click VideoPipeline "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/videoframeprocessor/VideoFrameProcessorPipeline.ts"
            DefaultVideoPipeline["DefaultVideoFrameProcessorPipeline"]:::core
            click DefaultVideoPipeline "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/videoframeprocessor/DefaultVideoFrameProcessorPipeline.ts"
            VideoProcessor["VideoFrameProcessor"]:::core
            click VideoProcessor "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/videoframeprocessor/VideoFrameProcessor.ts"
            VoiceFocusSpec["VoiceFocusSpec"]:::core
            click VoiceFocusSpec "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/voicefocus/VoiceFocusSpec.ts"
            VoiceFocusDevice["VoiceFocusTransformDevice"]:::core
            click VoiceFocusDevice "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/voicefocus/VoiceFocusTransformDevice.ts"
            VoiceFocusDelegate["VoiceFocusTransformDeviceDelegate"]:::core
            click VoiceFocusDelegate "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/voicefocus/VoiceFocusTransformDeviceDelegate.ts"
        end

        subgraph "Messaging"
            MessagingSession["DefaultMessagingSession"]:::core
            click MessagingSession "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/messagingsession/DefaultMessagingSession.ts"
            MessagingInterface["MessagingSession"]:::core
            click MessagingInterface "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/messagingsession/MessagingSession.ts"
            MessagingConfig["MessagingSessionConfiguration"]:::core
            click MessagingConfig "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/messagingsession/MessagingSessionConfiguration.ts"
        end

        subgraph "Readiness & Metrics"
            MeetingChecker["DefaultMeetingReadinessChecker"]:::core
            click MeetingChecker "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/meetingreadinesschecker/DefaultMeetingReadinessChecker.ts"
            MeetingConfig["MeetingReadinessCheckerConfiguration"]:::core
            click MeetingConfig "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/meetingreadinesschecker/MeetingReadinessCheckerConfiguration.ts"
            EventReporter["DefaultMeetingEventReporter"]:::core
            click EventReporter "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/eventreporter/DefaultMeetingEventReporter.ts"
            ClientMetric["ClientMetricReport"]:::core
            click ClientMetric "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/clientmetricreport/ClientMetricReport.ts"
            GlobalMetric["GlobalMetricReport"]:::core
            click GlobalMetric "https://github.com/aws/amazon-chime-sdk-js/blob/main/src/clientmetricreport/GlobalMetricReport.ts"
        end
    end

    %% Browser APIs
    subgraph "Browser APIs"
        direction TB
        MediaDevices["MediaDevices API"]:::browserapi
        RTCPeerConnection["RTCPeerConnection"]:::browserapi
        WebSocketAdapter["DefaultWebSocketAdapter"]:::browserapi
    end

    %% AWS Services
    subgraph "AWS Services"
        direction TB
        ChimeWS["Chime Signaling (WebSocket)"]:::aws
        ChimeREST["Chime REST API"]:::aws
        ChimeMsgAPI["Chime Messaging API"]:::aws
        Transcribe["AWS Transcribe"]:::aws
    end

    %% Connections
    BrowserDemo -->|"uses"| AudioVideoFacade
    BrowserDemo -->|"uses"| MessagingSession
    BrowserDemo -->|"uses"| MeetingChecker

    ServerlessDemo -->|"invokes"| MessagingSession
    ServerlessDemo -->|"invokes"| MeetingChecker

    AudioVideoFacade -->|"controls"| AudioVideoController
    AudioVideoControllerFacade --> AudioVideoController
    AudioVideoController -->|"calls"| DeviceController
    DeviceController -->|"factory"| MediaDeviceFactory
    AudioVideoController -->|"manages"| RTCPeerConnection
    AudioVideoController -->|"opens"| SignalingClient

    VideoPipeline -->|"processes frames"| VideoProcessor
    VideoPipeline -->|"integrates"| VoiceFocusDevice
    VideoPipeline -->|"runs on"| WebSocketAdapter

    MessagingSession -->|"sends messages via"| ChimeMsgAPI
    MessagingSession -->|"uses client"| MessagingInterface

    SignalingClient -->|"connects to"| ChimeWS
    AudioVideoController -->|"POST /control"| ChimeREST

    MediaDevices -->|"provide stream"| AudioVideoController
    RTCPeerConnection -->|"send/receive media"| AudioVideoController

    MeetingChecker -->|"checks via REST"| ChimeREST
    EventReporter -->|"reports to"| ChimeREST
    ClientMetric --> EventReporter
    GlobalMetric --> EventReporter

    Transcribe -->|"optional transcription"| AudioVideoController

    %% Styling
    classDef frontend fill:#c2f0c2,stroke:#2d7f4f,color:#000
    classDef core fill:#c2d9f8,stroke:#1f4e79,color:#000
    classDef browserapi fill:#fde9c4,stroke:#b36b00,color:#000
    classDef aws fill:#f8d7da,stroke:#a71d2a,color:#000
```
