# aws/deep-learning-containers architecture

> Architecture diagram of the GitHub repository aws/deep-learning-containers, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aws/deep-learning-containers
- Repository: https://github.com/aws/deep-learning-containers

## Overview

AWS Deep Learning Containers are prebuilt AI/ML container images for AWS workloads. The product spans framework and accelerator image variants, with SageMaker serving adapters and specialized inference services providing user-facing workflows. The clearest sampled paths are SageMaker request handling for OpenFold3 and TensorFlow, audio transcription with WhisperX, and Lambda engine startup. The tree and README establish broader image families, but unsampled files do not support detailed runtime links for most of them; those branches are represented without inferred call chains.

Main entry points: [`docker`](https://github.com/aws/deep-learning-containers/tree/main/docker) (Framework Images), [`scripts/docker/tensorflow/inference/sagemaker/python_service.py`](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/tensorflow/inference/sagemaker/python_service.py) (TF HTTP Routes), [`scripts/docker/openfold3/sagemaker_serve.py`](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/openfold3/sagemaker_serve.py) (OpenFold HTTP Adapter), [`scripts/docker/whisperx/server.py`](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/whisperx/server.py) (WhisperX API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_images["Container Images"]
  node_image_matrix["Framework Images"]
  node_base_images["CUDA Base Images"]
  node_pytorch_images["PyTorch Images"]
  node_tensorflow_images["TensorFlow Images"]
  node_other_images["Other Workloads"]
end

subgraph group_sagemaker["SageMaker Serving"]
  node_tf_manager["TF Service Manager<br/>[serve.py]"]
  node_tf_routes["TF HTTP Routes<br/>[python_service.py]"]
  node_tf_utils["TF Serving Utilities<br/>[tfs_utils.py]"]
  node_tf_mme_lock["Multi-Model Lock"]
  node_openfold_http["OpenFold HTTP Adapter<br/>[sagemaker_serve.py]"]
  node_openfold_handler["OpenFold Prediction"]
  node_autogluon_http["AutoGluon Adapter<br/>[server.py]"]
end

subgraph group_inference["Inference Services"]
  node_whisper_http["WhisperX API<br/>[server.py]"]
  node_whisper_model["Transcription Model<br/>[server.py]"]
end

subgraph group_lambda["Lambda Serving"]
  node_lambda_runtime["Lambda Runtime"]
  node_lambda_extension["Engine Extension"]
  node_lambda_handler["Invocation Handler"]
  node_engine_process["vLLM or SGLang"]
end

node_aws_user(("AWS Workload User"))

node_aws_user -->|"uses images"| node_image_matrix
node_aws_user -->|"invokes"| node_tf_routes
node_tf_manager -->|"uses utilities"| node_tf_utils
node_tf_routes -->|"parses requests"| node_tf_utils
node_tf_routes -->|"imports lock"| node_tf_mme_lock
node_aws_user -->|"invokes"| node_openfold_http
node_openfold_http -->|"calls handler"| node_openfold_handler
node_aws_user -->|"invokes"| node_autogluon_http
node_aws_user -->|"sends audio"| node_whisper_http
node_whisper_http -->|"transcribes"| node_whisper_model
node_lambda_runtime -->|"starts extension"| node_lambda_extension
node_lambda_extension -->|"starts server"| node_engine_process
node_lambda_runtime -.->|"dispatches requests"| node_lambda_handler
node_base_images -.->|"supports variants"| node_image_matrix
node_pytorch_images -.->|"provides variants"| node_image_matrix
node_tensorflow_images -.->|"provides variants"| node_image_matrix
node_other_images -.->|"provides variants"| node_image_matrix

click node_image_matrix "https://github.com/aws/deep-learning-containers/tree/main/docker"
click node_base_images "https://github.com/aws/deep-learning-containers/tree/main/docker/base"
click node_pytorch_images "https://github.com/aws/deep-learning-containers/tree/main/docker/pytorch"
click node_tensorflow_images "https://github.com/aws/deep-learning-containers/tree/main/docker/tensorflow"
click node_other_images "https://github.com/aws/deep-learning-containers/tree/main/docker"
click node_tf_manager "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/tensorflow/inference/sagemaker/serve.py"
click node_tf_routes "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/tensorflow/inference/sagemaker/python_service.py"
click node_tf_utils "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/tensorflow/inference/sagemaker/tfs_utils.py"
click node_tf_mme_lock "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/tensorflow/inference/sagemaker/multi_model_utils.py"
click node_openfold_http "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/openfold3/sagemaker_serve.py"
click node_openfold_handler "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/openfold3/sagemaker_handler.py"
click node_autogluon_http "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/autogluon/server.py"
click node_whisper_http "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/whisperx/server.py"
click node_whisper_model "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/whisperx/server.py"
click node_lambda_extension "https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/lambda/engine_extension.py"
click node_lambda_handler "https://github.com/aws/deep-learning-containers/tree/main/scripts/docker/lambda"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_image_matrix,node_base_images,node_pytorch_images,node_tensorflow_images,node_other_images,node_aws_user toneBlue
class node_tf_manager,node_tf_routes,node_tf_utils,node_tf_mme_lock,node_openfold_http,node_openfold_handler,node_autogluon_http toneAmber
class node_whisper_http,node_whisper_model toneMint
class node_lambda_runtime,node_lambda_extension,node_lambda_handler,node_engine_process toneRose
```

## Components

### Container Images

- [Framework Images](https://github.com/aws/deep-learning-containers/tree/main/docker): `docker`
- [CUDA Base Images](https://github.com/aws/deep-learning-containers/tree/main/docker/base): `docker/base`
- [PyTorch Images](https://github.com/aws/deep-learning-containers/tree/main/docker/pytorch): `docker/pytorch`
- [TensorFlow Images](https://github.com/aws/deep-learning-containers/tree/main/docker/tensorflow): `docker/tensorflow`
- [Other Workloads](https://github.com/aws/deep-learning-containers/tree/main/docker): `docker`

### SageMaker Serving

- [TF Service Manager](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/tensorflow/inference/sagemaker/serve.py): `scripts/docker/tensorflow/inference/sagemaker/serve.py`
- [TF HTTP Routes](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/tensorflow/inference/sagemaker/python_service.py): `scripts/docker/tensorflow/inference/sagemaker/python_service.py`
- [TF Serving Utilities](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/tensorflow/inference/sagemaker/tfs_utils.py): `scripts/docker/tensorflow/inference/sagemaker/tfs_utils.py`
- [Multi-Model Lock](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/tensorflow/inference/sagemaker/multi_model_utils.py): `scripts/docker/tensorflow/inference/sagemaker/multi_model_utils.py`
- [OpenFold HTTP Adapter](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/openfold3/sagemaker_serve.py): `scripts/docker/openfold3/sagemaker_serve.py`
- [OpenFold Prediction](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/openfold3/sagemaker_handler.py): `scripts/docker/openfold3/sagemaker_handler.py`
- [AutoGluon Adapter](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/autogluon/server.py): `scripts/docker/autogluon/server.py`

### Inference Services

- [WhisperX API](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/whisperx/server.py): `scripts/docker/whisperx/server.py`
- [Transcription Model](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/whisperx/server.py): `scripts/docker/whisperx/server.py`

### Lambda Serving

- **Lambda Runtime**
- [Engine Extension](https://github.com/aws/deep-learning-containers/blob/main/scripts/docker/lambda/engine_extension.py): `scripts/docker/lambda/engine_extension.py`
- [Invocation Handler](https://github.com/aws/deep-learning-containers/tree/main/scripts/docker/lambda): `scripts/docker/lambda`
- **vLLM or SGLang**

### Other components

- **AWS Workload User**

## Connections

- AWS Workload User → Framework Images: uses images
- AWS Workload User → TF HTTP Routes: invokes (evidence: `scripts/docker/tensorflow/inference/sagemaker/python_service.py`)
- TF Service Manager → TF Serving Utilities: uses utilities (evidence: `scripts/docker/tensorflow/inference/sagemaker/serve.py`)
- TF HTTP Routes → TF Serving Utilities: parses requests (evidence: `scripts/docker/tensorflow/inference/sagemaker/python_service.py`)
- TF HTTP Routes → Multi-Model Lock: imports lock (evidence: `scripts/docker/tensorflow/inference/sagemaker/python_service.py`)
- AWS Workload User → OpenFold HTTP Adapter: invokes (evidence: `scripts/docker/openfold3/sagemaker_serve.py`)
- OpenFold HTTP Adapter → OpenFold Prediction: calls handler (evidence: `scripts/docker/openfold3/sagemaker_serve.py`)
- AWS Workload User → AutoGluon Adapter: invokes (evidence: `scripts/docker/autogluon/server.py`)
- AWS Workload User → WhisperX API: sends audio (evidence: `scripts/docker/whisperx/server.py`)
- WhisperX API → Transcription Model: transcribes (evidence: `scripts/docker/whisperx/server.py`)
- Lambda Runtime → Engine Extension: starts extension (evidence: `scripts/docker/lambda/engine_extension.py`)
- Engine Extension → vLLM or SGLang: starts server (evidence: `scripts/docker/lambda/engine_extension.py`)
- Lambda Runtime → Invocation Handler: dispatches requests
- CUDA Base Images → Framework Images: supports variants
- PyTorch Images → Framework Images: provides variants
- TensorFlow Images → Framework Images: provides variants
- Other Workloads → Framework Images: provides variants
