# aws/sagemaker-distribution architecture

> Architecture diagram of the GitHub repository aws/sagemaker-distribution, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aws/sagemaker-distribution
- Repository: https://github.com/aws/sagemaker-distribution

## Overview

This repository publishes versioned SageMaker Distribution images: users pull a CPU/GPU image and work in JupyterLab or use the image for SageMaker inference. The repository also includes a CLI for package-staleness and package-size reports. Inference-server wiring is shown from the sampled v3.8 template; README documents other image capabilities, including Amazon Q Agentic Chat, but their implementation is not present in the sampled source. Report relationships are limited to the sampled CLI and SOURCE INDEX references.

Main entry points: [`template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py`](https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py) (Inference server), [`build_artifacts/v2/v2.10/v2.10.0`](https://github.com/aws/sagemaker-distribution/tree/main/build_artifacts/v2/v2.10/v2.10.0) (CPU/GPU images), [`template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/async_handler.py`](https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/async_handler.py) (Async handler), [`template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/sync_handler.py`](https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/sync_handler.py) (Sync handler)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_images["Distribution images"]
  node_image["CPU/GPU images<br/>[v2.10.0]"]
  node_jupyter["JupyterLab"]
end

subgraph group_inference["Inference serving"]
  node_inference_entry["Inference server<br/>[server.py]"]
  node_environment["Serving config<br/>[environment.py]"]
  node_async_handler["Async handler<br/>[async_handler.py]"]
  node_sync_handler["Sync handler<br/>[sync_handler.py]"]
  node_exceptions["Serving errors<br/>[exception.py]"]
  node_logger["Inference logging<br/>[logger.py]"]
  node_model_code["Model handler"]
end

subgraph group_reports["Package reports"]
  node_report_cli["Report CLI<br/>[main.py]"]
  node_config["CLI configuration<br/>[config.py]"]
  node_package_report["Package reports<br/>[package_report.py]"]
end

subgraph group_integrations["Image integrations"]
  node_q_chat["Amazon Q Chat<br/>[sagemaker-ui]"]
  node_workflow_client["Workflow client<br/>[workflow_client.py]"]
end

node_user(("Image user"))
node_sagemaker(("SageMaker endpoint"))

node_user -->|"pulls"| node_image
node_user -->|"uses"| node_jupyter
node_image -.->|"includes"| node_inference_entry
node_inference_entry -->|"reads"| node_environment
node_inference_entry -->|"dispatches"| node_async_handler
node_inference_entry -->|"dispatches"| node_sync_handler
node_inference_entry -->|"raises"| node_exceptions
node_inference_entry -->|"logs via"| node_logger
node_async_handler -->|"invokes"| node_model_code
node_sync_handler -->|"invokes"| node_model_code
node_async_handler -->|"serves routes"| node_sagemaker
node_sync_handler -->|"serves routes"| node_sagemaker
node_report_cli -->|"imports"| node_config
node_report_cli -->|"dispatches"| node_package_report
node_image -.->|"includes"| node_q_chat
node_image -.->|"includes"| node_workflow_client

click node_image "https://github.com/aws/sagemaker-distribution/tree/main/build_artifacts/v2/v2.10/v2.10.0"
click node_jupyter "https://github.com/aws/sagemaker-distribution/tree/main/template/v1"
click node_inference_entry "https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py"
click node_environment "https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/utils/environment.py"
click node_async_handler "https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/async_handler.py"
click node_sync_handler "https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/sync_handler.py"
click node_exceptions "https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/utils/exception.py"
click node_logger "https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/utils/logger.py"
click node_report_cli "https://github.com/aws/sagemaker-distribution/blob/main/src/main.py"
click node_config "https://github.com/aws/sagemaker-distribution/blob/main/src/config.py"
click node_package_report "https://github.com/aws/sagemaker-distribution/blob/main/src/package_report.py"
click node_q_chat "https://github.com/aws/sagemaker-distribution/tree/main/build_artifacts/v2/v2.10/v2.10.0/dirs/etc/sagemaker-ui"
click node_workflow_client "https://github.com/aws/sagemaker-distribution/blob/main/build_artifacts/v2/v2.10/v2.10.0/dirs/etc/sagemaker-ui/workflows/workflow_client.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_image,node_jupyter,node_user toneBlue
class node_inference_entry,node_environment,node_async_handler,node_sync_handler,node_exceptions,node_logger,node_model_code toneAmber
class node_report_cli,node_config,node_package_report toneMint
class node_q_chat,node_workflow_client toneRose
class node_sagemaker toneIndigo
```

## Components

### Distribution images

- [CPU/GPU images](https://github.com/aws/sagemaker-distribution/tree/main/build_artifacts/v2/v2.10/v2.10.0): `build_artifacts/v2/v2.10/v2.10.0`
- [JupyterLab](https://github.com/aws/sagemaker-distribution/tree/main/template/v1): `template/v1`

### Inference serving

- [Inference server](https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py): `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py`
- [Serving config](https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/utils/environment.py): `template/v3/v3.8/dirs/etc/sagemaker-inference-server/utils/environment.py`
- [Async handler](https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/async_handler.py): `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/async_handler.py`
- [Sync handler](https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/sync_handler.py): `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/sync_handler.py`
- [Serving errors](https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/utils/exception.py): `template/v3/v3.8/dirs/etc/sagemaker-inference-server/utils/exception.py`
- [Inference logging](https://github.com/aws/sagemaker-distribution/blob/main/template/v3/v3.8/dirs/etc/sagemaker-inference-server/utils/logger.py): `template/v3/v3.8/dirs/etc/sagemaker-inference-server/utils/logger.py`
- **Model handler**

### Package reports

- [Report CLI](https://github.com/aws/sagemaker-distribution/blob/main/src/main.py): `src/main.py`
- [CLI configuration](https://github.com/aws/sagemaker-distribution/blob/main/src/config.py): `src/config.py`
- [Package reports](https://github.com/aws/sagemaker-distribution/blob/main/src/package_report.py): `src/package_report.py`

### Image integrations

- [Amazon Q Chat](https://github.com/aws/sagemaker-distribution/tree/main/build_artifacts/v2/v2.10/v2.10.0/dirs/etc/sagemaker-ui): `build_artifacts/v2/v2.10/v2.10.0/dirs/etc/sagemaker-ui`
- [Workflow client](https://github.com/aws/sagemaker-distribution/blob/main/build_artifacts/v2/v2.10/v2.10.0/dirs/etc/sagemaker-ui/workflows/workflow_client.py): `build_artifacts/v2/v2.10/v2.10.0/dirs/etc/sagemaker-ui/workflows/workflow_client.py`

### Other components

- **Image user**
- **SageMaker endpoint**

## Connections

- Image user → CPU/GPU images: pulls (evidence: `README.md`)
- Image user → JupyterLab: uses (evidence: `README.md`)
- CPU/GPU images → Inference server: includes
- Inference server → Serving config: reads (evidence: `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py`)
- Inference server → Async handler: dispatches (evidence: `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py`)
- Inference server → Sync handler: dispatches (evidence: `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py`)
- Inference server → Serving errors: raises (evidence: `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py`)
- Inference server → Inference logging: logs via (evidence: `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/server.py`)
- Async handler → Model handler: invokes (evidence: `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/async_handler.py`)
- Sync handler → Model handler: invokes (evidence: `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/sync_handler.py`)
- Async handler → SageMaker endpoint: serves routes (evidence: `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/async_handler.py`)
- Sync handler → SageMaker endpoint: serves routes (evidence: `template/v3/v3.8/dirs/etc/sagemaker-inference-server/tornado_server/sync_handler.py`)
- Report CLI → CLI configuration: imports (evidence: `src/main.py`)
- Report CLI → Package reports: dispatches (evidence: `src/main.py`)
- CPU/GPU images → Amazon Q Chat: includes (evidence: `README.md`)
- CPU/GPU images → Workflow client: includes