# microsoft/onnxruntime architecture

> Architecture diagram of the GitHub repository microsoft/onnxruntime, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/onnxruntime
- Repository: https://github.com/microsoft/onnxruntime

## Overview

ONNX Runtime is a cross-platform inference and training accelerator. The principal workflow is an application supplying an ONNX model and inputs to a session, which runs inference and returns outputs; hardware execution providers and graph optimization support that runtime. A separate training path accelerates PyTorch training. The graph also preserves documented JavaScript browser/Node backends and the Stable Diffusion optimization and inference branches. Edges reflect README statements or the supplied source index and excerpts; unsampled implementation details are not inferred.

Main entry points: [`onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py) (Diffusion Pipeline), [`onnxruntime/core/session/inference_session.cc`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/session/inference_session.cc) (Inference Session), [`js/common/lib/inference-session.ts`](https://github.com/microsoft/onnxruntime/blob/main/js/common/lib/inference-session.ts) (JS Session API), [`js/node/src/inference_session_wrap.cc`](https://github.com/microsoft/onnxruntime/blob/main/js/node/src/inference_session_wrap.cc) (Node Binding)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Application APIs"]
  node_js_api["JS Session API"]
  node_node_binding["Node Binding"]
  node_wasm_handler["WASM Handler"]
  node_webgl_handler["WebGL Handler"]
  node_webgl_compute["WebGL Compute"]
  node_winml_engine["WinML Engine"]
  node_web_tensor["Web Tensor<br/>[tensor.ts]"]
  node_go_api["Go API"]
end

subgraph group_runtime["Core Runtime"]
  node_session["Inference Session"]
  node_c_api["C API"]
  node_cpp_api["C++ API"]
  node_cpp_inline["C++ API Inline"]
  node_status["Status Handling<br/>[status.h]"]
  node_common["Common Types<br/>[common.h]"]
  node_data_types["Data Types<br/>[data_types.h]"]
  node_internal_types["Internal Data Types"]
  node_data_transfer["Data Transfer"]
end

subgraph group_providers["Execution Providers"]
  node_cuda_qmoe["CUDA MoE Offload"]
  node_qnn_backend["QNN Backend"]
end

subgraph group_training["Training"]
  node_training_inference["ORTModule Inference"]
  node_training_pipeline["Training Pipeline<br/>[pipeline.cc]"]
  node_pipeline_transform["Pipeline Transformer"]
end

subgraph group_diffusion["Stable Diffusion"]
  node_sd_pipeline["Diffusion Pipeline"]
  node_sd_models["Diffusion Models"]
  node_sd_schedulers["Diffusion Schedulers"]
  node_sd_engine["Engine Builders<br/>[engine_builder.py]"]
  node_sd_optimize["Model Optimization"]
end

node_application(("Application"))

node_application -->|"creates session"| node_js_api
node_js_api -.->|"uses binding"| node_node_binding
node_node_binding -->|"calls API"| node_cpp_api
node_js_api -.->|"dispatches backend"| node_wasm_handler
node_wasm_handler -->|"converts tensors"| node_web_tensor
node_application -.->|"runs inference"| node_webgl_handler
node_webgl_handler -->|"converts tensors"| node_web_tensor
node_webgl_handler -->|"runs session"| node_webgl_compute
node_application -.->|"runs inference"| node_winml_engine
node_winml_engine -->|"runs session"| node_session
node_session -.->|"uses API"| node_c_api
node_node_binding -->|"constructs session"| node_session
node_go_api -.->|"declares C interface"| node_c_api
node_application -->|"trains model"| node_training_inference
node_training_inference -.->|"runs forward pass"| node_training_pipeline
node_pipeline_transform -->|"transforms pipeline"| node_training_pipeline
node_application -.->|"generates images"| node_sd_pipeline
node_sd_pipeline -->|"uses models"| node_sd_models
node_sd_pipeline -->|"uses scheduler"| node_sd_schedulers
node_sd_pipeline -->|"runs engines"| node_sd_engine
node_sd_optimize -.->|"optimizes models"| node_sd_pipeline
node_qnn_backend -.->|"provides backend"| node_session
node_cuda_qmoe -.->|"supports session runs"| node_session
node_data_transfer -->|"transfers tensors"| node_data_types
node_internal_types -.->|"extends types"| node_data_types

click node_js_api "https://github.com/microsoft/onnxruntime/blob/main/js/common/lib/inference-session.ts"
click node_node_binding "https://github.com/microsoft/onnxruntime/blob/main/js/node/src/inference_session_wrap.cc"
click node_wasm_handler "https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/wasm/session-handler-inference.ts"
click node_webgl_handler "https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/onnxjs/session-handler-inference.ts"
click node_webgl_compute "https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/onnxjs/backends/webgl/inference-handler.ts"
click node_winml_engine "https://github.com/microsoft/onnxruntime/blob/main/winml/lib/Api.Ort/OnnxruntimeEngine.cpp"
click node_session "https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/session/inference_session.cc"
click node_c_api "https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h"
click node_cpp_api "https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_cxx_api.h"
click node_cpp_inline "https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_cxx_inline.h"
click node_status "https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/common/status.h"
click node_common "https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/common/common.h"
click node_data_types "https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/framework/data_types.h"
click node_internal_types "https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/framework/data_types_internal.h"
click node_data_transfer "https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/framework/data_transfer_manager.cc"
click node_web_tensor "https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/onnxjs/tensor.ts"
click node_go_api "https://github.com/microsoft/onnxruntime/blob/main/go/onnxruntime/onnxruntime_c_api.h"
click node_cuda_qmoe "https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/providers/cuda/qmoe_session_controller.h"
click node_qnn_backend "https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/providers/qnn/builder/qnn_backend_manager.cc"
click node_training_inference "https://github.com/microsoft/onnxruntime/blob/main/orttraining/orttraining/python/training/ortmodule/_inference_manager.py"
click node_training_pipeline "https://github.com/microsoft/onnxruntime/blob/main/orttraining/orttraining/core/framework/pipeline.cc"
click node_pipeline_transform "https://github.com/microsoft/onnxruntime/blob/main/orttraining/orttraining/core/graph/pipeline_transformer.cc"
click node_sd_pipeline "https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py"
click node_sd_models "https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/diffusion_models.py"
click node_sd_schedulers "https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/diffusion_schedulers.py"
click node_sd_engine "https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/engine_builder.py"
click node_sd_optimize "https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/optimize_pipeline.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_js_api,node_node_binding,node_wasm_handler,node_webgl_handler,node_webgl_compute,node_winml_engine,node_web_tensor,node_go_api toneBlue
class node_session,node_c_api,node_cpp_api,node_cpp_inline,node_status,node_common,node_data_types,node_internal_types,node_data_transfer toneAmber
class node_cuda_qmoe,node_qnn_backend toneMint
class node_training_inference,node_training_pipeline,node_pipeline_transform toneRose
class node_sd_pipeline,node_sd_models,node_sd_schedulers,node_sd_engine,node_sd_optimize,node_application toneIndigo
```

## Components

### Application APIs

- [JS Session API](https://github.com/microsoft/onnxruntime/blob/main/js/common/lib/inference-session.ts): `js/common/lib/inference-session.ts`
- [Node Binding](https://github.com/microsoft/onnxruntime/blob/main/js/node/src/inference_session_wrap.cc): `js/node/src/inference_session_wrap.cc`
- [WASM Handler](https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/wasm/session-handler-inference.ts): `js/web/lib/wasm/session-handler-inference.ts`
- [WebGL Handler](https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/onnxjs/session-handler-inference.ts): `js/web/lib/onnxjs/session-handler-inference.ts`
- [WebGL Compute](https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/onnxjs/backends/webgl/inference-handler.ts): `js/web/lib/onnxjs/backends/webgl/inference-handler.ts`
- [WinML Engine](https://github.com/microsoft/onnxruntime/blob/main/winml/lib/Api.Ort/OnnxruntimeEngine.cpp): `winml/lib/Api.Ort/OnnxruntimeEngine.cpp`
- [Web Tensor](https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/onnxjs/tensor.ts): `js/web/lib/onnxjs/tensor.ts`
- [Go API](https://github.com/microsoft/onnxruntime/blob/main/go/onnxruntime/onnxruntime_c_api.h): `go/onnxruntime/onnxruntime_c_api.h`

### Core Runtime

- [Inference Session](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/session/inference_session.cc): `onnxruntime/core/session/inference_session.cc`
- [C API](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h): `include/onnxruntime/core/session/onnxruntime_c_api.h`
- [C++ API](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_cxx_api.h): `include/onnxruntime/core/session/onnxruntime_cxx_api.h`
- [C++ API Inline](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_cxx_inline.h): `include/onnxruntime/core/session/onnxruntime_cxx_inline.h`
- [Status Handling](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/common/status.h): `include/onnxruntime/core/common/status.h`
- [Common Types](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/common/common.h): `include/onnxruntime/core/common/common.h`
- [Data Types](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/framework/data_types.h): `include/onnxruntime/core/framework/data_types.h`
- [Internal Data Types](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/framework/data_types_internal.h): `include/onnxruntime/core/framework/data_types_internal.h`
- [Data Transfer](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/framework/data_transfer_manager.cc): `onnxruntime/core/framework/data_transfer_manager.cc`

### Execution Providers

- [CUDA MoE Offload](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/providers/cuda/qmoe_session_controller.h): `onnxruntime/core/providers/cuda/qmoe_session_controller.h`
- [QNN Backend](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/providers/qnn/builder/qnn_backend_manager.cc): `onnxruntime/core/providers/qnn/builder/qnn_backend_manager.cc`

### Training

- [ORTModule Inference](https://github.com/microsoft/onnxruntime/blob/main/orttraining/orttraining/python/training/ortmodule/_inference_manager.py): `orttraining/orttraining/python/training/ortmodule/_inference_manager.py`
- [Training Pipeline](https://github.com/microsoft/onnxruntime/blob/main/orttraining/orttraining/core/framework/pipeline.cc): `orttraining/orttraining/core/framework/pipeline.cc`
- [Pipeline Transformer](https://github.com/microsoft/onnxruntime/blob/main/orttraining/orttraining/core/graph/pipeline_transformer.cc): `orttraining/orttraining/core/graph/pipeline_transformer.cc`

### Stable Diffusion

- [Diffusion Pipeline](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py): `onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py`
- [Diffusion Models](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/diffusion_models.py): `onnxruntime/python/tools/transformers/models/stable_diffusion/diffusion_models.py`
- [Diffusion Schedulers](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/diffusion_schedulers.py): `onnxruntime/python/tools/transformers/models/stable_diffusion/diffusion_schedulers.py`
- [Engine Builders](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/engine_builder.py): `onnxruntime/python/tools/transformers/models/stable_diffusion/engine_builder.py`
- [Model Optimization](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/optimize_pipeline.py): `onnxruntime/python/tools/transformers/models/stable_diffusion/optimize_pipeline.py`

### Other components

- **Application**

## Connections

- Application → JS Session API: creates session (evidence: `js/common/lib/inference-session.ts`)
- JS Session API → Node Binding: uses binding
- Node Binding → C++ API: calls API (evidence: `js/node/src/inference_session_wrap.cc`)
- JS Session API → WASM Handler: dispatches backend
- WASM Handler → Web Tensor: converts tensors (evidence: `js/web/lib/wasm/session-handler-inference.ts`)
- Application → WebGL Handler: runs inference
- WebGL Handler → Web Tensor: converts tensors (evidence: `js/web/lib/onnxjs/session-handler-inference.ts`)
- WebGL Handler → WebGL Compute: runs session (evidence: `js/web/lib/onnxjs/session-handler-inference.ts`)
- Application → WinML Engine: runs inference
- WinML Engine → Inference Session: runs session (evidence: `winml/lib/Api.Ort/OnnxruntimeEngine.cpp`)
- Inference Session → C API: uses API
- Node Binding → Inference Session: constructs session (evidence: `js/node/src/inference_session_wrap.cc`)
- Go API → C API: declares C interface
- Application → ORTModule Inference: trains model (evidence: `README.md`)
- ORTModule Inference → Training Pipeline: runs forward pass
- Pipeline Transformer → Training Pipeline: transforms pipeline (evidence: `orttraining/orttraining/core/graph/pipeline_transformer.cc`)
- Application → Diffusion Pipeline: generates images
- Diffusion Pipeline → Diffusion Models: uses models (evidence: `onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py`)
- Diffusion Pipeline → Diffusion Schedulers: uses scheduler (evidence: `onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py`)
- Diffusion Pipeline → Engine Builders: runs engines (evidence: `onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py`)
- Model Optimization → Diffusion Pipeline: optimizes models
- QNN Backend → Inference Session: provides backend
- CUDA MoE Offload → Inference Session: supports session runs
- Data Transfer → Data Types: transfers tensors
- Internal Data Types → Data Types: extends types
