How microsoft/onnxruntime works
ONNX Runtime is a cross-platform inference and training accelerator. The principal workflow is an application supplying an ONNX model and inputs to a session, which runs inference and returns outputs; hardware execution providers and graph optimization support that runtime. A separate training path accelerates PyTorch training. The graph also preserves documented JavaScript browser/Node backends and the Stable Diffusion optimization and inference branches. Edges reflect README statements or the supplied source index and excerpts; unsampled implementation details are not inferred.
Main entry points: pipeline_stable_diffusion.py Diffusion Pipeline · inference_session.cc Inference Session · inference-session.ts JS Session API · inference_session_wrap.cc Node Binding
Components 28
Application APIs
- JS Session API
js/common/lib/inference-session.ts - Node Binding
js/node/src/inference_session_wrap.cc - WASM Handler
js/web/lib/wasm/session-handler-inference.ts - WebGL Handler
js/web/lib/onnxjs/session-handler-inference.ts - WebGL Compute
js/web/lib/onnxjs/backends/webgl/inference-handler.ts - WinML Engine
winml/lib/Api.Ort/OnnxruntimeEngine.cpp - Web Tensor
js/web/lib/onnxjs/tensor.ts - Go API
go/onnxruntime/onnxruntime_c_api.h
Core Runtime
- Inference Session
onnxruntime/core/session/inference_session.cc - C API
include/onnxruntime/core/session/onnxruntime_c_api.h - C++ API
include/onnxruntime/core/session/onnxruntime_cxx_api.h - C++ API Inline
include/onnxruntime/core/session/onnxruntime_cxx_inline.h - Status Handling
include/onnxruntime/core/common/status.h - Common Types
include/onnxruntime/core/common/common.h - Data Types
include/onnxruntime/core/framework/data_types.h - Internal Data Types
include/onnxruntime/core/framework/data_types_internal.h - Data Transfer
onnxruntime/core/framework/data_transfer_manager.cc
Execution Providers
- CUDA MoE Offload
onnxruntime/core/providers/cuda/qmoe_session_controller.h - QNN Backend
onnxruntime/core/providers/qnn/builder/qnn_backend_manager.cc
Training
- ORTModule Inference
orttraining/orttraining/python/training/ortmodule/_inference_manager.py - Training Pipeline
orttraining/orttraining/core/framework/pipeline.cc - Pipeline Transformer
orttraining/orttraining/core/graph/pipeline_transformer.cc
Stable Diffusion
- Diffusion Pipeline
onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py - Diffusion Models
onnxruntime/python/tools/transformers/models/stable_diffusion/diffusion_models.py - Diffusion Schedulers
onnxruntime/python/tools/transformers/models/stable_diffusion/diffusion_schedulers.py - Engine Builders
onnxruntime/python/tools/transformers/models/stable_diffusion/engine_builder.py - Model Optimization
onnxruntime/python/tools/transformers/models/stable_diffusion/optimize_pipeline.py
Other components
- Application
Connections 25
- Application → JS Session API: creates session
js/common/lib/inference-session.ts - JS Session API → Node Binding: uses binding
- Node Binding → C++ API: calls API
js/node/src/inference_session_wrap.cc - JS Session API → WASM Handler: dispatches backend
- WASM Handler → Web Tensor: converts tensors
js/web/lib/wasm/session-handler-inference.ts - Application → WebGL Handler: runs inference
- WebGL Handler → Web Tensor: converts tensors
js/web/lib/onnxjs/session-handler-inference.ts - WebGL Handler → WebGL Compute: runs session
js/web/lib/onnxjs/session-handler-inference.ts - Application → WinML Engine: runs inference
- WinML Engine → Inference Session: runs session
winml/lib/Api.Ort/OnnxruntimeEngine.cpp - Inference Session → C API: uses API
- Node Binding → Inference Session: constructs session
js/node/src/inference_session_wrap.cc - Go API → C API: declares C interface
- Application → ORTModule Inference: trains model
README.md - ORTModule Inference → Training Pipeline: runs forward pass
- Pipeline Transformer → Training Pipeline: transforms pipeline
orttraining/orttraining/core/graph/pipeline_transformer.cc - Application → Diffusion Pipeline: generates images
- Diffusion Pipeline → Diffusion Models: uses models
onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py - Diffusion Pipeline → Diffusion Schedulers: uses scheduler
onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py - Diffusion Pipeline → Engine Builders: runs engines
onnxruntime/python/tools/transformers/models/stable_diffusion/pipeline_stable_diffusion.py - Model Optimization → Diffusion Pipeline: optimizes models
- QNN Backend → Inference Session: provides backend
- CUDA MoE Offload → Inference Session: supports session runs
- Data Transfer → Data Types: transfers tensors
- Internal Data Types → Data Types: extends types