# alibaba/mnn architecture

> Architecture diagram of the GitHub repository alibaba/mnn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alibaba/mnn
- Repository: https://github.com/alibaba/mnn

## Overview

MNN is a lightweight deep-learning framework for on-device inference and training, with MNN-LLM and MNN-Diffusion as model-runtime solutions built on its engine. The main workflow is to bring a supported model into an application, execute it through the runtime and a platform backend, and return results on-device; the repository also documents multimodal chat and local speech/avatar capabilities. The graph preserves these product branches and the sampled execution, memory, and model-schema layers. Some links describe README-documented capabilities; unsampled internals are not inferred as direct calls.

Main entry points: [`source/core/Pipeline.cpp`](https://github.com/alibaba/mnn/blob/master/source/core/Pipeline.cpp) (Inference runtime), [`transformers/llm/engine/src/llm.cpp`](https://github.com/alibaba/mnn/blob/master/transformers/llm/engine/src/llm.cpp) (LLM runtime), [`apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/llm/LlmSession.kt`](https://github.com/alibaba/mnn/blob/master/apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/llm/LlmSession.kt) (Android chat session), [`apps/Android/MnnTaoAvatar/app/src/main/java/com/taobao/meta/avatar/llm/LlmService.kt`](https://github.com/alibaba/mnn/blob/master/apps/Android/MnnTaoAvatar/app/src/main/java/com/taobao/meta/avatar/llm/LlmService.kt) (Avatar LLM service)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_model["Model preparation"]
  node_converter["Model converter"]
  node_schema["MNN model schema<br/>[MNN_generated.h]"]
end

subgraph group_engine["MNN engine"]
  node_engine_runtime["Inference runtime<br/>[Pipeline.cpp]"]
  node_express_pipeline["Express pipelines<br/>[PipelineModule.cpp]"]
  node_backend["Backend interface<br/>[Backend.hpp]"]
  node_geometry["Geometry execution"]
  node_tensor_utils["Tensor utilities<br/>[TensorUtils.hpp]"]
  node_schedule["Execution schedule<br/>[Schedule.hpp]"]
  node_memory["Memory ownership<br/>[AutoStorage.h]"]
  node_shape_inference["Plugin shape inference"]
end

subgraph group_generative["Generative runtimes"]
  node_llm_runtime["LLM runtime<br/>[llm.cpp]"]
  node_diffusion_runtime["Diffusion runtime<br/>[sana_llm.hpp]"]
  node_llm_export["LLM model export<br/>[llmexport.py]"]
end

subgraph group_apps["On-device products"]
  node_android_chat["Android chat session<br/>[LlmSession.kt]"]
  node_chat_data["Chat multimodal data<br/>[ChatDataItem.kt]"]
  node_ios_chat["iOS inference app"]
  node_cli["CLI LLM manager<br/>[llm_manager.cpp]"]
end

subgraph group_speech["Speech support"]
  node_speech_embed["Speaker embeddings"]
  node_avatar_llm["Avatar LLM service<br/>[LlmService.kt]"]
  node_speech_stack["ASR and TTS"]
end

node_developer(("Model developer"))
node_user(("App user"))
node_model_artifact["Model artifact"]

node_developer -->|"converts models"| node_converter
node_converter -.->|"encodes model"| node_schema
node_schema -->|"describes model"| node_engine_runtime
node_engine_runtime -->|"creates executions"| node_backend
node_engine_runtime -->|"uses schedule"| node_schedule
node_engine_runtime -->|"uses tensor metadata"| node_tensor_utils
node_engine_runtime -.->|"uses geometry"| node_geometry
node_engine_runtime -.->|"owns buffers"| node_memory
node_express_pipeline -->|"loads pipeline"| node_engine_runtime
node_user -->|"sends prompt"| node_android_chat
node_android_chat -.->|"runs inference"| node_llm_runtime
node_android_chat -.->|"uses chat data"| node_chat_data
node_llm_runtime -->|"uses MNN engine"| node_engine_runtime
node_user -->|"uses chat app"| node_ios_chat
node_ios_chat -.->|"runs inference"| node_llm_runtime
node_user -.->|"requests inference"| node_cli
node_cli -->|"creates LLM"| node_llm_runtime
node_llm_export -.->|"exports model"| node_model_artifact
node_user -->|"requests generation"| node_diffusion_runtime
node_diffusion_runtime -->|"uses MNN engine"| node_engine_runtime
node_user -->|"talks to avatar"| node_avatar_llm
node_avatar_llm -->|"uses local speech"| node_speech_stack
node_avatar_llm -.->|"generates responses"| node_llm_runtime
node_speech_embed -.->|"supports speaker matching"| node_speech_stack

click node_converter "https://github.com/alibaba/mnn/blob/master/tools/converter/source/optimizer/onnxextra/OnnxExtraManager.cpp"
click node_schema "https://github.com/alibaba/mnn/blob/master/schema/current/MNN_generated.h"
click node_engine_runtime "https://github.com/alibaba/mnn/blob/master/source/core/Pipeline.cpp"
click node_express_pipeline "https://github.com/alibaba/mnn/blob/master/express/module/PipelineModule.cpp"
click node_backend "https://github.com/alibaba/mnn/blob/master/source/core/Backend.hpp"
click node_geometry "https://github.com/alibaba/mnn/blob/master/source/geometry/GeometryComputer.hpp"
click node_tensor_utils "https://github.com/alibaba/mnn/blob/master/source/core/TensorUtils.hpp"
click node_schedule "https://github.com/alibaba/mnn/blob/master/source/core/Schedule.hpp"
click node_memory "https://github.com/alibaba/mnn/blob/master/source/core/AutoStorage.h"
click node_shape_inference "https://github.com/alibaba/mnn/blob/master/source/plugin/PluginShapeInference.cpp"
click node_llm_runtime "https://github.com/alibaba/mnn/blob/master/transformers/llm/engine/src/llm.cpp"
click node_diffusion_runtime "https://github.com/alibaba/mnn/blob/master/transformers/diffusion/engine/include/diffusion/sana_llm.hpp"
click node_llm_export "https://github.com/alibaba/mnn/blob/master/transformers/llm/export/llmexport.py"
click node_android_chat "https://github.com/alibaba/mnn/blob/master/apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/llm/LlmSession.kt"
click node_chat_data "https://github.com/alibaba/mnn/blob/master/apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/chat/model/ChatDataItem.kt"
click node_ios_chat "https://github.com/alibaba/mnn/blob/master/apps/iOS/MNNLLMChat/MNNLLMiOS/InferenceEngine/LLMInferenceEngineWrapper.h"
click node_cli "https://github.com/alibaba/mnn/blob/master/apps/mnncli/src/llm_manager.cpp"
click node_speech_embed "https://github.com/alibaba/mnn/blob/master/apps/frameworks/sherpa-mnn/sherpa-mnn/csrc/speaker-embedding-manager.cc"
click node_avatar_llm "https://github.com/alibaba/mnn/blob/master/apps/Android/MnnTaoAvatar/app/src/main/java/com/taobao/meta/avatar/llm/LlmService.kt"
click node_speech_stack "https://github.com/alibaba/mnn/tree/master/apps/frameworks/sherpa-mnn"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_converter,node_schema,node_user toneBlue
class node_engine_runtime,node_express_pipeline,node_backend,node_geometry,node_tensor_utils,node_schedule,node_memory,node_shape_inference toneAmber
class node_llm_runtime,node_diffusion_runtime,node_llm_export toneMint
class node_android_chat,node_chat_data,node_ios_chat,node_cli toneRose
class node_speech_embed,node_avatar_llm,node_speech_stack,node_developer,node_model_artifact toneIndigo
```

## Components

### Model preparation

- [Model converter](https://github.com/alibaba/mnn/blob/master/tools/converter/source/optimizer/onnxextra/OnnxExtraManager.cpp): `tools/converter/source/optimizer/onnxextra/OnnxExtraManager.cpp`
- [MNN model schema](https://github.com/alibaba/mnn/blob/master/schema/current/MNN_generated.h): `schema/current/MNN_generated.h`

### MNN engine

- [Inference runtime](https://github.com/alibaba/mnn/blob/master/source/core/Pipeline.cpp): `source/core/Pipeline.cpp`
- [Express pipelines](https://github.com/alibaba/mnn/blob/master/express/module/PipelineModule.cpp): `express/module/PipelineModule.cpp`
- [Backend interface](https://github.com/alibaba/mnn/blob/master/source/core/Backend.hpp): `source/core/Backend.hpp`
- [Geometry execution](https://github.com/alibaba/mnn/blob/master/source/geometry/GeometryComputer.hpp): `source/geometry/GeometryComputer.hpp`
- [Tensor utilities](https://github.com/alibaba/mnn/blob/master/source/core/TensorUtils.hpp): `source/core/TensorUtils.hpp`
- [Execution schedule](https://github.com/alibaba/mnn/blob/master/source/core/Schedule.hpp): `source/core/Schedule.hpp`
- [Memory ownership](https://github.com/alibaba/mnn/blob/master/source/core/AutoStorage.h): `source/core/AutoStorage.h`
- [Plugin shape inference](https://github.com/alibaba/mnn/blob/master/source/plugin/PluginShapeInference.cpp): `source/plugin/PluginShapeInference.cpp`

### Generative runtimes

- [LLM runtime](https://github.com/alibaba/mnn/blob/master/transformers/llm/engine/src/llm.cpp): `transformers/llm/engine/src/llm.cpp`
- [Diffusion runtime](https://github.com/alibaba/mnn/blob/master/transformers/diffusion/engine/include/diffusion/sana_llm.hpp): `transformers/diffusion/engine/include/diffusion/sana_llm.hpp`
- [LLM model export](https://github.com/alibaba/mnn/blob/master/transformers/llm/export/llmexport.py): `transformers/llm/export/llmexport.py`

### On-device products

- [Android chat session](https://github.com/alibaba/mnn/blob/master/apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/llm/LlmSession.kt): `apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/llm/LlmSession.kt`
- [Chat multimodal data](https://github.com/alibaba/mnn/blob/master/apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/chat/model/ChatDataItem.kt): `apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/chat/model/ChatDataItem.kt`
- [iOS inference app](https://github.com/alibaba/mnn/blob/master/apps/iOS/MNNLLMChat/MNNLLMiOS/InferenceEngine/LLMInferenceEngineWrapper.h): `apps/iOS/MNNLLMChat/MNNLLMiOS/InferenceEngine/LLMInferenceEngineWrapper.h`
- [CLI LLM manager](https://github.com/alibaba/mnn/blob/master/apps/mnncli/src/llm_manager.cpp): `apps/mnncli/src/llm_manager.cpp`

### Speech support

- [Speaker embeddings](https://github.com/alibaba/mnn/blob/master/apps/frameworks/sherpa-mnn/sherpa-mnn/csrc/speaker-embedding-manager.cc): `apps/frameworks/sherpa-mnn/sherpa-mnn/csrc/speaker-embedding-manager.cc`
- [Avatar LLM service](https://github.com/alibaba/mnn/blob/master/apps/Android/MnnTaoAvatar/app/src/main/java/com/taobao/meta/avatar/llm/LlmService.kt): `apps/Android/MnnTaoAvatar/app/src/main/java/com/taobao/meta/avatar/llm/LlmService.kt`
- [ASR and TTS](https://github.com/alibaba/mnn/tree/master/apps/frameworks/sherpa-mnn): `apps/frameworks/sherpa-mnn`

### Other components

- **Model developer**
- **App user**
- **Model artifact**

## Connections

- Model developer → Model converter: converts models (evidence: `README.md`)
- Model converter → MNN model schema: encodes model
- MNN model schema → Inference runtime: describes model (evidence: `source/core/Pipeline.cpp`)
- Inference runtime → Backend interface: creates executions (evidence: `source/core/Pipeline.cpp`)
- Inference runtime → Execution schedule: uses schedule (evidence: `source/core/Pipeline.cpp`)
- Inference runtime → Tensor utilities: uses tensor metadata (evidence: `source/core/Pipeline.cpp`)
- Inference runtime → Geometry execution: uses geometry
- Inference runtime → Memory ownership: owns buffers
- Express pipelines → Inference runtime: loads pipeline (evidence: `express/module/PipelineModule.cpp`)
- App user → Android chat session: sends prompt (evidence: `README.md`)
- Android chat session → LLM runtime: runs inference
- Android chat session → Chat multimodal data: uses chat data (evidence: `apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/llm/LlmSession.kt`)
- LLM runtime → Inference runtime: uses MNN engine (evidence: `README.md`)
- App user → iOS inference app: uses chat app (evidence: `README.md`)
- iOS inference app → LLM runtime: runs inference
- App user → CLI LLM manager: requests inference
- CLI LLM manager → LLM runtime: creates LLM (evidence: `apps/mnncli/src/llm_manager.cpp`)
- LLM model export → Model artifact: exports model
- App user → Diffusion runtime: requests generation (evidence: `README.md`)
- Diffusion runtime → Inference runtime: uses MNN engine (evidence: `README.md`)
- App user → Avatar LLM service: talks to avatar (evidence: `README.md`)
- Avatar LLM service → ASR and TTS: uses local speech (evidence: `README.md`)
- Avatar LLM service → LLM runtime: generates responses
- Speaker embeddings → ASR and TTS: supports speaker matching (evidence: `apps/frameworks/sherpa-mnn/sherpa-mnn/csrc/speaker-embedding-manager.cc`)