How alibaba/mnn works
MNN is a lightweight deep-learning framework for on-device inference and training, with MNN-LLM and MNN-Diffusion as model-runtime solutions built on its engine. The main workflow is to bring a supported model into an application, execute it through the runtime and a platform backend, and return results on-device; the repository also documents multimodal chat and local speech/avatar capabilities. The graph preserves these product branches and the sampled execution, memory, and model-schema layers. Some links describe README-documented capabilities; unsampled internals are not inferred as direct calls.
Main entry points: Pipeline.cpp Inference runtime · llm.cpp LLM runtime · LlmSession.kt Android chat session · LlmService.kt Avatar LLM service
Components 23
Model preparation
- Model converter
tools/converter/source/optimizer/onnxextra/OnnxExtraManager.cpp - MNN model schema
schema/current/MNN_generated.h
MNN engine
- Inference runtime
source/core/Pipeline.cpp - Express pipelines
express/module/PipelineModule.cpp - Backend interface
source/core/Backend.hpp - Geometry execution
source/geometry/GeometryComputer.hpp - Tensor utilities
source/core/TensorUtils.hpp - Execution schedule
source/core/Schedule.hpp - Memory ownership
source/core/AutoStorage.h - Plugin shape inference
source/plugin/PluginShapeInference.cpp
Generative runtimes
- LLM runtime
transformers/llm/engine/src/llm.cpp - Diffusion runtime
transformers/diffusion/engine/include/diffusion/sana_llm.hpp - LLM model export
transformers/llm/export/llmexport.py
On-device products
- Android chat session
apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/llm/LlmSession.kt - Chat multimodal data
apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/chat/model/ChatDataItem.kt - iOS inference app
apps/iOS/MNNLLMChat/MNNLLMiOS/InferenceEngine/LLMInferenceEngineWrapper.h - CLI LLM manager
apps/mnncli/src/llm_manager.cpp
Speech support
- Speaker embeddings
apps/frameworks/sherpa-mnn/sherpa-mnn/csrc/speaker-embedding-manager.cc - Avatar LLM service
apps/Android/MnnTaoAvatar/app/src/main/java/com/taobao/meta/avatar/llm/LlmService.kt - ASR and TTS
apps/frameworks/sherpa-mnn
Other components
- Model developer
- App user
- Model artifact
Connections 24
- Model developer → Model converter: converts models
README.md - Model converter → MNN model schema: encodes model
- MNN model schema → Inference runtime: describes model
source/core/Pipeline.cpp - Inference runtime → Backend interface: creates executions
source/core/Pipeline.cpp - Inference runtime → Execution schedule: uses schedule
source/core/Pipeline.cpp - Inference runtime → Tensor utilities: uses tensor metadata
source/core/Pipeline.cpp - Inference runtime → Geometry execution: uses geometry
- Inference runtime → Memory ownership: owns buffers
- Express pipelines → Inference runtime: loads pipeline
express/module/PipelineModule.cpp - App user → Android chat session: sends prompt
README.md - Android chat session → LLM runtime: runs inference
- Android chat session → Chat multimodal data: uses chat data
apps/Android/MnnLlmChat/app/src/main/java/com/alibaba/mnnllm/android/llm/LlmSession.kt - LLM runtime → Inference runtime: uses MNN engine
README.md - App user → iOS inference app: uses chat app
README.md - iOS inference app → LLM runtime: runs inference
- App user → CLI LLM manager: requests inference
- CLI LLM manager → LLM runtime: creates LLM
apps/mnncli/src/llm_manager.cpp - LLM model export → Model artifact: exports model
- App user → Diffusion runtime: requests generation
README.md - Diffusion runtime → Inference runtime: uses MNN engine
README.md - App user → Avatar LLM service: talks to avatar
README.md - Avatar LLM service → ASR and TTS: uses local speech
README.md - Avatar LLM service → LLM runtime: generates responses
- Speaker embeddings → ASR and TTS: supports speaker matching
apps/frameworks/sherpa-mnn/sherpa-mnn/csrc/speaker-embedding-manager.cc