# espressif/esp-dl architecture

> Architecture diagram of the GitHub repository espressif/esp-dl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/espressif/esp-dl
- Repository: https://github.com/espressif/esp-dl

## Overview

ESP-DL is an embedded neural-network inference framework for Espressif chips. Its main workflow is to obtain a quantized model in the ESP-DL FlatBuffers format, load it, plan tensor memory, and execute model operators to produce results. The repository also provides image, vision, audio, and recognition capabilities, plus a zoo of ready-to-integrate models. Relationships below are limited to README statements and source-index references; several domain implementations are represented by their listed directories because their source was not sampled.

Main entry points: [`esp-dl/dl/module/include/dl_module_base.hpp`](https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/module/include/dl_module_base.hpp) (Operator Modules), [`esp-dl/dl/model/include/dl_model_base.hpp`](https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/model/include/dl_model_base.hpp) (Model Execution), [`esp-dl/fbs_loader/include/fbs_model.hpp`](https://github.com/espressif/esp-dl/blob/master/esp-dl/fbs_loader/include/fbs_model.hpp) (ESP-DL Model), [`models/speaker_verification/speaker_verification.cpp`](https://github.com/espressif/esp-dl/blob/master/models/speaker_verification/speaker_verification.cpp) (Speaker Verification)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_model_runtime["Model Runtime"]
  node_loader["FlatBuffers Loader<br/>[fbs_loader.cpp]"]
  node_model_execution["Model Execution<br/>[dl_model_base.hpp]"]
  node_memory_planner["Memory Planner"]
  node_model_context["Model Context"]
  node_tensor["Tensor Data<br/>[dl_tensor_base.hpp]"]
end

subgraph group_operator_compute["Operator Compute"]
  node_module_runtime["Operator Modules<br/>[dl_module_base.hpp]"]
  node_operator_kernels["Operator Kernels<br/>[dl_base.hpp]"]
  node_isa_kernels["Chip Kernels<br/>[dl_base_isa.hpp]"]
end

subgraph group_media_inputs["Media Processing"]
  node_image_processing["Image Processing"]
  node_audio_features["Speech Features<br/>[dl_fbank.hpp]"]
end

subgraph group_domain_tasks["AI Tasks"]
  node_vision_tasks["Vision Tasks"]
  node_recognition["Feature Recognition"]
  node_speech_verification["Speaker Verification"]
  node_feature_database[("Feature Verification DB")]
end

subgraph group_model_assets["Models And Tools"]
  node_espdl_format["ESP-DL Model<br/>[fbs_model.hpp]"]
  node_model_zoo["Model Zoo"]
  node_quantization_tools["Quantization Helpers<br/>[run_iteration.py]"]
end

node_developer(("Application Developer"))
node_quantizer["ESP-PPQ"]

node_developer -->|"quantizes model"| node_quantizer
node_quantizer -->|"exports model"| node_espdl_format
node_developer -->|"loads model"| node_loader
node_loader -->|"reads model"| node_espdl_format
node_developer -->|"runs model"| node_model_execution
node_model_execution -.->|"dispatches operators"| node_module_runtime
node_module_runtime -->|"uses context"| node_model_context
node_module_runtime -->|"uses tensors"| node_tensor
node_module_runtime -->|"uses kernels"| node_operator_kernels
node_module_runtime -->|"deserializes nodes"| node_espdl_format
node_operator_kernels -->|"selects kernels"| node_isa_kernels
node_memory_planner -->|"reads model metadata"| node_espdl_format
node_memory_planner -->|"plans allocations"| node_model_context
node_model_context -->|"holds tensors"| node_tensor
node_image_processing -.->|"prepares images"| node_vision_tasks
node_audio_features -->|"extracts features"| node_speech_verification
node_speech_verification -->|"runs model"| node_model_execution
node_speech_verification -->|"uses features"| node_audio_features
node_feature_database -.->|"verifies embeddings"| node_model_execution
node_developer -->|"integrates models"| node_model_zoo
node_quantization_tools -.->|"assists tuning"| node_quantizer

click node_espdl_format "https://github.com/espressif/esp-dl/blob/master/esp-dl/fbs_loader/include/fbs_model.hpp"
click node_loader "https://github.com/espressif/esp-dl/blob/master/esp-dl/fbs_loader/src/fbs_loader.cpp"
click node_model_execution "https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/model/include/dl_model_base.hpp"
click node_memory_planner "https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/model/src/dl_memory_manager_greedy.cpp"
click node_model_context "https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/model/include/dl_model_context.hpp"
click node_tensor "https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/tensor/include/dl_tensor_base.hpp"
click node_module_runtime "https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/module/include/dl_module_base.hpp"
click node_operator_kernels "https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/base/dl_base.hpp"
click node_isa_kernels "https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/base/isa/dl_base_isa.hpp"
click node_image_processing "https://github.com/espressif/esp-dl/tree/master/esp-dl/vision/image"
click node_audio_features "https://github.com/espressif/esp-dl/blob/master/esp-dl/audio/speech_features/dl_fbank.hpp"
click node_vision_tasks "https://github.com/espressif/esp-dl/tree/master/esp-dl/vision"
click node_recognition "https://github.com/espressif/esp-dl/tree/master/esp-dl/vision/recognition"
click node_speech_verification "https://github.com/espressif/esp-dl/blob/master/models/speaker_verification/speaker_verification.cpp"
click node_feature_database "https://github.com/espressif/esp-dl/blob/master/models/feat_database/dl_feat_verification_database.cpp"
click node_model_zoo "https://github.com/espressif/esp-dl/tree/master/models"
click node_quantization_tools "https://github.com/espressif/esp-dl/blob/master/tools/agents/skills/espdl-quantize/scripts/run_iteration.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_loader,node_model_execution,node_memory_planner,node_model_context,node_tensor toneBlue
class node_module_runtime,node_operator_kernels,node_isa_kernels toneAmber
class node_image_processing,node_audio_features toneMint
class node_vision_tasks,node_recognition,node_speech_verification,node_feature_database toneRose
class node_espdl_format,node_model_zoo,node_quantization_tools,node_developer,node_quantizer toneIndigo
```

## Components

### Model Runtime

- [FlatBuffers Loader](https://github.com/espressif/esp-dl/blob/master/esp-dl/fbs_loader/src/fbs_loader.cpp): `esp-dl/fbs_loader/src/fbs_loader.cpp`
- [Model Execution](https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/model/include/dl_model_base.hpp): `esp-dl/dl/model/include/dl_model_base.hpp`
- [Memory Planner](https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/model/src/dl_memory_manager_greedy.cpp): `esp-dl/dl/model/src/dl_memory_manager_greedy.cpp`
- [Model Context](https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/model/include/dl_model_context.hpp): `esp-dl/dl/model/include/dl_model_context.hpp`
- [Tensor Data](https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/tensor/include/dl_tensor_base.hpp): `esp-dl/dl/tensor/include/dl_tensor_base.hpp`

### Operator Compute

- [Operator Modules](https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/module/include/dl_module_base.hpp): `esp-dl/dl/module/include/dl_module_base.hpp`
- [Operator Kernels](https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/base/dl_base.hpp): `esp-dl/dl/base/dl_base.hpp`
- [Chip Kernels](https://github.com/espressif/esp-dl/blob/master/esp-dl/dl/base/isa/dl_base_isa.hpp): `esp-dl/dl/base/isa/dl_base_isa.hpp`

### Media Processing

- [Image Processing](https://github.com/espressif/esp-dl/tree/master/esp-dl/vision/image): `esp-dl/vision/image`
- [Speech Features](https://github.com/espressif/esp-dl/blob/master/esp-dl/audio/speech_features/dl_fbank.hpp): `esp-dl/audio/speech_features/dl_fbank.hpp`

### AI Tasks

- [Vision Tasks](https://github.com/espressif/esp-dl/tree/master/esp-dl/vision): `esp-dl/vision`
- [Feature Recognition](https://github.com/espressif/esp-dl/tree/master/esp-dl/vision/recognition): `esp-dl/vision/recognition`
- [Speaker Verification](https://github.com/espressif/esp-dl/blob/master/models/speaker_verification/speaker_verification.cpp): `models/speaker_verification/speaker_verification.cpp`
- [Feature Verification DB](https://github.com/espressif/esp-dl/blob/master/models/feat_database/dl_feat_verification_database.cpp): `models/feat_database/dl_feat_verification_database.cpp`

### Models And Tools

- [ESP-DL Model](https://github.com/espressif/esp-dl/blob/master/esp-dl/fbs_loader/include/fbs_model.hpp): `esp-dl/fbs_loader/include/fbs_model.hpp`
- [Model Zoo](https://github.com/espressif/esp-dl/tree/master/models): `models`
- [Quantization Helpers](https://github.com/espressif/esp-dl/blob/master/tools/agents/skills/espdl-quantize/scripts/run_iteration.py): `tools/agents/skills/espdl-quantize/scripts/run_iteration.py`

### Other components

- **Application Developer**
- **ESP-PPQ**

## Connections

- Application Developer → ESP-PPQ: quantizes model (evidence: `README.md`)
- ESP-PPQ → ESP-DL Model: exports model (evidence: `README.md`)
- Application Developer → FlatBuffers Loader: loads model (evidence: `README.md`)
- FlatBuffers Loader → ESP-DL Model: reads model (evidence: `esp-dl/fbs_loader/src/fbs_loader.cpp`)
- Application Developer → Model Execution: runs model (evidence: `README.md`)
- Model Execution → Operator Modules: dispatches operators
- Operator Modules → Model Context: uses context (evidence: `esp-dl/dl/module/include/dl_module_base.hpp`)
- Operator Modules → Tensor Data: uses tensors (evidence: `esp-dl/dl/module/include/dl_module_base.hpp`)
- Operator Modules → Operator Kernels: uses kernels (evidence: `esp-dl/dl/module/include/dl_module_base.hpp`)
- Operator Modules → ESP-DL Model: deserializes nodes (evidence: `esp-dl/dl/module/include/dl_module_base.hpp`)
- Operator Kernels → Chip Kernels: selects kernels (evidence: `esp-dl/dl/base/isa/dl_base_isa.hpp`)
- Memory Planner → ESP-DL Model: reads model metadata
- Memory Planner → Model Context: plans allocations
- Model Context → Tensor Data: holds tensors (evidence: `esp-dl/dl/model/include/dl_model_context.hpp`)
- Image Processing → Vision Tasks: prepares images
- Speech Features → Speaker Verification: extracts features
- Speaker Verification → Model Execution: runs model
- Speaker Verification → Speech Features: uses features
- Feature Verification DB → Model Execution: verifies embeddings
- Application Developer → Model Zoo: integrates models (evidence: `README.md`)
- Quantization Helpers → ESP-PPQ: assists tuning