Releases: espressif/esp-dl
ESP-DL Release v3.2.0
ESP-DL v3.2.0 continues to provide a lightweight, efficient neural network inference framework optimized for ESP series chips, now with broader model and operator compatibility and enhanced profiling tools for better application optimization.
New Key Features
A comprehensive profiling API has been introduced to help developers optimize their applications:
- Model memory profiling capabilities to analyze memory usage
- Performance profiling tools for detailed execution analysis
- API for model testing and profiling during runtime
- Optimized memory allocation and usage statistics
- Model lazy loading for improved memory efficiency
More Models Supported
ESP-DL v3.2.0 introduces support for a broader range of AI models, including:
- YOLO11n and YOLO11n-pose for advanced object detection and pose estimation
- ESPDet-Pico cat detection model for lightweight object detection
- LSTM and GRU models for sequence processing
- Enhanced human face detection and recognition models
Expanded Operator Support
The ESP-DL operator interface is aligned with ONNX. The opset 18 is recommended to export ONNX. Currently, the following 56 operators have been implemented and tested.
| Operator | int8 | int16 | float32 | Restrictions |
|---|---|---|---|---|
| Add(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | Support up to 4D |
| AveragePool(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | Support 1d/2d, don't support dilation |
| Clip(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| Concat(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| Conv(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | Support 1d/2d conv, groups only support 1 or input_channels |
| Div(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | Support up to 4D |
| Elu(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| Equal(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | |
| Exp(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| Flatten(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| Gather(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| Gemm(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | |
| GlobalAveragePool(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | Support 1d/2d |
| Greater(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | |
| GreaterOrEqual(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | |
| GRU(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| HardSigmoid(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| HardSwish(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| LeakyRelu(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| Less(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | |
| LessOrEqual(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | |
| Log(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| LSTM(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| MatMul(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | Support up to 4D |
| MaxPool(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | Support 1d/2d, don't support dilation |
| Mul(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | Support up to 4D |
| Neg(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| Pad(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | Do not support wrap mode |
| Pow(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | Support up to 4D, multidirectional broadcasting |
| PRelu(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | |
| ReduceL1(ESP-DL)(ONNX) ... |
ESP-DL Release v3.0.0
We are excited to introduce ESP-DL v3.0.0 , a lightweight and efficient neural network inference framework tailored for ESP series chips. This release brings significant enhancements and new features, making it easier than ever to develop AI applications on Espressif's System on Chips (SoCs). However, please be aware that v3.0.0 is not compatible with v2.0. Below are the key updates and improvements in this release:
Key Features and Improvements
ESP-DL Standard Model Format
Introduced a new model format similar to ONNX but optimized for ESP-DL, using FlatBuffers instead of Protobuf. This format supports zero-copy deserialization and has a file extension of .espdl.
Efficient Operator Implementation
Enhanced support by ESP32-S3 and ESP32-P4 PIE (Processor Instruction Extensions) for common AI operators such as Conv, Gemm, Add, and Mul.
Static Memory Planner
A new memory planner automatically allocates layers to the optimal memory location based on the user-specified internal RAM size, ensuring both efficient memory usage and faster overall performance.
Dual-Core Scheduling
Introduced automatic dual-core scheduling to fully utilize the dual-core computing power of ESP chips. Currently, Conv2D and DepthwiseConv2D operators support dual-core scheduling.
8-bit LUT Activation
All activation functions except ReLU and PReLU are implemented using an 8-bit Look-Up Table (LUT) method to accelerate inference. This allows for flexible activation function usage while maintaining consistent computational complexity.
Supported Models
Supported Operators
The ESP-DL operator interface is aligned with ONNX. The opset 13 is recommended to export ONNX.
Currently, the following 31 operators have been implemented and tested. Some operators do not implement all functionalities and attributes. Please refer to the description of each operator or test cases for details.
| Operator | int8 | int16 | Description |
|---|---|---|---|
| Add(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| AveragePool(ESP-DL)(ONNX) | ✔ | ✔ | |
| Clip(ESP-DL)(ONNX) | ✔ | ✔ | |
| Concat(ESP-DL)(ONNX) | ✔ | ✔ | |
| Conv(ESP-DL)(ONNX) | ✔ | ✔ | Groups only support 1 or input_channels |
| Div(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| Exp(ESP-DL)(ONNX) | ✔ | ✔ | |
| Flatten(ESP-DL)(ONNX) | ✔ | ✔ | |
| Gemm(ESP-DL)(ONNX) | ✔ | ✔ | |
| GlobalAveragePool(ESP-DL)(ONNX) | ✔ | ✔ | |
| HardSigmoid(ESP-DL)(ONNX) | ✔ | ✔ | |
| HardSwish(ESP-DL)(ONNX) | ✔ | ✔ | |
| LeakyRelu(ESP-DL)(ONNX) | ✔ | ✔ | |
| Log(ESP-DL)(ONNX) | ✔ | ✔ | |
| MatMul(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| MaxPool(ESP-DL)(ONNX) | ✔ | ✔ | |
| Mul(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| Pad(ESP-DL)(ONNX) | ✔ | ✔ | Do not support wrap mode |
| PRelu(ESP-DL)(ONNX) | ✔ | ✔ | |
| Reshape(ESP-DL)(ONNX) | ✔ | ✔ | |
| Resize(ESP-DL)(ONNX) | ✔ | ✖ | Only support nearest and do not support roi |
| Sigmoid(ESP-DL)(ONNX) | ✔ | ✔ | |
| Slice(ESP-DL)(ONNX) | ✔ | ✔ | |
| Softmax(ESP-DL)(ONNX) | ✔ | ✔ | Dtype of output is float32 |
| Split(ESP-DL)(ONNX) | ✔ | ✔ | |
| Sqrt(ESP-DL)(ONNX) | ✔ | ✔ | |
| Squeeze(ESP-DL)(ONNX) | ✔ | ✔ | |
| Sub(ESP-DL)(ONNX) | ✔ | ✔ | Support up to 4D |
| Tanh(ESP-DL)(ONNX) | ✔ | ✔ | |
| Transpose(ESP-DL)(ONNX) | ✔ | ✔ | |
| Unsqueeze(ESP-DL)(ONNX) | ✔ | ✔ |
Important Note
- **v3.0.0 is not backward compatib...
release/v2.0
ESP-DL Release v2.0
We're excited to announce the release of ESP-DL v2.0! This is a significant update which includes new features and improvements over the previous version.
New Features
Introduction of TVM: In this new release, we have introduced support for TVM. TVM is an open-source machine learning compiler stack that aims to enable efficient deployment of machine learning models on a variety of hardware platforms. This integration significantly streamlines the process of deploying models, providing a seamless workflow from training to deployment. Now, you can easily deploy your models trained with popular platforms such as TensorFlow and PyTorch using TVM, improving efficiency and flexibility.
New Documentation System with ESP-Docs: We're happy to announce that we've adopted the ESP-Docs system to build our documentation. This new system provides an improved user experience and allows us to maintain up-to-date and comprehensive documentation. We're confident that this change will make it even easier for users to find the information they need. You can now view our documentation on the web at ESP-DL User Guide.
Feedback
We hope you find the new TVM integration and updated documentation system helpful in improving your model deployment workflows. If you have any questions or encounter any issues, please submit a issue.
Thank you for supporting ESP-DL!