Skip to content

Commit 8ac67e6

Browse files
jason-maojason-mao
committed
Merge branch 'feature/add_esp_image_effects_libs' into 'master'
Add esp_image_effects library See merge request adf/esp-adf-libs!284
2 parents 611914b + 81749da commit 8ac67e6

40 files changed

+3124
-0
lines changed

‎.github/workflows/upload_component.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,5 +23,6 @@ jobs:
2323
esp_audio_effects;
2424
esp_video_codec;
2525
esp_libsrtp;
26+
esp_image_effects;
2627
namespace: "espressif"
2728
api_token: ${{ secrets.IDF_COMPONENT_API_TOKEN }}

‎esp_image_effects/CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Changelog
2+
3+
## v1.0.0
4+
5+
### Features
6+
7+
- Initial version of `esp-image-effects`
8+
- Add image effects module for `scale`, `crop`, `color convert`, `rotate`
9+

‎esp_image_effects/CMakeLists.txt‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
idf_component_register(INCLUDE_DIRS "include")
2+
if("${IDF_VERSION_MAJOR}.${IDF_VERSION_MINOR}" VERSION_GREATER "5.3")
3+
if (${IDF_TARGET} STREQUAL "esp32p4")
4+
message(FATAL_ERROR "If use esp32p4, please use idf v5.3.")
5+
endif()
6+
endif()
7+
target_link_libraries(${COMPONENT_LIB} INTERFACE "-L ${CMAKE_CURRENT_SOURCE_DIR}/lib/${CONFIG_IDF_TARGET}")
8+
target_link_libraries(${COMPONENT_LIB} INTERFACE esp_image_effects)

‎esp_image_effects/LICENSE‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
ESPRESSIF MIT License
2+
3+
Copyright (c) 2025 <ESPRESSIF SYSTEMS (SHANGHAI) CO.，LTD>
4+
5+
Permission is hereby granted for use on all ESPRESSIF SYSTEMS products, in which case,
6+
it is free of charge, to any person obtaining a copy of this software and associated
7+
documentation files (the "Software"), to deal in the Software without restriction, including
8+
without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
9+
and/or sell copies of the Software, and to permit persons to whom the Software is furnished
10+
to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all copies or
13+
substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
17+
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
18+
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
19+
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
20+
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

‎esp_image_effects/README.md‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# ESP_IMAGE_EFFECTS
2+
3+
ESP Image Effects (ESP_IMGFX) is an image processing engine that integrates basic functions such as rotation, color space conversion, scaling, cropping, and more. It serves as one of the core components of Espressif's audio and video development platform. The ESP Image Effects module undergoes deep reconstruction of underlying algorithms, combined with efficient memory management and hardware acceleration, achieving the triple characteristics of high performance, low power consumption, and low memory usage. Additionally, each image processing function adopts a consistent API architecture design, reducing the learning cost for users and facilitating rapid development. This engine is widely applicable in various fields, including the Internet of Things, smart cameras, industrial vision, and more.
4+
5+
# FEATURES
6+
7+
## IMAGE COLOR CONVERT
8+
9+
- Supported arbitrary input resolutions
10+
- Supported bypass mode for identical input/output formats
11+
- Supports BT.601/BT.709/BT.2020 color space standards
12+
- Supported faster color convert algorithm for formats and resolutions
13+
- Comprehensive format support matrix:
14+
| Input Format | Supported Output Formats |
15+
|----------------------------|-----------------------------------------------------------|
16+
| RGB/BGR565_LE/BE RGB/BGR888 | RGB565_LE/BGR/RGB565_LE/BE RGB/BGR888 YUV_PLANAR/PACKET YUYV/UYVY O_UYY_E_VYY/I420 |
17+
| ARGB/BGR888 | RGB565_LE/BGR/RGB565_LE/BE RGB/BGR888 YUV_PLANAR O_UYY_E_VYY/I420 |
18+
| YUV_PACKET/UYVY/YUYV | RGB565_LE/BGR/RGB565_LE/BE RGB/BGR888 O_UYY_E_VYY/I420 |
19+
| O_UYY_E_VYY/I420 | RGB565_LE/BGR/RGB565_LE/BE RGB/BGR888 O_UYY_E_VYY |
20+
21+
## IMAGE ROTATE
22+
23+
- Supported bypass
24+
- Supported arbitrary input resolutions
25+
- Supported clockwise rotation at any degree
26+
- Supported ESP_IMG_PIXEL_FMT_Y/RGB565/BGR565/RGB888/BGR888/YUV_PACKET
27+
- Supported faster clockwise rotation algorithm for specific degrees formats resolutions
28+
29+
## IMAGE SCALE
30+
31+
- Supported bypass
32+
- Supported arbitrary input resolutions
33+
- Supported both upscaling and downscaling operations
34+
- Supported ESP_IMG_PIXEL_FMT_RGB565/BGR565/RGB888/BGR888/YUV_PACKET
35+
- Supported multiple filter algorithms: optimized downsampling and bilinear interpolation
36+
37+
## IMAGE CROP
38+
39+
- Supported bypass
40+
- Supported arbitrary input resolutions
41+
- Supported flexible region selection
42+
- Supported ESP_IMG_PIXEL_FMT_Y/RGB565/BGR565/RGB888/BGR888/YUV_PACKET
43+
44+
# PERFORMANCE
45+
46+
The performance on esp32p4 :`./doc/PERFORMANCE_ESP32P4.md`
47+
48+
# USAGE
49+
For more details on API usage, please refer to the source files in `test_app/main`
50+
51+
# SUPPORTED CHIP
52+
| chip | C code | ASM code |
53+
| -------- | --------| ---------|
54+
| ESP32 | √ | x |
55+
| ESP32-S2 | √ | x |
56+
| ESP32-S3 | √ | x |
57+
| ESP32-C3 | √ | x |
58+
| ESP32-C5 | √ | x |
59+
| ESP32-C6 | √ | x |
60+
| ESP32-P4 | √ | √ |
61+
62+
# FAQ
63+
64+
---
65+
### 1. Why is image data size alignment and address alignment recommended?
66+
- Image data size alignment and address alignment are recommended for hardware compatibility and performance optimization.
67+
- Alignment improves cache efficiency, reducing memory access latency and enhancing overall performance.
68+
- SIMD instruction sets, such as those requiring data size alignment to 16 bytes, benefit from alignment.
69+
---
70+
71+
### 2. Why Concurrent Access to `esp_imgfx_*_set_cfg` and `esp_imgfx_*_process` is Prohibited？
72+
1. **Reason**
73+
- Modifying configuration parameters with `esp_imgfx_*_set_cfg` directly impacts `esp_imgfx_*_process`, making concurrent access unsafe in multi-threaded environments.
74+
2. **Solution**
75+
- Ensure atomicity between configuration updates and processing in multi-threaded scenarios by using mutexes or atomic operations.
76+
---
77+
78+
### 3. Reasons for `esp_imgfx_*_process` Returning `ESP_IMGFX_ERR_DATA_LACK`
79+
1. **Reason**
80+
- Input image buffer size is smaller than configured requirements, causing data lack errors (e.g., mismatch between configured 1920x1080 and input buffer size of 1920x720).
81+
2. **Solution**
82+
- Obtain required input dimensions with `esp_imgfx_get_image_size` and allocate sufficient data buffers accordingly.
83+
84+
---
85+
86+
### 4. Reasons for `esp_imgfx_*_process` Returning `ESP_IMGFX_ERR_BUFF_NOT_ENOUGH`
87+
1. **Reason**
88+
- Output image buffer size is smaller than configured requirements, leading to buffer not enough errors (e.g., configured for 1920x1080 but output buffer allocated for 1920x720).
89+
2. **Solution**
90+
- Pre-calculate output dimensions with `esp_imgfx_get_image_size` to match requirements and adjust buffer sizes accordingly.
91+
92+
---
93+
94+
### 5. Can the configuration parameters be reset?
95+
Yes, users can reset the configuration parameters by calling esp_imgfx_*_set_cfg. After that, esp_imgfx_*_process should be executed with the updated configuration.
96+
97+
---
98+
99+
### 6. What's reason of the build error `If use esp32p4, please use idf v5.3.`?
100+
1. **Reason**
101+
"Currently, compilers above version 5.3 have a compatibility issue, so P4 must be compiled with version 5.3. This will be fixed in a future update."
102+
2. **Solution**
103+
- Use idf5.3
104+
---

‎esp_image_effects/doc/PERFORMANCE_ESP32P4.md‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Test on esp32p4
2+
3+
## IMAGE COLOR CONVERT
4+
5+
| Width | Height | Input format | Output format | OPT Frame Per Second(fps) | C Frame Per Second(fps) |
6+
|-------|--------|----------------------------------|----------------------------------|---------------------------|-------------------------|
7+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_I420 | ESP_IMG_PIXEL_FMT_I420 | 34.8958 | 26.9247 |
8+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_I420 | ESP_IMG_PIXEL_FMT_O_UYY_E_VYY | 30.1942 | 19.815 |
9+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_I420 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 18.1374 | 4.5326 |
10+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_I420 | ESP_IMG_PIXEL_FMT_RGB/BGR888 | 17.5294 | 4.4743 |
11+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_O_UYY_E_VYY | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 18.8097 | 4.7104 |
12+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_O_UYY_E_VYY | ESP_IMG_PIXEL_FMT_RGB/BGR888 | 17.9523 | 5.6457 |
13+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | ESP_IMG_PIXEL_FMT_I420 | 20.1829 | 2.4178 |
14+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | ESP_IMG_PIXEL_FMT_O_UYY_E_VYY | 20.4147 | 2.4455 |
15+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 26.3158 | 20.2532 |
16+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | ESP_IMG_PIXEL_FMT_YUYV/UYVY | 18.3592 | 2.2755 |
17+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | ESP_IMG_PIXEL_FMT_RGB/BGR888 | 17.9272 | 7.6518 |
18+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | ESP_IMG_PIXEL_FMT_YUV_PLANNER | 13.8708 | 2.7918 |
19+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | ESP_IMG_PIXEL_FMT_YUV_PACKET | 14.7262 | 3.0223 |
20+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_YUYV/UYVY | ESP_IMG_PIXEL_FMT_I420 | 26.936 | 12.8928 |
21+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_YUYV/UYVY | ESP_IMG_PIXEL_FMT_O_UYY_E_VYY | 35.8744 | 13.256 |
22+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_YUYV/UYVY | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 18.8679 | 3.805 |
23+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_YUYV/UYVY | ESP_IMG_PIXEL_FMT_RGB/BGR888 | 15.1515 | 4.6893 |
24+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR888 | ESP_IMG_PIXEL_FMT_I420 | 17.4292 | 2.6161 |
25+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR888 | ESP_IMG_PIXEL_FMT_O_UYY_E_VYY | 17.094 | 2.6729 |
26+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR888 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 18.5185 | 7.3733 |
27+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR888 | ESP_IMG_PIXEL_FMT_YUYV/UYVY | 15.6863 | 2.5707 |
28+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR888 | ESP_IMG_PIXEL_FMT_YUV_PLANNER | 11.7302 | 3.0361 |
29+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR888 | ESP_IMG_PIXEL_FMT_YUV_PACKET | 15.4143 | 3.1008 |
30+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_YUV_PACKET | ESP_IMG_PIXEL_FMT_I420 | 21.2766 | 10.8844 |
31+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_YUV_PACKET | ESP_IMG_PIXEL_FMT_O_UYY_E_VYY | 25.9109 | 11.7045 |
32+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_YUV_PACKET | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 14.7194 | 2.9701 |
33+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_YUV_PACKET | ESP_IMG_PIXEL_FMT_RGB/BGR888 | 13.0826 | 3.043 |
34+
35+
36+
## IMAGE ROTATE
37+
38+
| Width | Height | Format | degree | OPT Frame Per Second(fps) | C Frame Per Second(fps) |
39+
|-------|--------|-------------------------------|--------|---------------------------|-------------------------|
40+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 0 | | 20.9249 |
41+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 90 | | 14.4444 |
42+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 180 | | 4.2105 |
43+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 270 | | 15.3181 |
44+
| 1920 | 1088 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 0 | 20.7632 | |
45+
| 1920 | 1088 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 90 | 19.4413 | |
46+
| 1920 | 1088 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 180 | 23.679 | |
47+
| 1920 | 1088 | ESP_IMG_PIXEL_FMT_RGB/BGR565_LE/BE | 270 | 19.4525 | |
48+
49+
## IMAGE SCALE
50+
51+
| Input Width | Input Height | Format | Scale Width | Scale height | Filter type | C Frame Per Second(fps) |
52+
|-------------|--------------|-----------------------------|-------------|--------------|------------------------------------------|-------------------------|
53+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 240 | 540 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 44.3213 |
54+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 240 | 1080 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 22.2531 |
55+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 480 | 540 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 26.0586 |
56+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 480 | 1080 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 13.0719 |
57+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 720 | 540 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 18.0383 |
58+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 720 | 1080 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 9.06 |
59+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 960 | 540 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 14.1218 |
60+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 960 | 1080 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 7.1365 |
61+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1200 | 540 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 11.6959 |
62+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1200 | 1080 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 5.848 |
63+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1440 | 540 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 9.6502 |
64+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1440 | 1080 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 4.7933 |
65+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1680 | 540 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 8.8154 |
66+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1680 | 1080 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 4.374 |
67+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1920 | 540 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 8.0808 |
68+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1920 | 1080 | ESP_IMG_SCALE_FILTER_TYPE_DOWN_RESAMPLE | 4.0414 |
69+
70+
## IMAGE CROP
71+
note: The starting coordinate is (0, 0)
72+
73+
| Input Width | Input Height | Format | Clip Width | Clip height | C Frame Per Second(fps) |
74+
|-------------|--------------|-----------------------------|------------|-------------|-------------------------|
75+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 240 | 540 | 306.2201 |
76+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 240 | 1080 | 158.6121 |
77+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 480 | 540 | 161.8205 |
78+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 480 | 1080 | 83.1709 |
79+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 720 | 540 | 110.4400 |
80+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 720 | 1080 | 56.3628 |
81+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 960 | 540 | 85.0498 |
82+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 960 | 1080 | 43.1412 |
83+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1200 | 540 | 68.4126 |
84+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1200 | 1080 | 34.5852 |
85+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1440 | 540 | 57.2707 |
86+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1440 | 1080 | 28.9462 |
87+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1680 | 540 | 48.8550 |
88+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1680 | 1080 | 24.6438 |
89+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1920 | 540 | 42.5249 |
90+
| 1920 | 1080 | ESP_IMG_PIXEL_FMT_RGB565_LE | 1920 | 1080 | 21.2625 |

‎esp_image_effects/idf_component.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
dependencies:
2+
idf:
3+
version: '>=5.0'
4+
description: Espressif image effects
5+
issues: https://github.com/espressif/esp-adf/issues
6+
repository: https://github.com/espressif/esp-adf-libs.git
7+
url: https://github.com/espressif/esp-adf-libs/tree/master/esp_image_effects
8+
version: 1.0.0
9+
tags:
10+
- multimedia
11+
- scale
12+
- rotation
13+
- crop
14+
- color_conversion

0 commit comments

Comments
 (0)