Skip to content

Commit 0b37d3c

Browse files
jason-maojason-mao
committed
Merge branch 'feature/adf_add_media_protocols_component' into 'master'
Add esp_media_protocols library See merge request adf/esp-adf-libs!294
2 parents f9ee36a + 2e5c479 commit 0b37d3c

32 files changed

+1031
-19
lines changed

‎.github/workflows/upload_component.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,5 +24,6 @@ jobs:
2424
esp_video_codec;
2525
esp_libsrtp;
2626
esp_image_effects;
27+
esp_media_protocols;
2728
namespace: "espressif"
28-
api_token: ${{ secrets.IDF_COMPONENT_API_TOKEN }}
29+
api_token: ${{ secrets.IDF_COMPONENT_API_TOKEN }}

‎CMakeLists.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,8 @@ IF (IDF_VERSION_MAJOR GREATER 3)
7373
PRIV_REQUIRES esp-adf-libs)
7474
add_prebuilt_library(esp_new_jpeg "${CMAKE_CURRENT_SOURCE_DIR}/esp_new_jpeg/lib/${CONFIG_IDF_TARGET}/libesp_new_jpeg.a"
7575
PRIV_REQUIRES esp-adf-libs)
76+
add_prebuilt_library(esp_media_protocols "${CMAKE_CURRENT_SOURCE_DIR}/esp_media_protocols/lib/${CONFIG_IDF_TARGET}/libesp_media_protocols.a"
77+
PRIV_REQUIRES esp-adf-libs)
7678
add_prebuilt_library(esp_h264 "${CMAKE_CURRENT_SOURCE_DIR}/${esp_h264_name}/lib/${CONFIG_IDF_TARGET}/lib${esp_h264_name}.a"
7779
PRIV_REQUIRES esp-adf-libs)
7880
add_prebuilt_library(esp_muxer "${CMAKE_CURRENT_SOURCE_DIR}/esp_muxer/lib/${CONFIG_IDF_TARGET}/libesp_muxer.a"

‎esp_media_protocols/CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Changelog
2+
3+
## v0.5.0
4+
5+
### Features
6+
7+
- Initial version of `esp_media_protocols`
8+
- Add media protocols library
9+
- Add a breaking change for the pts added to esp rtsp send stream

‎esp_media_protocols/CMakeLists.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
idf_component_register(INCLUDE_DIRS "include")
2+
get_filename_component(BASE_DIR ${CMAKE_CURRENT_SOURCE_DIR} NAME)
3+
add_prebuilt_library(${BASE_DIR} "${CMAKE_CURRENT_SOURCE_DIR}/lib/${CONFIG_IDF_TARGET}/libesp_media_protocols.a"
4+
PRIV_REQUIRES ${BASE_DIR})
5+
target_link_libraries(${COMPONENT_LIB} INTERFACE "-L ${CMAKE_CURRENT_SOURCE_DIR}/lib/${CONFIG_IDF_TARGET}")
6+
target_link_libraries(${COMPONENT_LIB} INTERFACE esp_media_protocols)

‎esp_media_protocols/LICENSE‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
ESPRESSIF MIT License
2+
3+
Copyright (c) 2025 <ESPRESSIF SYSTEMS (SHANGHAI) CO.，LTD>
4+
5+
Permission is hereby granted for use on all ESPRESSIF SYSTEMS products, in which case,
6+
it is free of charge, to any person obtaining a copy of this software and associated
7+
documentation files (the "Software"), to deal in the Software without restriction, including
8+
without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
9+
and/or sell copies of the Software, and to permit persons to whom the Software is furnished
10+
to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all copies or
13+
substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
17+
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
18+
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
19+
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
20+
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

‎esp_media_protocols/README.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# ESP_MEDIA_PROTOCOLS
2+
[中文版](./README_CN.md)
3+
4+
## Overview
5+
6+
The ESP Media Protocols library provides comprehensive implementations of popular media streaming and communication protocols optimized for Espressif Socs. This library enables developers to build sophisticated multimedia applications including VoIP systems, streaming servers, smart home devices, and synchronized audio systems.
7+
8+
# Features
9+
10+
## Supported Protocols
11+
12+
| Protocol |Description |
13+
|------------|------------------------------------------------------------------|
14+
| SIP | Session Initiation Protocol, suitable for establishing, modifying and terminating calls, commonly used in VoIP communications |
15+
| RTSP | Real Time Streaming Protocol, supports media streaming play, pause, seek and etc. |
16+
| RTMP | Real-Time Messaging Protocol, suitable for pushing and pulling live streaming |
17+
| MRM | Multi-Room Music, supports audio distribution and sync across devices |
18+
| UPnP | Universal Plug and Play, allows devices to discovery and interoperate, commonly used for home network devices |
19+
20+
### SIP Protocol
21+
22+
Session Initiation Protocol implementation for audio and video communication over IP networks.
23+
- **Readme:** [ESP_SIP](docs/en/SIP_README.md)
24+
- **Headers:** [`esp_rtc.h`](include/esp_rtc.h), [`esp_sip.h`](include/esp_sip.h)(deprecated)
25+
- **Key Features:**
26+
- Multiple audio/video codecs
27+
- DTMF support (RFC2833)
28+
- SSL/TLS encryption
29+
- Custom SIP headers
30+
- **Use Cases:**
31+
- IP phones and VoIP applications
32+
- Video conferencing systems
33+
- Intercom systems
34+
- Emergency communication devices
35+
36+
### RTSP Protocol
37+
38+
Real-Time Streaming Protocol for audio and video streaming applications.
39+
- **Readme:** [ESP_RTSP](docs/en/RTSP_README.md)
40+
- **Headers:** [`esp_rtsp.h`](include/esp_rtsp.h)
41+
- **Key Features:**
42+
- Multiple audio/video codecs
43+
- UDP/TCP transport
44+
- Server and client modes
45+
- Push and pull operations
46+
- Stream management (DESCRIBE, SETUP, PLAY, TEARDOWN)
47+
- **Use Cases:**
48+
- IP cameras and surveillance systems
49+
- Video streaming servers
50+
- Media players and recorders
51+
- Remote monitoring applications
52+
53+
### RTMP Protocol
54+
55+
Real-Time Messaging Protocol for live streaming to platforms and custom servers.
56+
- **Readme:** [ESP_RTMP](docs/en/RTMP_README.md)
57+
- **Headers:** [`esp_rtmp_push.h`](include/esp_rtmp_push.h), [`esp_rtmp_server.h`](include/esp_rtmp_server.h), [`esp_rtmp_src.h`](include/esp_rtmp_src.h), [`esp_rtmp_types.h`](include/esp_rtmp_types.h)
58+
- **Key Features:**
59+
- Multiple audio/video codecs
60+
- SSL/TLS encryption(RTMPS)
61+
- RTMP client (push streams)
62+
- RTMP server (receive and distribute streams)
63+
- RTMP source (pull streams)
64+
- **Use Cases:**
65+
- Live streaming to YouTube/Twitch
66+
- Custom streaming platforms
67+
- IoT camera streaming
68+
- Content distribution networks
69+
70+
### MRM Protocol
71+
72+
Multi-Room Music Protocol for whole-home multi-device synchronized audio playback system.
73+
- **Readme:** [ESP_MRM_Client](docs/en/MRM_README.md)
74+
- **Headers:** [`esp_mrm_client.h`](include/esp_mrm_client.h)
75+
- **Key Features:**
76+
- Multicast communication
77+
- Master-slave architecture
78+
- Dynamic role assignment
79+
- Network time synchronization
80+
- Automatic audio sync adjustment
81+
- **Use Cases:**
82+
- Whole-home audio systems
83+
- Synchronized speaker groups
84+
- Commercial audio installations
85+
- Party mode applications
86+
87+
### UPnP Protocol
88+
89+
Universal Plug and Play with DLNA compliance for seamless media sharing and device discovery.
90+
- **Readme:** [ESP_UPnP](docs/en/UPnP_README.md)
91+
- **Headers:** [`esp_ssdp.h`](include/esp_ssdp.h), [`esp_upnp.h`](include/esp_upnp.h), [`esp_upnp_service.h`](include/esp_upnp_service.h), [`esp_upnp_notify.h`](include/esp_upnp_notify.h)
92+
- **Key Features:**
93+
- HTTP server integration
94+
- XML-based service descriptions
95+
- Device discovery via SSDP
96+
- SOAP-based control actions
97+
- Event notifications
98+
- Media server/renderer/controller roles
99+
- **Use Cases:**
100+
- Media servers and NAS devices
101+
- Smart TV and streaming devices
102+
- Home automation systems
103+
- Media center applications
104+
105+
## Protocol Comparison Matrix
106+
107+
| Protocol | Real-time | Streaming | Control | Discovery | SSL/TLS | Complexity |
108+
|----------|-----------|-----------|---------|-----------|---------|------------|
109+
| SIP | High | Yes | Yes | Manual | Yes | Medium |
110+
| RTSP | High | Yes | Yes | Manual | No | Medium |
111+
| RTMP | Medium | Yes | Basic | Manual | Yes | Medium |
112+
| MRM | High | Yes | Yes | Auto | No | Low |
113+
| UPnP | Low | Yes | Yes | Auto | No | Medium |
114+
115+
# Supported chip
116+
117+
The following table shows the support of ESP_MEDIA_PROTOCOLS for Espressif SoCs. The "&#10004;" means supported, and the "&#10006;" means not supported.
118+
119+
|Chip | v0.5.0 |
120+
|:-----------:|:----------------:|
121+
|ESP32 | &#10004; |
122+
|ESP32-S2 | &#10004; |
123+
|ESP32-S3 | &#10004; |
124+
|ESP32-C2 | &#10004; |
125+
|ESP32-C3 | &#10004; |
126+
|ESP32-C5 | &#10004; |
127+
|ESP32-C6 | &#10004; |
128+
|ESP32-P4 | &#10004; |

‎esp_media_protocols/README_CN.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# ESP_MEDIA_PROTOCOLS
2+
[English](./README.md)
3+
4+
## 概述
5+
6+
ESP Media Protocols 库为 Espressif SoC 提供了主流媒体流和通信协议的完整实现。该库帮助开发者快速构建包括 VoIP 系统、流媒体服务器、智能家居设备、音频同步系统等多种多媒体应用。
7+
8+
# 特性
9+
10+
## 支持的协议
11+
12+
| 协议 | 描述 |
13+
|-----------|--------------------------------------------------------|
14+
| SIP | 会话发起协议，适用于建立、修改和终止呼叫，常用于 VoIP 通信 |
15+
| RTSP | 实时流协议，支持媒体流的播放、暂停、快进等操作 |
16+
| RTMP | 实时消息协议，适用于推流和拉流直播 |
17+
| MRM | 多设备共享音乐协议，多设备音频分发与同步 |
18+
| UPnP | 通用即插即用协议，实现设备发现与互操作，常用于家庭网络设备 |
19+
20+
### SIP 协议
21+
22+
Session Initiation Protocol，基于 IP 网络实现音视频通话。
23+
- **说明文档:** [ESP_SIP](docs/zh_CN/SIP_README_CN.md)
24+
- **头文件:** [`esp_rtc.h`](include/esp_rtc.h)，[`esp_sip.h`](include/esp_sip.h)(deprecated)
25+
- **功能特性:**
26+
- 支持多种音视频格式
27+
- DTMF 支持（RFC2833）
28+
- SSL/TLS 加密
29+
- 支持自定义 SIP 头
30+
- **典型应用:**
31+
- IP 电话与 VoIP 应用
32+
- 视频会议系统
33+
- 可视对讲
34+
- 紧急通信设备
35+
36+
### RTSP 协议
37+
38+
Real-Time Streaming Protocol，实现音视频流传输控制。
39+
- **说明文档:** [ESP_RTSP](docs/zh_CN/RTSP_README_CN.md)
40+
- **头文件:** [`esp_rtsp.h`](include/esp_rtsp.h)
41+
- **功能特性:**
42+
- 支持多种音视频格式
43+
- UDP/TCP 传输
44+
- 支持服务端和客户端模式
45+
- 支持推流和拉流
46+
- 媒体流管理（DESCRIBE, SETUP, PLAY, TEARDOWN）
47+
- **典型应用:**
48+
- 网络摄像头与安防系统
49+
- 视频流媒体服务器
50+
- 媒体播放器与录像机
51+
- 远程监控
52+
53+
### RTMP 协议
54+
55+
Real-Time Messaging Protocol，实现与平台或自建服务器的直播流应用。
56+
- **说明文档:** [ESP_RTMP](docs/zh_CN/RTMP_README_CN.md)
57+
- **头文件:** [`esp_rtmp_push.h`](include/esp_rtmp_push.h)，[`esp_rtmp_server.h`](include/esp_rtmp_server.h)，[`esp_rtmp_src.h`](include/esp_rtmp_src.h)，[`esp_rtmp_types.h`](include/esp_rtmp_types.h)
58+
- **功能特性:**
59+
- 支持多种音视频格式
60+
- SSL/TLS 加密（RTMPS）
61+
- RTMP 客户端（推流）
62+
- RTMP 服务端（接收与分发流）
63+
- RTMP 源（拉流）
64+
- **典型应用:**
65+
- YouTube/Twitch 等平台直播
66+
- 自建流媒体平台
67+
- IoT 摄像头推流
68+
- 网络分发
69+
70+
### MRM 协议
71+
72+
Multi-Room Music，实现多设备音频同步播放。
73+
- **说明文档:** [ESP_MRM_Client](docs/zh_CN/MRM_README_CN.md)
74+
- **头文件:** [`esp_mrm_client.h`](include/esp_mrm_client.h)
75+
- **功能特性:**
76+
- 组播通信
77+
- 主从架构
78+
- 动态角色分配
79+
- 网络时间同步
80+
- 自动音频同步调整
81+
- **典型应用:**
82+
- 全屋音响系统
83+
- 多设备同步播放
84+
- 商业音频系统
85+
- 派对应用
86+
87+
### UPnP 协议
88+
89+
Universal Plug and Play，实现媒体共享和设备发现，支持 DLNA 规范。
90+
- **说明文档:** [ESP_UPnP](docs/zh_CN/UPnP_README_CN.md)
91+
- **头文件:** [`esp_ssdp.h`](include/esp_ssdp.h)，[`esp_upnp.h`](include/esp_upnp.h)，[`esp_upnp_service.h`](include/esp_upnp_service.h)，[`esp_upnp_notify.h`](include/esp_upnp_notify.h)
92+
- **功能特性:**
93+
- 集成 HTTP 服务器
94+
- 基于 XML 的服务描述
95+
- 通过 SSDP 设备发现
96+
- 基于 SOAP 的控制
97+
- 事件通知
98+
- 支持媒体服务器/渲染器/控制器
99+
- **典型应用:**
100+
- 媒体服务器与 NAS
101+
- 智能电视与流媒体设备
102+
- 家庭自动化系统
103+
- 媒体中心应用
104+
105+
## 协议特性对比
106+
107+
| 协议 | 实时性 | 数据流 | 控制流 | 设备发现 | 加密 | 复杂度 |
108+
|--------|------- |-------|-------|---------|--------|--------|
109+
| SIP | 高 | 支持 | 支持 | 手动 | 支持 | 中 |
110+
| RTSP | 高 | 支持 | 支持 | 手动 | 不支持 | 中 |
111+
| RTMP | 中 | 支持 | 基本 | 手动 | 支持 | 中 |
112+
| MRM | 高 | 支持 | 支持 | 自动 | 不支持 | 低 |
113+
| UPnP | 低 | 支持 | 支持 | 自动 | 不支持 | 中 |
114+
115+
# 支持的芯片
116+
117+
下表展示了 ESP_MEDIA_PROTOCOLS 对 Espressif SoC 的支持情况。“&#10004;”表示支持，“&#10006;”表示不支持。
118+
119+
|芯片名称 | v0.5.0 |
120+
|:----------:|:----------------:|
121+
|ESP32 | &#10004; |
122+
|ESP32-S2 | &#10004; |
123+
|ESP32-S3 | &#10004; |
124+
|ESP32-C2 | &#10004; |
125+
|ESP32-C3 | &#10004; |
126+
|ESP32-C5 | &#10004; |
127+
|ESP32-C6 | &#10004; |
128+
|ESP32-P4 | &#10004; |

‎esp_media_protocols/docs/en/MRM_README.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# MRM Protocol
2+
[中文版](../zh_CN/MRM_README_CN.md)
3+
4+
Multi-Room Music Protocol for whole-home multi-device synchronized audio playback system.
5+
6+
# Features
7+
8+
- Multicast communication
9+
- Master-slave architecture
10+
- Dynamic role assignment
11+
- Network time synchronization
12+
- Automatic audio sync adjustment
13+
14+
# Use Cases
15+
16+
- Whole-home audio systems
17+
- Synchronized speaker groups
18+
- Commercial audio installations
19+
- Party mode applications
20+
21+
# Performance
22+
23+
Tested with the following system configuration:
24+
| Chip | IDF Version | CPU Frequency | SPI Ram Frequency |
25+
| --- | --- | --- | --- |
26+
| ESP32-S3N16R8 | v5.3 | 240MHz | 80MHz |
27+
28+
## Resource Utilization
29+
30+
| Total Memory(Byte)| Inram Memory(Byte) | Psram Memory(Byte) | CPU loading(%) |
31+
| --- | --- | --- | --- |
32+
| 11784 | 1020 | 10764 | < 0.01 |
33+
34+
Note:
35+
1) The CPU load values in the table represent average measurements.
36+
2) The test result only represents protocol stack. For details on data including media stream processing, please refer to the example docs below.
37+
38+
# Best Practice
39+
40+
Here is an example of using [Multi-Room Music](https://github.com/espressif/esp-adf/tree/master/examples/advanced_examples/multi-room)
41+
42+
# FAQ
43+
44+
refer to [mrm issue](https://github.com/espressif/esp-adf/issues?q=is%3Aissue%20mrm)

0 commit comments

Comments
 (0)