About the Live Streaming Issue with RTMP Chunk Buffer, Custom Video Effect when RTMP Stream and Stream Recorder simultaneously run

[wchklaus97]

Version of 'HaishinKit', '2.0.0-rc.1'

I am working on an R&D project about complex layers in live streaming with simultaneous saving of the live stream into photos on iOS. The project involves frequent updates to the complex layer, up to every 0.1 seconds.

I noticed your last commit (697c793) where you improved and optimized the usage of RTMPChunkBuffer handling. I am facing similar issues and wanted to ask about the kinds of problems you encountered.

My project is causing issues due to heavy video effects being applied with frequent updates (as I have to convert custom UI in Swift to CIImage and trigger video effect updates). I found that there's a 20% chance of encountering ChunkBuffer issues in my project. Additionally, I've noticed that under these complex conditions, CPU and memory usage grow very rapidly (I have optimized thread safety and queue execution in my project).

However, I haven't yet applied any solutions to handle the reuse of the complex layer. Before optimizing my project further, I would like to hear your ideas. Thank you.

The screenshot of the RTMPChunkBuffer issue is attached below.

In my current setup, I have applied two queues: one called updateQueue used to handle updates, and another called renderQueue used to handle rendering. This approach was implemented to avoid issues with the 'RTMP Chunk Buffer'. I found that while this solution works, it is only suitable for lightweight custom layers (simple UI with fewer video effects).

[shogo4405]

Simply put, crashes related to buffer overrun/underrun were occurring. Memory optimization has been implemented, and memory is now properly allocated to prevent buffer overruns. #1605

The issue was occurring when the video bitrate was increased to 2Mbps

[wchklaus97]

Sorry to bother you, but is the buffer handling feature also ready to be added and supported in version 2.0.0-rc.1? Since I am using this version for my R&D, it would be great if the buffer handling feature could be included in this pre-release version, as it would greatly assist me in my further research and development.

[shogo4405]

No, it’s not possible since version 2.0.0-rc.1 has already been released. It will be included in the next 2.0.0-rc.2 release.

Since you are in the research stage, I thought you could pull from the main branch, but would that be too much trouble? If you are using SPM or CocoaPods, I believe you should be able to do it.