Fills and Dropped Frames Under Restricted Bandwidth (3 Mbps)

[nirmala-ncompass]

Hello,
We recently updated RootEncoder from v2.7.0 to v2.8.0 and have been testing streaming across different Samsung device models.
Note : It is happening when adding filters.
Below are our observations:

1. Without bandwidth restriction
   Streaming works fine on most devices.
   On Samsung S22, we are observing fills and dropped frames.
   When streaming continuously for more than one hour, the S22 device also gets heated.
   We are observing these issues even when the network connection is stable.
   On Samsung A17, & Samsung M56 we have not observed these issues.
2. With bandwidth restricted to 3 Mbps
   With the bandwidth restricted to 3 Mbps, we are observing fills and dropped frames.
   We also noticed that audio and video are missed/interrupted at certain points during the stream. When checking the stream in MediaLive, we are seeing audio and video alerts.
   We also tested the same 3 Mbps restricted bandwidth scenario with RootEncoder v2.7.0, and we observed similar behavior, including the audio alerts.
   So, the audio/video issue under the 3 Mbps bandwidth restriction does not appear to be specific to v2.8.0.
3. Devices tested
   Samsung S22 – Occur fills and dropped frames with and without restriction; device heating during long streaming
   Samsung S24 – Samsung A17 – Without restriction working fine.
   Samsung M56 – Occur fills and dropped frames with 3 Mbps bandwidth restriction
   Questions
   Could you please help us understand:
   Why are we seeing fills and dropped frames on some Samsung models, particularly the S22, even when the network is stable?
   With bandwidth restricted to 3 Mbps, why are we seeing fills/dropped frames and audio/video interruptions?
   Since we also see similar audio alerts with v2.7.0 under the same 3 Mbps restriction, could this be related to the encoder, bitrate, network handling, or device behavior rather than the v2.8.0 upgrade?
   Is there any recommended configuration or approach in RootEncoder to handle these scenarios more reliably?
   Please let us know if you need any additional logs, encoder configuration, bitrate/resolution settings, or other details from our side.
   Thank you.

[pedroSG94]

Hello,

First of all, I need to know the video/audio configuration used to test and the way you did the bandwidth restriction to try investigate the case.
Also, let me know the media server name and version used to try use the same environment and if you check the fill and dropped frames on server side.
I'm assuming that you are streaming with rtmp

[pedroSG94]

Meanwhile, try this version:

  implementation "com.github.pedroSG94.RootEncoder:library:90bdca8c1e"

This is the last commit of master and include a possible fix for your case in video PTS.
Other 2 things that changes from version 2.7.0 that could help your case:

• Camera2Source now use dynamic fps disabled by default. Enabled it with:

camera2Source.setDynamicFps(true)

• If you are using AndroidViewfilter, that filter changed the way it is rendering and maybe produce the overheat. Try reduce the fps target:

androidViewFilterRender.targetFps = 15

[nirmala-ncompass]

Hi, We are using RtmpCamera2.
(rtmpCamera2.prepareAudio() && rtmpCamera2.prepareVideo(1280, 720, 30, (int) (3.20 * 1000 * 1000), 2, CameraHelper.getCameraOrientation(context)))
This is the initial target bitrate configuration and we are using adaptive bitrate logic in library.
We are not using any servers and versions as we are using aws medialive completely.
For the above changes I will try and let you know. Thanks

[pedroSG94]

Hello,

setDynamicFps method is available using RtmpCamera2 too.

I need know the way you use to restrict bandwidth in your devices and the way you check frames drops to reproduce the case 2 and help you in that way.

[nirmala-ncompass]

Hi @pedroSG94
For restricting the bandwidth, we are using an tplink router with QoS enabled, and we set the upload bandwidth limit to 3 Mbps.

While streaming, we monitor the MediaLive Health Metrics to check the stream health. We use these metrics to identify:

• Audio alerts
• Video alerts
• Frame drops and other streaming-related issues

This is how we are currently reproducing and monitoring the issue. Please let us know if you need any additional details.

We also tested the latest commit version you shared, but unfortunately we are not seeing any difference in the behavior.

The issues are still persisting:

• Fills and dropped frames are still occurring.
• The Samsung S22 is still getting heated during longer streaming sessions.
• Additionally, we noticed one more issue: the camera appears to jerk or refocus approximately every minute while streaming.
• We did not observe this camera focusing/jerking behavior previously.

Please let us know what else we can check from our side.

[pedroSG94]

Hello,

I did this commit fixing a bug in BitrateAdapter that maybe help with your fill / dropped frames. Try using it:

  implementation "com.github.pedroSG94.RootEncoder:library:67007d634a"

Test it without setDynamicFps that maybe is the problem with your jerk or refocus problem but let me know if you used it in the last tests as reference.

About the heated in s22, this is difficult to measure, are you sure that with version 2.7.0 you never had this problem? Do you have this problem using the commit shared version but without the filters? Which filters are you using?
If you are using AndroidViewFilterRender, are you using a webview? if yes, did you tried change the web to a static web to reduce the CPU usage to check if the problem is related with the webview?

Also, while streaming, try to record a video using startRecord method and share it, this will help measure possible pts problems maybe related with your fill / dropped frames (of course use stopRecord when you stop stream to close properly the video recorded).

Do 2 tests and all tests using the samsung s22 if possible:

• One test with your banwidth limited to 3mbps but using 3,2mbps in the library as you said
• Other test with the restriction to 3mbs but now using a bitrate lower than the limit. Like 2,5mbps in the library ( this case remember change the setmaxbitrate of bitrateadapter to the new library limit of 2,5mbps)

Just in case, the only tests of 1h that you need to do is about the heated in s22. Doing tests large enough to check if you have fill/dropped frames in 15-20 mins of stream is fine.

[nirmala-ncompass]

Hi @pedroSG94

We will test this commit version and let you know. Regarding your questions:

1. Heated in s22, it happened 2.7.0 version too.
2. We are using AndroidViewFilterRender.

One more thing I would like to ask, it's different issue, we are using AndroidMuxerRecordController for the record, we are modified in that class and using it. In this newer version I am facing issue in this modified class. could you please help me to resolve, issue is inside write method, there is calculateBitrate method called, in that second parameter I am getting error. I am using Java code. In library now it's modified everything into Kotlin. I am adding my modified code. Just help me to resolve? Meanwhile I will test the dropped frames issue and get back to you. Thanks!

AndroidMuxerRecordControllerModified.java

[nirmala-ncompass]

Hi, I removed setDynamicFps line and tested the above commit version, still I can see fills and dropped frames in restricted bandwidth, I attached screenshot of the graph, in this I can noticed fps is not maintained at 30. In my code I mentioned 30 fps in prepare video. I tested my Samsung M56 and shared this graph. Heat issue persists in S22 as well.

[pedroSG94]

Hello,

I did a new commit that should solve the problem with your AndroidMuxerRecordControllerModified class.

About the fill / dropped frames. This seems like a problem in the current BitrateAdapter class. I think that the library is not discarding or dropping frames, it is in server side. The library stuck sometimes sending packets because the badwidth is not enough and the current BitrateAdapter no help to it. So the sender stuck and when the packet that suck the sender is released the library send all the queue so the packets come too late and the server dropp it producing that graph.
I added a new bitrate adapter with a better algorithm that could reduce the problem of your fill / dropped frames. You can check it here:
https://github.com/pedroSG94/RootEncoder/blob/cd6ea8378ddfd62506f03f495402f1c5a42ba3de/app/src/main/java/com/pedro/streamer/rotation/CameraFragment.kt

This is the gradle to test:

  implementation "com.github.pedroSG94.RootEncoder:library:cd6ea8378d"

I'm waiting for:

• The recorded files
• The test using bandwidth restriction of 3mbps and bitrate of the library to 2,5mbps (a graph) to confirm that all is working with enough badwidth
• Confirm me that you are using a webview inside AndroidViewRenderFilter and if you tested using a static web and without filters using s22.

[nirmala-ncompass]

Hello,
Thank you for the new commit and for the updated BitrateAdapter. I tested the latest commit as well, but unfortunately the fills and dropped frames issue is still present. I also noticed that the FPS is not consistently maintained at 30 FPS during the test.
Regarding the points you requested:
1. Recorded file
I tested the latest commit for around 15 minutes and have attached the recorded video for your reference via link.
https://drive.google.com/drive/folders/1p2tnc0LGEn8Q_meKkSaCkLV62UTNojah?usp=sharing
2. 3 Mbps bandwidth restriction / bitrate test
For now, I tested with the library bitrate set to 3.2 Mbps and bandwidth restricted to 3 Mbps. I have attached the graph from this test as well.
During the 15-minute test, I observed audio and video alerts around 2–3 times. Whenever these alerts occurred, we stopped and restarted the streaming, as this is our current requirement when such an alert occurs.
The fills and dropped frames are still observed, and the FPS is also fluctuating instead of being consistently maintained at 30 FPS.
3. AndroidViewRenderFilter / WebView
We are using AndroidViewRenderFilter, but we are not using a WebView inside it. The view being rendered is an XML layout file.

[pedroSG94]

Hello,

First of all, after check the video recorded, we can discard a pts problem with the library. This is a problem on the bitrate adapter and the way medialive measures that graph.
The library send all correctly but some frames are received too late by medialive in bursts, producing the unstable fps graph, so they are discarded by medialive dropping it and generating fills to replace too late frames.

I updated the library to improve the new bitrate adapter class here:

  implementation "com.github.pedroSG94.RootEncoder:library:969d335f13"

This should stabilize the input video frame rate and reduce fill and dropped frames producing a result similar to stream with a bitrate under the bandwidth limit few seconds after start the stream.

[nirmala-ncompass]

Hi @pedroSG94
I have tested 15 minutes of stream, Its working fine. There is no fills and dropped frames issues. FPS also maintained. Will test for long stream and let you know. And also we noticed one thing which I mentioned earlier, In this above commit version, I am facing that jerk/focus issue every second.

[tejas-Ncompass]

Hello,
We tested the stream for a longer duration with the bandwidth restricted to 2.5 Mbps, and we are still experiencing fills and frame drops.
I have shared the graph below for your reference.

Additionally, the jerk/focus issues mentioned previously still persist. The stream video loses and regains focus every second, causing a continuous jerk in the video.
This is visible on both the preview surface and within the final stream video, this issue only occurs when autofocus is enabled (set to true).

[pedroSG94]

Hello,

According with the last graph. It is working as expected.
The alerts after few seconds is 0, the input fps is a lot more stable and the fill/dropped frames are now residual compared with the previous graph. You can check it with the lateral number, the value is reduced a lot:

• Fill milliseconds: peak went from 15,400 ms to 51 ms
• Dropped Frame Rates: from 6.3 to 0.8
• Input Video Frame Rate: from 8 – 38 to 26 – 31
• And Active Alerts stayed at 0 for the whole 50 minutes after the first seconds

About the focus, I can't reproduce it in my device but I did a fix in the enableAutoFocus that maybe help with your case here:

  implementation "com.github.pedroSG94.RootEncoder:library:e34e12c99e"

Test it and try with setDynamicFps(true) (you need call it before prepareVideo method) and without it.

[nirmala-ncompass]

Hi @pedroSG94
Thank you for the update. I tested the latest commit (e34e12c) on Samsung, Motorola, and Pixel devices.
My observations are:

1. Samsung: With setDynamicFps(true), the stream has noticeable camera jerking/stuttering. Without dynamic FPS, the stream is smooth.
2. Motorola: Camera jerking is present both with and without dynamic FPS.
3. All tested devices: Fill and dropped frames are still occurring.
4. I have attached the graph showing the fill and dropped-frame behaviour.
   https://drive.google.com/drive/folders/19mWlOV59fr_UrIA7rm59EVVriat9pFI7?usp=share_link
   So, the autofocus issue is still reproducible, and the fill/dropped-frame issue is occurring across all tested devices.
   Please help us resolve these issues as soon as possible.