fix: reverse VP8 simulcast encodings for low-res Chromium (fixes #2939)

[4rayaditya]

Problem

VP8 simulcast on Chromium at resolutions <640px transmits heavily downscaled video:

• 320×240 camera → 80×60 output (should be 320×240)
• 480×360 camera → 240×180 output (should be 480×360)

Root cause: Chromium limits simulcast layers at low resolutions and picks the first encoding. Default order [4x, 2x, 1x] causes it to select the 4× downscaled layer.

Solution

Reverse encoding order to [1x, 2x, 4x] for VP8 camera tracks <640px on Chromium, ensuring the highest quality layer is selected.

Changes

• modules/RTC/TPCUtils.ts: Added conditional reversal in _getVideoStreamEncodings()
• modules/RTC/TPCUtils.spec.ts: Added tests for 320px and 480px resolutions

Fixes #2939

[jitsi-jenkins]

Hi, thanks for your contribution!
If you haven't already done so, could you please make sure you sign our CLA (https://jitsi.org/icla for individuals and https://jitsi.org/ccla for corporations)? We would unfortunately be unable to merge your patch unless we have that piece :(.

[4rayaditya]

I have made the changes changing the check from !firefox to isChromiumBased()

[4rayaditya]

Ok se when we reversed the array, SIM_LAYERS[0] no longer meant 4.0, it meant 1.0! This broke the activation logic. I used the values like 4.0 and 1.0 to solve this error

[jallamsetty1]

This will break the default encoding behavior on Chromium which has always been 4:2:1

[jallamsetty1]

Have you tested this on older versions of Chromium? What about other codecs that support simulcast, AV1 and VP9?

[4rayaditya]

@jallamsetty1
I think it preserves the encoding as 4:2:1 but for VP8 only. I am working on AV1 and VP9 right now.
As for testing, I have not tried on older versions of chromium yet. But I'll look at it.

[saghul]

I'm wondering what the value of simulcast is at those resolutions? Would disabling it be acceptable there @jallamsetty1 ?

[jallamsetty1]

The correct way to fix this would be to configure the correct number of encodings based on the captured resolution. If only one layer is needed then we would be effectively disabling simulcast.

[4rayaditya]

The correct way to fix this would be to configure the correct number of encodings based on the captured resolution. If only one layer is needed then we would be effectively disabling simulcast.

can i work on it?

[jallamsetty1]

The correct way to fix this would be to configure the correct number of encodings based on the captured resolution. If only one layer is needed then we would be effectively disabling simulcast.

can i work on it?

Yes go ahead and give it a try.

[jallamsetty1]

This needs to be fixed by configuring the right number of encodings based on the captured resolution.

[4rayaditya]

@jallamsetty1 So we need to dynamically configure the number of simulcast encodings based on resolution instead of hardcoding

[4rayaditya]

The code checks the captured video resolution and determines how many simulcast layers are appropriate (1, 2, or 3).
For VP8 on Chromium at low resolutions (<640px), it reverses the encoding order so the highest quality layer is sent.
For other codecs (VP9, AV1, H264), and higher resolutions, it configures layers and scalability modes according to browser and config settings.
If only one layer is needed, simulcast is effectively disabled.
All these decisions are made dynamically, so the encoding adapts to the user's device, browser, and video settings without hardcoded values.
@saghul @jallamsetty1

[jallamsetty1]

The code checks the captured video resolution and determines how many simulcast layers are appropriate (1, 2, or 3). For VP8 on Chromium at low resolutions (<640px), it reverses the encoding order so the highest quality layer is sent. For other codecs (VP9, AV1, H264), and higher resolutions, it configures layers and scalability modes according to browser and config settings. If only one layer is needed, simulcast is effectively disabled. All these decisions are made dynamically, so the encoding adapts to the user's device, browser, and video settings without hardcoded values. @saghul @jallamsetty1

How did you test if your solution is working as expected? If you already determining the # of simulcast layers based on the capture resolution, why do you have to reverse the encoding order?

[4rayaditya]

@jallamsetty1 I checked it on Chrome and tested it using WebRTC internal, although my camera doesnt support 720p or more. I tested it for the low resolution ones and found single encoding in the peer connection.
Although the code determines the number of simulcast layers dynamically but Chromium always selects the fist encoding in the array when one layer is possible at low resolutions. Reversing means highest quality appears first, so it picks best resolution.

[4rayaditya]

@jallamsetty1 did you get to check the issues?
Should I try and improve something?

[jallamsetty1]

How is this code getting executed if it after the return statement? Am I missing something here?

[jallamsetty1]

Instead of changing the encodings config here based on the captured resolution, you should figure out a way to change the number of simulcast layers needed and configure

lib-jitsi-meet/service/RTC/StandardVideoSettings.ts

Line 4 in f470b5f

export const SIM_LAYERS = [

accordingly. That value is used elsewhere to determine the number of simulcast streams and is used for SDP munging and other calculations. This needs to be done only when the video source is configured for the first time it is added to the conference.

[4rayaditya]

@jallamsetty1 please could you rate this approach

• The function _getVideoStreamEncodings dynamically determines the number of simulcast layers numLayers based on the video track’s capture resolution:

  • 1 layer for low resolutions (<360p)
  • 2 layers for medium resolutions (360p–719p)
  • 3 layers for high resolutions (≥720p)

• For each layer, it creates a simulcast encoding entry with a unique rid and a scaleFactor that determines how much the resolution is downscaled for that layer.

• The function builds arrays for the scale factors and bitrates, matching the number of layers.

• It then constructs the simulcastEncodings array, where each entry specifies:

  • Whether the layer is active
  • The maximum bitrate for that layer
  • The rid and scaleResolutionDownBy (downscale factor)

• For VP8 on Chromium browsers at low resolutions, the array is reversed to ensure the highest quality layer is first (due to a browser quirk).

• If scalability mode is enabled, the function sets the scalabilityMode property for each encoding as needed.

• The function returns the constructed array, which is then used to configure the RTCRtpSender.