Limbs disappearing and flickering with virtual backgrounds

[anaismahmed]

I've been encountering this in a Flutter app but can also repro it in the Twilio demo app.

• I have verified that the issue occurs with the latest twilio-video-processors.js release and is not marked as a known issue in the CHANGELOG.md.
• I reviewed the Common Issues and open GitHub issues and verified that this report represents a potentially new issue.
• I am not sharing any Personally Identifiable Information (PII)
  or sensitive account information (API keys, credentials, etc.) when reporting this issue.

Code to reproduce the issue:

This is present in the demo app in examples.

Expected behavior:

Limbs stay present when using virtual backgrounds and don't disappear or flicker.

Actual behavior:

Limbs flicker in and out and disappear suddenly.

Software versions:

• Browser(s): Firefox, Chrome 127.0.6533.89
• Operating System: Mac OS Sonoma 14.5
• twilio-video-processors.js: 2.2.0
• Third-party libraries (e.g., Angular, React, etc.): Flutter (but also present in the demo app)

[luisrivas]

Hi @anaismahmed!

The accuracy of segmentation masks can be impacted by certain conditions, such as uneven lighting and increased body movement. Could you provide more details about your test? Additionally, I recommend using our new v3.x example.

https://twilio.github.io/twilio-video-processors.js/versions/3.0.0-beta.1/examples/virtualbackground/

[anaismahmed]

Hello @luisrivas,

I've been trying the new example you provided in a variety of lighting conditions. However, I still have trouble with flickering around the peripheries of my limbs, even when keeping them still, as well as around my hair. Is there a setting that I can use to try and mitigate some of this?