feat(webview): optional unload on pause

[kariudo]

Summary

As #979 caused me to use up several GB of mobile data in short order due to closing the app with a dashboard of 10 cameras open via the frigate card, I thought I would just fix this myself and solve my own problem.

My solution (which works for me in my testing), is to offer a setting to support a highly effective (if not slightly aggressive) approach (somewhat recommended by google for freeing resources in webviews):

• Add Unload when backgrounded setting
• If onPause event fires (minimize, lock screeen, etc.):
  • Confirm settings and state, and unload the webviews resources by loading about:blank
• If onResume fires (screen unlocked, app restored from background):
  • Confirm settings and state, and simply "go back".
  • User will return to the last screen (URL).

The obvious caveat here to the solution is that it means pausing the app means the previous view must be reloaded (however, this is obviously needed when unloading has happened and ensures that no broken weirdness from trying to suspend the webview happens, at the trade-off of a reload).

Screenshots

Link to pull request in Documentation repository

Documentation: home-assistant/companion.home-assistant#

Any other notes

[jpelgrom]

This makes no sense to me as an end-user feature. There is a bug, and either 1) the bug should be fixed in the frontend/webview/exoplayer/wherever the problem lies, or 2) there should be something smart to do this. You're proposing implementing a workaround. As a normal user, or if the issue is fixed, this has no value.

[kariudo]

The bug is that there is an undisposed web view, so this conditionally in the most convenient way disposes of the resources. The only other practical option is to completely dispose the web view activity which seems silly.

This is a major problem for anyone who uses cameras.

Functionally this isn't so different than the existing option to navigate to the default dashboard URL every time the app opens; however that doesn't happen until the resume event, not the stop event.

[andonevris]

This makes no sense to me as an end-user feature. There is a bug, and either 1) the bug should be fixed in the frontend/webview/exoplayer/wherever the problem lies, or 2) there should be something smart to do this. You're proposing implementing a workaround. As a normal user, or if the issue is fixed, this has no value.

#979 has been ongoing for almost 5 years, yes that's 5 YEARS.

At this point I will take ANY fix even if it is a workaround, there seems to be no interest if fixing this the "proper way" whatever that means. I see no issue putting this fix in, it's optional and people who don't need it can leave things as they are.

For the people that this issue affects it's the most infuriating bug, coming back to your phone and realising you've just dumped 50% battery for no reason is ridiculous.

[MikeVensel]

This issue REALLY needs to be fixed and I see no reason not to take this fix. I don't even understand how this could be viewed as a workaround. The app is in the background. There should be no view refreshing in the background. Hell this "workaround" setting should be enabled by default. I can't imagine anyone expects an app to be using data and power to update a view they can't see in an app that is currently suspended.

[dshokouhi]

So the reason why we call this a workaround is because it needs to be discovered. Users dont know about #979 until they get bit. Likewise users will not know about the fix until they get bit AND they do research to find the setting to use it.

Considering the current changes, we had a user also mention Chrome has the same exact issue (#979 (comment)). Chrome does not navigate to a blank page when it is paused to free up resources. So this fix would only benefit the app when the issue exists elsewhere out of the apps control.

@kariudo have you tested the changes when PiP is used? What happens there when this is enabled and user triggers PiP? We need to consider existing functionality here

Also, you mentioned that Google sorta recommends this for freeing up resources can you share that link with us?

[kariudo]

So the reason why we call this a workaround is because it needs to be discovered. Users dont know about #979 until they get bit. Likewise users will not know about the fix until they get bit AND they do research to find the setting to use it.

That's probably unavoidable downstream short of defaulting to release of resources onPause.

Considering the current changes, we had a user also mention Chrome has the same exact issue (#979 (comment)). Chrome does not navigate to a blank page when it is paused to free up resources. So this fix would only benefit the app when the issue exists elsewhere out of the apps control.

I think their expectations are that consumer has context for what should be done, rather than behaving as iOS does (to my knowledge) and always disposing for resources.

@kariudo have you tested the changes when PiP is used? What happens there when this is enabled and user triggers PiP? We need to consider existing functionality here

I had not; however, I just did and in terms of frigate video streams on the frontend hosted in the Android app (I assume that's your question), the onPause event is still emit when pip is active (full screen does not cause an emit). So presumably for pip support to work with these changes it would need to check for:
https://developer.android.com/reference/android/app/Activity#isInPictureInPictureMode()

Also, you mentioned that Google sorta recommends this for freeing up resources can you share that link with us?

Sure. It is in the webview API doc, following the deprecation notice for the former clearview() method as the recommendation for reliable release of resources:

https://developer.android.com/reference/android/webkit/WebView#clearView()

[jpelgrom]

Even if PiP is adapted, I expect this still breaks other use cases that depend on the frontend not disappearing immediately like:

• Quickly switching between the app, for example for to-do lists or copying text from somewhere else
• External dialogs like Assist/server switching
• Improv Wi-Fi device onboarding
• QR code scanning

[kariudo]

Good morning,

Even if PiP is adapted, I expect this still breaks other use cases that depend on the frontend not disappearing immediately like:

Hence the optional setting... if you depend on it running forever in the background, then you probably want to be opting for the behavior you want need. Forcing the consumption of resources in the background seems far worse, regardless of downstream responsibility.

We can agree that it would be better if this was fixed in chromium/webview/etc; however, after 5 years that clearly isn't something we should be waiting for...

• Quickly switching between the app, for example for to-do lists or copying text from somewhere else

There are two options here, provide a timeout setting instead of just a boolean state to provide for "quick switching"; however, the whole point of this behavior currently is to immediately release all of the resources being consumed. I could see an additional setting to tune how long before it's disposed as a way to dial in the expectation.

If you have specific steps you would like me to try for any of the following, please feel free to provide them and I will give them a try on my build:

• External dialogs like Assist/server switching
• Improv Wi-Fi device onboarding
• QR code scanning

[TimoPtr]

I think unloading the WebView in this manner might be too aggressive I tried your proposal and the UX is not very nice. As discussed in the issue thread, the frontend already has a mechanism for unloading after 5 minutes (https://github.com/home-assistant/frontend/blob/54cc096b1acdd481ee5407a46439b071bdf509c1/src/layouts/partial-panel-resolver.ts#L140). However, this mechanism doesn't disconnect if there are iframes present. If you're using iframes for your camera, that could explain why it's not working as expected.
Have you considered using the new native WebRTC support for your camera? It might be a better solution.

@jpelgrom, perhaps we could introduce a delay before unloading, and check for Picture-in-Picture (PiP) instead of iframes. This could help avoid these situations where people are upset because of the data consumption. However, it might also mask underlying configuration issues on the user's end.

[kariudo]

I think unloading the WebView in this manner might be too aggressive I tried your proposal and the UX is not very nice. As discussed in the issue thread, the frontend already has a mechanism for unloading after 5 minutes (https://github.com/home-assistant/frontend/blob/54cc096b1acdd481ee5407a46439b071bdf509c1/src/layouts/partial-panel-resolver.ts#L140). However, this mechanism doesn't disconnect if there are iframes present. If you're using iframes for your camera, that could explain why it's not working as expected. Have you considered using the new native WebRTC support for your camera? It might be a better solution.

5 minutes is a very long time in terms of possible wasted resources, when the point of this proposal IS to be aggressive for those of us who need to avoid that scenario.

Quick estimation math would imply that someone with a dashboard with 10x 1080p camera feeds on it (I have 12 on one of mine currently), if each consumes 5Mbps, then in 5 minutes that could consume over 1.8 GB of data. That is unacceptable to me personally, and based on the comments of others here and the bug, I am not alone in this feeling.

I have been using my build for a month now (since this PR was opened), and I'm not sure I would describe the UX as so far as "not very nice", the end result is I have subscribed to the tradeoff of seeing the front end load when I open the app (which is probably only noticeable really because of the logo splash it uses), in exchange for significant resource savings. I can open my cameras on a cellular connection without fear of eating a whole plans worth of data short of having to immediately force quit the app to stop it from eating data.

If a delay (as I suggested above) is to be considered, it would also need to be configurable, as in my scenario any delay is unacceptable, but others may prefer some trade-off, or if this method was used in a default configuration a delay might make sense for them.

I would be perfectly interested in another solution that instantly stops resource usage when the app is closed; however, I have not seen any alternative suggested thus far.

@jpelgrom, perhaps we could introduce a delay before unloading, and check for Picture-in-Picture (PiP) instead of iframes. This could help avoid these situations where people are upset because of the data consumption. However, it might also mask underlying configuration issues on the user's end.

Tested scenarios were with native WebRTC support via both standard camera entity components as well as the (formerly Frigate) Advanced Camera Card integration and all supported engines go2rtc/webrtc/etc.

[TimoPtr]

Did you consider submitting this proposal to the frontend team instead of the app. Users accessing the platform through a browser on a paid network would face the same issue as those using the app. It's important to maintain consistent behavior across all dashboards to provide a uniform user experience.

From a UX standpoint, seeing the whole dashboard reload when switching from one app to another doesn't look good, especially when trying to quickly check something. This can disrupt the user flow and make the platform feel less responsive, especially when dashboards are big.

Additionally, we could leverage the system flag NET_CAPABILITY_NOT_METERED that indicates if the user is on an unlimited network. By detecting this flag, we can make this PR a bit smarter and manage data usage more effectively based on the user's network conditions.

[andonevris]

Additionally, we could leverage the system flag NET_CAPABILITY_NOT_METERED that indicates if the user is on an unlimited network. By detecting this flag, we can make this PR a bit smarter and manage data usage more effectively based on the user's network conditions.

It's not just high network use that's the issue but also battery drain, for my use case it is finding my phone with no battery left because I forgot to kill the app after looking at camera feeds that's the problem.

[TimoPtr]

Additionally, we could leverage the system flag NET_CAPABILITY_NOT_METERED that indicates if the user is on an unlimited network. By detecting this flag, we can make this PR a bit smarter and manage data usage more effectively based on the user's network conditions.

It's not just high network use that's the issue but also battery drain, for my use case it is finding my phone with no battery left because I forgot to kill the app after looking at camera feeds that's the problem.

In this specific case 5mn should be ok, and if you have a battery drain it is probably because you are using an iframe like I've describe in my previous answer. You should migrate to webRTC it would also improve the general experience with cameras.