Scrollbar Styling tests don't work with overlay scrollbars #898
Description
Test List
/css/css-overflow/scrollbar-gutter-001.html
/css/css-overflow/scrollbar-gutter-dynamic-001.html
/css/css-overflow/scrollbar-gutter-propagation-001.html
/css/css-overflow/scrollbar-gutter-propagation-002.html
/css/css-overflow/scrollbar-gutter-propagation-003.html
/css/css-overflow/scrollbar-gutter-propagation-007.html
/css/css-overflow/scrollbar-gutter-rtl-001.html
/css/css-overflow/scrollbar-gutter-vertical-lr-001.html
/css/css-overflow/scrollbar-gutter-vertical-rl-001.html
/css/css-scrollbars/scrollbar-width-001.html
/css/css-scrollbars/scrollbar-width-002.html
/css/css-scrollbars/scrollbar-width-003.html
/css/css-scrollbars/scrollbar-width-004.html
/css/css-scrollbars/scrollbar-width-005.html
/css/css-scrollbars/scrollbar-width-006.html
/css/css-scrollbars/scrollbar-width-008.html
/css/css-scrollbars/scrollbar-width-009.html
/css/css-scrollbars/scrollbar-width-011.html
/css/css-scrollbars/scrollbar-width-013.html
/css/css-scrollbars/scrollbar-width-015.html
/css/css-scrollbars/scrollbar-width-016.html
Rationale
WPT does not have any way to force classic scrollbars (web-platform-tests/wpt#36422) nor force some mocked scrollbars to be used (web-platform-tests/wpt#42697) to use classic scrollbars where the underlying system does not support them.
web-platform-tests/wpt#44196 / web-platform-tests/wpt#44195 reported a number of problems with scrollbars, if overlay scrollbars are used, including in the tests in our focus area.
This has caused a significant regression in the focus area score for Safari when we moved to Apple-managed infrastructure: https://wpt.fyi/results/css?q=label%3Ainterop-2024-scrollbar&diff&filter=ADC&run_id=5116283788197888&run_id=5200055011377152.
We currently plan on changing the configuration of macOS on the WPT bots to always use classic scrollbars, but the problem here is more fundamental insofar as we have tests relying on a scrollbar type that WPT does not guarantee nor can WPT tests opt in to.
There's no immediate actual test change being proposed here, as the only possible change with current WPT semantics would be to exclude large numbers of tests from the Scrollbar Styling focus area (and it's very hard to test scrollbars when they're overlay scrollbars which are only showed while scrolling!), however the Interop group should be aware of the problems here.