Ability is needed to capture styles of elements to ensure proper optimization

[westonruter]

Sometimes the Intersection Observer doesn't provide all of the information needed in order to perform optimizations. For example, images which have visibility:hidden, display:none, or opacity:0 should in theory get fetchpriority=low when they are in the initial viewport per #1587, but these won't

Conversely, from #1308 (comment), images which are inside of elements which have content-visibility:auto will have a zeroed-out boundingClientRect which Image Prioritizer then interprets as being in the initial viewport (since the boundingClientRect.top of 0 is less than viewport.height) and so they are getting fetchpriority=low when they should actually get loading=lazy.

These issues should perhaps have a way to efficiently query the computed style of certain elements to store their properties with the element in URL Metrics. There could be an async getComputedStyle function exported to the initialize and finalize async functions in extensions which could include some additional performance safeguards, like returning a Promise that involves requestAnimationFrame() or maybe even first requestIdleCallback(). Calling getComputedStyle() can trigger a layout calculation (reflow), so we need to be careful to not negatively impact the performance of the page.

I put together a Codepen that shows what Intersection Observer reports for various scenarios: https://codepen.io/westonruter/pen/zxYoNJP?editors=1111

Notice importantly that elements with visibility:hidden and opacity:0 are considered fully visible as far as Intersection Observer is concerned, leading to them not getting fetchpriority=low. And notice how the element with content-visibility:auto has a boundingClientRect with zeroed out values, leading it to not get loading=lazy by Image Prioritizer.

In addition to the possibility of a new JS API, we should extend the element schema to include a computedStyle object. In fact, let's say instead of getComputedStyle() there was a captureElementStyles() function passed to initialize and finalize. It could take two args: the XPath of the element and an array of the styles properties to capture from the computed style. This could then automatically extend the computedStyle object of the element. Since the computedStyle object would be a core property it would not be available for extendElementData() to override. Image Prioritizer should do the computed style capturing should be done during initialize, since we want to know whether an image is visible up front.

[westonruter]

Also, we could consider avoiding the need to introduce any JS to capture the desired styles for an element. For example, the context object passed to the tag visitors has a track_tag() method on it. This could take an arg which is the array of style properties which should be captured automatically by Optimization Detective when it is measuring elements on the page. In this way, the new core property for the element schema should be initialComputedStyles. The tag visitor could then access that new property on the OD_Element object to do the correct optimizations. The list of style properties which should be captured can be added to a data-od-initial-styles meta attribute which can then be read from the DOM.

[westonruter]

Helpful article I had open in a tab about efficiency reading computer styles: https://webperf.tips/tip/layout-thrashing/

Since we're not doing any writes to the DOM during detection, we shouldn't be at fault for any layout thrashing. But using rAF seems like a good idea anyway.

[b1ink0]

@westonruter, I have done some tinkering and come up with a few approaches and their POCs that could be used for capturing styles. Note: all the POCs are currently made for the image prioritizer.

1. Using the getComputedStyle function only on elements with the attribute data-od-xpath.

   POC: b1ink0@d6e2fbf

   This would work well if all the styles affecting the element are on the element whose styles we are calculating. But when the element’s ancestors affect the descendants, the getComputedStyle function returns wrong values. For example, when an ancestor has display: none and a descendant has display: block, using getComputedStyle on that descendant will return block for the display property. Similarly, if an ancestor has opacity: 0 and the descendant hasn’t set opacity, it will return 1. getComputedStyle gives correct values for visibility only if ancestors don’t have display: none or content-visibility: hidden. And because a CSS property like content-visibility is set on the ancestor to affect the descendant, this approach will always return the wrong value for content visibility. Also, if an element’s ancestor has display: none or content-visibility: hidden, or the element is inside a <details> element, getComputedStyle can return incorrect values.

   So the above POC only works if all the styles are on the element with the attribute data-od-xpath and ancestors don’t affect the element, but in the real world this ideal scenario will not be possible. So, to get actual property values, we will have to traverse the ancestors. I did create a other POC for ancestor traversing, which is below.

2. Using the getComputedStyle function with ancestor traversal.

   POC: b1ink0@7c2c25f

   In this approach, I tried to create a generalized function to get the actual visible value for the element. I first tried to just traverse up the ancestors to get the topmost element’s value, but this will most likely return the wrong value. To make it work, I had to define the value at which we stop traversing. This solved the issue of display: none, since if an ancestor has display: none, the DOM below will not be rendered. This also solved the issue where content-visibility is set on ancestors, which correctly got the value.

   However, there was another issue with content-visibility: if we stop at the value auto and the ancestor’s ancestor has content-visibility: hidden, the logic will return the wrong value. So I added a priority array to determine the final stop value.

   Values like visibility are inherited and can be overridden by descendants, so one call to getComputedStyle on the element will probably give the correct value. Still, this doesn’t handle an element inside a <details> tag, and if we hardcode this check, it will no longer be a generalized API.

   For handling only four different CSS properties, there were multiple edge cases, and if we want a truly generalized API, we will have to find every possible edge case. We will also need to handle things like getting the accurate visual value of opacity or transform, accounting for the multiplicative nature of opacity and transforms.

   I think creating a JS API will be complex, and in my opinion the better way to handle this is to shift the responsibility of getting the correct value to the extensions. For example, if Image Prioritizer needs to handle these specific CSS properties, then it should do so in its detect.js, and Optimization Detective will only provide a function to include the computedStyles in URL metrics. This has the advantage of using a specific built-in functionality that can return correct values, for Image Prioritizer, we can use checkVisibility. The POC below uses this approach.

3. Extensions handle their own style capture using custom logic.

   POC: b1ink0@c926609

   This approach adds a method to capture the actual visual styles for Image Prioritizer. For Image Prioritizer, we want to know the actual visual state of the IMAGE element, here we can use checkVisibility, which does exactly what we want. This function correctly provides the actual visual state of the element with different parameters, allowing us to know if the element is hidden due to opacity: 0 or visibility: hidden. It also lets us know if the element is affected by content-visibility: auto. This approach is a lot simpler than the above methods.

   We can also pass all the options to checkVisibility, which will then return a single value indicating whether the element is visible after considering all cases. This is useful if we don’t need to know the specific style that made the element invisible and only care about whether it’s visible or not.

   const visible = element.checkVisibility({
       opacityProperty: true,
       visibilityProperty: true,
       contentVisibilityAuto: true
   })

   But some of the issues this method could cause are that it can’t tell whether the element was not rendered due to display: none, content-visibility: hidden, or the element being inside a <details> tag. In these cases, checkVisibility will still return the correct result (that the element is not visible), but it won’t specify which of these conditions caused it.

   Another issue is support. Currently, the properties required for checking visibility and opacity are supported in 91.45% of browsers according to Can I use, while content-visibility is supported in 84.83% according to Can I use.