Time to Interactive calculation when Visibility is Hidden

[nicjansma]

We should ensure the Time to Visually Ready (TTVR) and Time to Interactive (TTI) metrics are correct (or skipped) when the page is loaded hidden (or goes through hidden-visible transitions).

If the page loads hidden, the First Paint and First Contentful Paint events may not fire until the page is foregrounded. In addition, setTimeout, requestAnimationFrame and other things we use to monitor TTVR/TTI may not fire reliably. In many cases, it's not until the page comes back to visible that these paints/callbacks happen. This can skew our calculations and make TTVR/TTI reflect the user-interaction (of foregrounding) rather than the page's lifecycle.

Related:

• c.tti metric calculation #245 for a probable occurrence
• surprising (if not understandable) results for first-paint when document.visibilityState === 'hidden' w3c/paint-timing#40

[ceckoslab]

Hello @nicjansma

I would like to have 2 parts (A and B) in this comment:

A. Reproducing issue when we have very high TTI.

I managed to reproduce the issue that Jud White has reported here: #245

Steps for manual test:

1. We need to have this Boomerang test configuration:

BOOMR_test.init({
	testAfterOnBeacon: true,
	Continuity: {
		enabled: true,
		afterOnload: true,
		 waitAfterOnload: 1000
	}
});

2. We create a test with this configuration. Then right click on the test link and open the link in new tab. The idea is that the test loads in hidden tab.

3. We wait the test to load in hidden tab.

4. We wait some time e.g. 10 seconds.

5. If we use reverse proxy (in my case Charles Proxy) we will see very high c.tti value in Quit beacon.

Next step will be to try to recreate this in e2e test find out the real reason why this happens.

B. Metrics that may not be correct when page is loaded in hidden tab.

We could execute one of the points below but I am not sure how often this happens and before I really implement something I would like to see some real life:

If the page loaded hidden we do not send:

• TTVR
• TTI
• Time To Hero Images Ready
• First Paint
• First Contentful Paint

[ceckoslab]

Hello @nicjansma

I looked deeper at this problem and it became obvious that we do wrong calculations because of the way setTimeout() works in inactive browser tabs.

In Continuity plugin we are wrapping the call of checkTti() in setTimeout(). Conclusion from my experiments and what I read regarding setTimeout() behavior in inactive tabs is that it does not execute while the tab is inactive/hidden.

On the other hand TTVR had fine value.

I see 2 possible options:

1. Do not send TTI If tab tab was inactive.
2. Send TTVR value for TTI If tab tab was inactive.

Good place where we can implement this is: impl.timeline.analyze()

Reference to checkTti():

if (impl.waitAfterOnload) {
	var start = BOOMR.now();

	setTimeout(function checkTti() {
		// wait for up to the defined time after onload
		if (BOOMR.now() - start > impl.waitAfterOnload) {
			// couldn't calculate TTI, send the beacon anyways
			impl.complete = true;
			BOOMR.sendBeacon();
		}
		else {
			// run the TTI calculation
			impl.timeline.analyze();

			// if we got something, mark as complete and send
			if (externalMetrics.timeToInteractive()) {
				impl.complete = true;
				BOOMR.sendBeacon();
			}
			else {
				// poll again
				setTimeout(checkTti, TIME_TO_INTERACTIVE_WAIT_POLL_PERIOD);
			}
		}
	}, TIME_TO_INTERACTIVE_WAIT_POLL_PERIOD);
}
else {
	impl.complete = true;
}