Quarantine EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation

[captainsafia]

Failing Test(s)

• Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation(enableStreaming: True, useEnhancedNavigation: True, programmaticNavigation: False)

Error Message

System.Exception : Failed to execute script after 3 retries.

Stacktrace

at Microsoft.AspNetCore.Components.E2ETest.WebDriverExtensions.GetElementPositionWithRetry(IWebDriver browser, String elementId, Int32 retryCount, Int32 delayBetweenRetriesMs) in /home/vsts/work/1/s/src/Components/test/E2ETest/Infrastructure/WebDriverExtensions/WebDriverExtensions.cs:line 69
at Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation(Boolean enableStreaming, Boolean useEnhancedNavigation, Boolean programmaticNavigation) in /home/vsts/work/1/s/src/Components/test/E2ETest/ServerRenderingTests/EnhancedNavigationTest.cs:line 694
at InvokeStub_EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation(Object, Span`1)
at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

Logs

Build

• https://dev.azure.com/dnceng-public/public/_build/results?buildId=975768&view=results

[javiercn]

@ilonatommy can you take a look at this test? Seems that this has started failing possibly due to some of the changes we did here.

[ilonatommy]

How can we check the frequency of failing? Is there a search engine to browse through logs? I want e.g. to check if the test fails only with the mentioned configuration of parameters.

edit - results of manual azdo search releals that only 2 combinations have problem with flakiness, both with enhanced navigation being off:

• true, false, false (with streaming) https://dev.azure.com/dnceng-public/public/_build/results?buildId=977834&view=logs&j=fe94c0c9-bb8c-5d6f-3b51-887173cc2f5c&t=b4bd3469-f776-5941-b623-568498d38bde
• false, false, false (no streaming) https://dev.azure.com/dnceng-public/public/_build/results?buildId=979233&view=logs&j=fe94c0c9-bb8c-5d6f-3b51-887173cc2f5c&t=b4bd3469-f776-5941-b623-568498d38bde

I will change the quarantine to block only these test cases. During the search I found a few different issues that I will address in the PR.

[ilonatommy]

There are 2 types of failures.

1. line var button1Pos = Browser.GetElementPositionWithRetry(button1Id); - less frequent

Failed Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation(enableStreaming: True, useEnhancedNavigation: False, programmaticNavigation: False) [12 s]
  Error Message:
   System.Exception : Failed to execute script after 3 retries.
  Stack Trace:
     at Microsoft.AspNetCore.Components.E2ETest.WebDriverExtensions.GetElementPositionWithRetry(IWebDriver browser, String elementId, Int32 retryCount, Int32 delayBetweenRetriesMs) in /home/vsts/work/1/s/src/Components/test/E2ETest/Infrastructure/WebDriverExtensions/WebDriverExtensions.cs:line 69
   at Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation(Boolean enableStreaming, Boolean useEnhancedNavigation, Boolean programmaticNavigation) in /home/vsts/work/1/s/src/Components/test/E2ETest/ServerRenderingTests/EnhancedNavigationTest.cs:line 694
   at InvokeStub_EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation(Object, Span`1)
   at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

2. lines with AssertEnhancedNavigation calls

 Failed Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnBackwardsForwardsAction(enableStreaming: False, useEnhancedNavigation: False, programmaticNavigation: False) [5 s]
  Error Message:
   System.Exception : Expected to use browser navigation because 'suppress-enhanced-navigation' is set to true but the element from previous path is not stale
  Stack Trace:
     at Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.AssertEnhancedNavigation(Boolean useEnhancedNavigation, IWebElement elementForStalenessCheck, Int32 retryCount, Int32 delayBetweenRetriesMs) in /home/vsts/work/1/s/src/Components/test/E2ETest/ServerRenderingTests/EnhancedNavigationTest.cs:line 843
   at Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnBackwardsForwardsAction(Boolean enableStreaming, Boolean useEnhancedNavigation, Boolean programmaticNavigation) in /home/vsts/work/1/s/src/Components/test/E2ETest/ServerRenderingTests/EnhancedNavigationTest.cs:line 759
   at InvokeStub_EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnBackwardsForwardsAction(Object, Span`1)
   at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
[xUnit.net 00:19:39.62] Microsoft.AspNetCore.Components.E2ETests: [Long Running Test] 'Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnBackwardsForwardsAction(enableStreaming: True, useEnhancedNavigation: True, programmaticNavigation: True)', Elapsed: 00:00:31


 Failed Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation(enableStreaming: True, useEnhancedNavigation: False, programmaticNavigation: False) [17 s]
  Error Message:
   System.Exception : Expected to use browser navigation because 'suppress-enhanced-navigation' is set to true but the element from previous path is not stale
  Stack Trace:
     at Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.AssertEnhancedNavigation(Boolean useEnhancedNavigation, IWebElement elementForStalenessCheck, Int32 retryCount, Int32 delayBetweenRetriesMs) in /home/vsts/work/1/s/src/Components/test/E2ETest/ServerRenderingTests/EnhancedNavigationTest.cs:line 841
   at Microsoft.AspNetCore.Components.E2ETests.ServerRenderingTests.EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation(Boolean enableStreaming, Boolean useEnhancedNavigation, Boolean programmaticNavigation) in /home/vsts/work/1/s/src/Components/test/E2ETest/ServerRenderingTests/EnhancedNavigationTest.cs:line 703
   at InvokeStub_EnhancedNavigationTest.EnhancedNavigationScrollBehavesSameAsBrowserOnNavigation(Object, Span`1)
   at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)