Skip to content

Two flaky tests that need investigated #38268

Open
@MiYanni

Description

@MiYanni

Summary

While being domestic cat, I noticed a couple of tests that are inconsistent/flaky. These tests will cause build failures, but a rerun of the PR CI pipeline will often result in success. These test failures can be seen when searching the Helix test log for [FAIL]. The Helix test log link can be accessed when checking the build failure by following the Failure log: link:

image

Tests

  • In the Windows_NT Build_Release job, the test GivenDotnetRunIsInterrupted.ItTerminatesTheChildWhenKilled shows this failure:
    Microsoft.DotNet.Cli.Run.Tests.GivenDotnetRunIsInterrupted.ItTerminatesTheChildWhenKilled [FAIL]
      Expected boolean to be true, but found False.
      Stack Trace:
           at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
           at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
           at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
           at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
           at FluentAssertions.Primitives.BooleanAssertions`1.BeTrue(String because, Object[] becauseArgs)
        /_/test/dotnet-run.Tests/GivenDotnetRunIsInterrupted.cs(152,0): at Microsoft.DotNet.Cli.Run.Tests.GivenDotnetRunIsInterrupted.ItTerminatesTheChildWhenKilled()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
      Output:
        > C:\h\w\AD9409B3\p\d\dotnet.exe run
        
        Exit Code: -1
  • In the Ubuntu_22_04 Build_Release job, the test GivenThatWeHaveAPackageReferenceWithAliases.CanBuildProjectWithMultiplePackageReferencesWithAliases shows this failure:
�[31;1m    Microsoft.NET.Build.Tests.GivenThatWeHaveAPackageReferenceWithAliases.CanBuildProjectWithMultiplePackageReferencesWithAliases [FAIL]
�[m�[37m      Expected command to pass but it did not.
�[m�[37m      File Name: /datadisks/disk1/work/A9B509B9/p/d/dotnet
�[m�[37m      Arguments: msbuild /t:Build /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj /restore
�[m�[37m      Exit Code: 1
�[m�[37m      StdOut:
�[m�[37m      MSBuild version 17.10.0-preview-24073-02+d51ae5297 for .NET
�[m�[37m        Determining projects to restore...
�[m�[37m        Restored /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj (in 1.83 sec).
�[m�[37m      /datadisks/disk1/work/A9B509B9/p/d/sdk/9.0.100-ci/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(313,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj]
�[m�[37m      /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.cs(9,13): error CS0118: 'First.A' is a namespace but is used like a type [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj]
�[m�[37m      StdErr:
�[m�[37m      
�[m�[37m      
�[m�[30;1m      Stack Trace:
�[m�[37m           at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
�[m�[37m           at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
�[m�[37m           at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
�[m�[37m           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
�[m�[37m           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
�[m�[37m           at FluentAssertions.Execution.AssertionScope.FailWith(String message)
�[m�[37m        /_/test/Microsoft.NET.TestFramework/Assertions/CommandResultAssertions.cs(29,0): at Microsoft.NET.TestFramework.Assertions.CommandResultAssertions.Pass()
�[m�[37m        /_/test/Microsoft.NET.Build.Tests/GivenThatWeHaveAPackageReferenceWithAliases.cs(91,0): at Microsoft.NET.Build.Tests.GivenThatWeHaveAPackageReferenceWithAliases.CanBuildProjectWithMultiplePackageReferencesWithAliases()
�[m�[37m           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
�[m�[37m           at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
�[m�[30;1m      Output:
�[m�[37m        > /datadisks/disk1/work/A9B509B9/p/d/dotnet msbuild /t:Pack /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/A.csproj /restore
�[m�[37m        MSBuild version 17.10.0-preview-24073-02+d51ae5297 for .NET
�[m�[37m          Determining projects to restore...
�[m�[37m          Restored /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/A.csproj (in 246 ms).
�[m�[37m        /datadisks/disk1/work/A9B509B9/p/d/sdk/9.0.100-ci/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(313,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/A.csproj]
�[m�[37m          A -> /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/bin/Debug/net9.0/A.dll
�[m�[37m          The package A.1.0.0 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important.
�[m�[37m          Successfully created package '/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/bin/Debug/A.1.0.0.nupkg'.
�[m�[37m        > /datadisks/disk1/work/A9B509B9/p/d/dotnet msbuild /t:Pack /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/B.csproj /restore
�[m�[37m        MSBuild version 17.10.0-preview-24073-02+d51ae5297 for .NET
�[m�[37m          Determining projects to restore...
�[m�[37m          Restored /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/B.csproj (in 232 ms).
�[m�[37m        /datadisks/disk1/work/A9B509B9/p/d/sdk/9.0.100-ci/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(313,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/B.csproj]
�[m�[37m          B -> /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/bin/Debug/net9.0/B.dll
�[m�[37m          The package B.1.0.0 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important.
�[m�[37m          Successfully created package '/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/bin/Debug/B.1.0.0.nupkg'.
�[m�[37m        > /datadisks/disk1/work/A9B509B9/p/d/dotnet msbuild /t:Build /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj /restore
�[m�[37m        MSBuild version 17.10.0-preview-24073-02+d51ae5297 for .NET
�[m�[37m          Determining projects to restore...
�[m�[37m          Restored /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj (in 1.83 sec).
�[m�[37m        /datadisks/disk1/work/A9B509B9/p/d/sdk/9.0.100-ci/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(313,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj]
�[m�[37m        /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.cs(9,13): error CS0118: 'First.A' is a namespace but is used like a type [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj]
�[m�[37m        Exit Code: 1

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions