.NET 9 dotnet format Azure DevOps hang

[nietras]

We use GitVersioning for a dotnet project and have without issue for some time. But when upgrading to .NET 9 we suddenly saw dotnet format --verify-no-changes hanging forever when running in CI. Not when run manually or locally. After lots of bisecting and removing stuff, we have isolated this to

  <ItemGroup>
    <PackageReference Include="Nerdbank.GitVersioning" Version="3.6.146"
                      PrivateAssets="all" Condition="!Exists('packages.config')" />
  </ItemGroup>

which is included in Directory.Build.props. If this is removes the hang disappears. Only dotnet format hangs. No logging output gives any details as to why, but it appears to occur just before dotnet format shows effective editorconfig files for each project. CI output is:

D:\agent\_work\_tool\dotnet\dotnet.exe format --verify-no-changes -v diagnostic
  The dotnet runtime version is '9.0.0'.
  The dotnet CLI version is '9.0.100'.
  Using MSBuild.exe located in 'D:\agent\_work\_tool/dotnet\sdk\9.0.100\'.
  Formatting code files in workspace 'D:\agent\_work\210\s\Test.sln'.
  Loading workspace.
    Determining projects to restore...
  Restored D:\agent\_work\210\s\src\XYZ\XYZ.csproj (in 198 ms)

at which point the task hangs for 60 minutes until Azure DevOps cancels the job.

Presumably, GitVersioning must be doing something during dotnet format that causes the hang and for some reason then only for new .NET 9 SDK perhaps.

[AArnott]

I have no idea why that would be.
I use .NET 9 SDK in combination with the exact technique you're using with NB.GV in several repos without a problem.

It's conceivable that the msbuild targets that NB.GV adds to your projects are executed during nuget restore, which dotnet format runs. But as you're well aware, that doesn't generally cause a hang. In fact I've never seen nb.gv hang.

I see you've already turned on -v diagnostic. Is the diagnostic log available and does it show more than you've shared here? I would expect the diagnostic log to include a ton of stuff including which msbuild targets have or are running. That would be very valuable.

You may want to file a bug against dotnet format to see what they can figure out too.

[nietras]

You are seeing the full diagnostic logs, I cannot get more, only 1 csproj, very simple. Usually after this it would log editorconfig paths. Logging seems very sparse for dotnet format, are they even calling msbuild? don't know...

Are you using it with net9.0 in CI on Azure DevOps?

[AArnott]

I guess the diagnostic log switch is not passed on to msbuild, which they do invoke (or NB.GV would never have any opportunity to have an impact.)

Yes, I'm using .NET 9 SDK on CI in Azure DevOps and calling dotnet format, as you can see here.

In my case, I'm executing that step on linux. The Windows agent skips it (just to avoid redundancy).

[AArnott]

Well, now I have the hang -- on linux no less. I'm filing a bug against the .NET SDK.

[AArnott]

@nietras consider applying this workaround to your repo:
dotnet/Nerdbank.Streams@20e285c

[nietras]

@AArnott thank you for both reporting the issue and finding a simple workaround which appears to work, naturally given it simply removed the package during dotnet format. I also thought this might be related to new terminal logger or something and perhaps related to how dotnet format "invokes" msbuild then. But given only occurs in context of CI on build servers a bit harder to debug. Hopefully, MS will find the issue. I'll leave this open for now, but you may close if you are confident this is a sdk issue (most likely).