NuGet pack raises a warning, which is turned into an error, but the build still passes

[eerhardt]

I'm building NuGet packages in my repo, and newer SDKs are now raising a warning (which my project says warnings as errors) but that isn't causing my build to fail.

C:\Program Files\dotnet\sdk\3.0.100\Sdks\NuGet.Build.Tasks.Pack\buildCrossTargeting\NuGet.Build.Tasks.Pack.targets(198,5): error NU5048: The 'PackageIconUrl'/'iconUrl' element is deprecated. Consider using the 'PackageIcon'/'icon' element instead. Learn more at https://aka.ms/deprecateIconUrl [F:\git\machinelearning\pkg\Microsoft.ML.CpuMath\Microsoft.ML.CpuMath.nupkgproj]

Steps to reproduce

1. Unzip MSBuildBug.zip
2. dotnet build
   notice the red errors, but the output saying Build succeeded.
3. echo $LASTEXITCODE

returns: 0

Expected behavior

I expect the build should fail if there are errors during the build.

Actual behavior

The build prints errors to the console (even in red), but the build succeeds and the $LASTEXITCODE returns 0. So this won't fail any Azure DevOps CI builds.

Environment data

> dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.0.100
 Commit:    04339c3a26

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.0.100\

Host (useful for support):
  Version: 3.0.0
  Commit:  7d57652f33

[JayBazuzi]

+1 we also see this and want want to have the warn-as-error protection in the build system.

See lines 18, 20, 22, and 69 here:

https://ci.appveyor.com/project/JayBazuzi/pipelines/builds/30988237#L18

[benvillalobos]

Can confirm that this happens on 3.0.103, does not happen on 3.1.100, and does not happen on 5.0.

Another observation is that it fails only the first time. If I hit save in vs code (with no changes) and run dotnet build again, it will fail.

[benvillalobos]

A bit more investigating. It looks like the nuget pack task understands both the warningsaserrors flag and treatwarningsaserrors property.

using msbuild

=== Without TreatWarningsAsErrors in the project ===

1. run msbuild, it warns. echo %ERRORLEVEL% = 0
2. run msbuild /warnaserror, It errors. echo %ERRORLEVEL% = 1
   === With TreatWarningsAsErrors set to true in the project ===
3. run msbuild, it errors, build succeeds, echo %ERRORLEVEL% = 0
4. run msbuild /warnaserror, it errors, build fails, echo %ERRORLEVEL% = 1

using dotnet build

=== Without TreatWarningsAsErrors in the project ===

1. dotnet build: warnings, build succeeds, errorlevel=0
2. dotnet build /warnaserror: errors, build fails, errorlevel=1
   === With TreatWarningsAsErrors set to true in the project ===
3. dotnet build: errors, build succeeds, %ERRORLEVEL% = 0
4. dotnet build /warnaserror: errors, build fails, %ERRORLEVEL% = 1 <-- this alternates between builds?

Looks like this boils down to TreatWarningsAsErrors being treated differently than passing /warnaserror on the cmd line.

[benvillalobos]

PR's that merged these features (for quick reference):
#1928
#1355

[benvillalobos]

Turns out there's a distinction between TreatWarningsAsErrors and MSBuildTreatWarningsAsErrors. The former has to do with:
WinMDExp
Csc
and Vbc from what I could find.

The latter has to do with warnings logged through the build system. So setting MSBuildWarningsAsErrors to true will have msbuild fail properly. Let me know if this works for you!

[benvillalobos]

Closing as resolved.