Assembly loading of WebApplicationFactory behaves differently than the hosted application

[bjuraga]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

The WebApplicationFactory loads DLLs differently than the hosted app does.

It seems like the WebApplicationFactory loads the DLLs (NuGet packages) by dll name only, while the host loads them by name PLUS ASSEMBLY VERSION.

This causes for potential runtime errors to slip through and reach production undetected. It happened to me by simply not passing /p:ProductVersion=2.0.0 to the dotnet publish of the nuget packages in a central pipeline.

Expected Behavior

The WebApplicationFactory loads DLLs in the same way the hosted app does and not just by file name.

Steps To Reproduce

Prerequisites

1. Have an arbitrary nuget project that you can pack and publish to a local filesystem for ease of testing. Set it's version to something other than the default 1.0.0.
2. Have an WEB API project that references that nuget package. No need to use functionality from the package.
3. Create a WebApplicationFactory for the WEB API.
4. Create a test that calls an API on the WEB API you created.
5. Observe the test passing, or make adjustments until it does.

Prepare to reproduce the issue

1. Change the NuGet publishing project such that the version is something like 2.0.0, but make sure it does not get written to the assembly (.dll)
2. Clean/Rebuild the WEB API.
3. In the bin folder locate the nuget package dll, go to properties, Details and see that the version is the default 1.0.0 and not the 2.0.0 as expected.
4. Run the tests - they should still pass <- This passes and exposes the BUG

Reproduce the issue

1. Host the API (or do dotnet webapi.dll) to start it.
2. Observe that the api throws Could not load file or assembly with version 2.0.0

Exceptions (if any)

Could not load file or assembly

.NET Version

8.0.403

Anything else?

 dotnet --info
.NET SDK:
 Version:           8.0.403
 Commit:            c64aa40a71
 Workload version:  8.0.400-manifests.e99c892e
 MSBuild version:   17.11.9+a69bbaaf5

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19045
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\8.0.403\

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
 [aspire]
   Installation Source: VS 17.11.35327.3
   Manifest Version:    8.1.0/8.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.1.0\WorkloadManifest.json
   Install Type:        FileBased


Host:
  Version:      8.0.10
  Architecture: x64
  Commit:       81cabf2857

.NET SDKs installed:
  6.0.427 [C:\Program Files\dotnet\sdk]
  8.0.403 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.35 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.35 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.35 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download