Regression with package restore: RID Graph not respected when selecting native library from nuget package

[erikest]

According to RID Graph, when restoring nuget packages, it should be the case that the when a library for a specific rid is available it should prefer that before walking the fallback graph. Instead, the behavior I'm seeing is that it conflicts with a library in a fallback rid.

I've added support for gRPC with .net core on raspberry pi with a package for that specific rid: linux-arm. My nuget package uses the convention for native libraries runtimes\<rid>\native\<lib>. The grpc core package also uses this convention, but doesn't have the linux-arm rid, just linux. Both packages are required when publishing for arm, but with the native library from mine and the .net assembly from grpc core.

When publishing with -r linux-arm, I see the following:

Encountered conflict between 'CopyLocal:C:\Users\Geocoder\.nuget\packages\grpc.core\2.26.0\runtimes\linux\native\libgrpc_csharp_ext.x86.so' and 'CopyLocal:C:\Users\Geocoder\.nuget\packages\libgrpc_csharp_ext.arm7\1.0.9\runtimes\linux-arm\native\libgrpc_csharp_ext.x86.so'. Choosing 'CopyLocal:C:\Users\Geocoder\.nuget\packages\grpc.core\2.26.0\runtimes\linux\native\libgrpc_csharp_ext.x86.so' arbitrarily as both items are copy-local and have equal file and assembly versions.

Here are steps to reproduce:

create a new project and add the packages

dotnet new console -o grpcTest
dotnet add package Grpc.Core
dotnet add package Grpc.Net.Client
dotnet add package Grpc.Tools
dotnet add package libgrpc_csharp_ext.arm7 --version 1.1.0

publish to arm

dotnet publish -r linux-arm --self-contained

[bitzeal-johan]

Is there a work-around?

[Liriel]

@bitzeal-johan yes there is a workaround. You can compile the lib yourself as @erikest outlined in https://github.com/erikest/libgrpc_csharp_ext and then replace the binary in the output folder

dotnet publish -r linux-arm --self-contained
cp libgrpc_csharp_ext.x86.so ./bin/Debug/netcoreapp3.1/linux-arm/publish/

I used this method a while ago (on net3.1 though)

[richlander]

I just tried this with the .NET 6 SDK. I don't see this error.

This is what I see in terms of the .so file that was mentioned.

root@a1e6b08d8845:/grpcTest# find . | grep libgrpc_csharp_ext.x86.so
./bin/Debug/net6.0/linux-arm/publish/libgrpc_csharp_ext.x86.so
./bin/Debug/net6.0/linux-arm/libgrpc_csharp_ext.x86.so

Perhaps the packages have been updated.

[Liriel]

@richlander also true for .NET 5

$ find . | grep libgrpc_csharp_ext.x86.so
./bin/Debug/net5.0/linux-arm/publish/libgrpc_csharp_ext.x86.so
./bin/Debug/net5.0/linux-arm/libgrpc_csharp_ext.x86.so

the package libgrpc_csharp_ext.arm7 remains unchanged however

[kleisauke]

The issue persists with .NET 8.0 and .NET 9.0 preview, see:

$ dotnet new console -o netvips
$ cd netvips/
$ dotnet add package NetVips.Native.linux-arm64 --version 8.15.2
$ dotnet add package NetVips.Native.linux-musl-arm64 --version 8.15.2
$ dotnet publish -r linux-musl-arm64 -o .
$ echo "3aa9e0d2865e8a65a0d4d7fa4011e1ab2b6e5c313b53d7584363da817db2b45f libvips.so.42" | sha256sum --check --quiet && echo "arm64-musl libvips.so.42 detected"
libvips.so.42: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match
$ echo "d9a06f7adce8a07213c68ba0be7046a255d2952893174fed5b1187589e770aa2 libvips.so.42" | sha256sum --check --quiet && echo "arm64-glibc libvips.so.42 detected"
arm64-glibc libvips.so.42 detected
$ readelf -d libvips.so.42 | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libresolv.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

Introducing the linux-glibc-arm64 RID could possibly fix this, as it ensures that this fallback never happens, see: dotnet/runtime#7318

This was originally reported at kleisauke/net-vips#233.

[kleisauke]

I further debugged this issue using the following command:

$ dotnet publish -r linux-musl-arm64 -o . -v diag | grep "arbitrarily"

Which outputs:

Encountered conflict between 'CopyLocal:/home/kleisauke/.nuget/packages/netvips.native.linux-arm64/8.15.2/runtimes/linux-arm64/native/libvips.so.42' and 'CopyLocal:/home/kleisauke/.nuget/packages/netvips.native.linux-musl-arm64/8.15.2/runtimes/linux-musl-arm64/native/libvips.so.42'. Choosing 'CopyLocal:/home/kleisauke/.nuget/packages/netvips.native.linux-arm64/8.15.2/runtimes/linux-arm64/native/libvips.so.42' arbitrarily as both items are copy-local and have equal file and assembly versions.

This message is printed as part of the conflict resolution logic in the .NET SDK, specifically:

sdk/src/Tasks/Common/ConflictResolution/ConflictResolver.cs

Lines 329 to 348 in 941ec62

	if (item1.ItemType == ConflictItemType.CopyLocal && item2.ItemType == ConflictItemType.CopyLocal)
	{
	// If two items are copy local, we must pick one even if versions are identical, as only
	// one of them can be copied locally. The policy here must be deterministic, but it can
	// be chosen arbitrarily. The assumption is that the assemblies are fully semantically
	// equivalent.
	//
	// We choose ordinal string comparison of package id as a final tie-breaker for this case.
	// We will get here in the real case of frameworks with overlapping assemblies (including
	// version) and self-contained apps. The assembly we choose here is not guaranteed to match
	// the assembly that would be chosen by the host for a framework-dependent app. The host
	// is free to make its own deterministic but arbitrary choice.
	int cmp = string.CompareOrdinal(item1.PackageId, item2.PackageId);
	if (cmp != 0)
	{
	var arbitraryWinner = cmp < 0 ? item1 : item2;
	LogMessage(conflictMessage, Strings.ChoosingCopyLocalArbitrarily_Info, arbitraryWinner.DisplayName);
	return arbitraryWinner;
	}
	}

The resolver always prefers the NetVips.Native.linux-arm64 package over NetVips.Native.linux-musl-arm64 because, in an ordinal string comparison, linux-arm64 precedes linux-musl-arm64.

The assumption that the assemblies are fully semantically equivalent is questionable, as it does not hold true in this case.

[am11]

As it stands, we can condition the glibc package like this: Condition="$(RuntimeIdentifier.StartsWith('linux-')) and !$(RuntimeIdentifier.StartsWith('linux-musl-'))" and musl one: Condition="$(RuntimeIdentifier.StartsWith('linux-musl-'))"which gives toolchain proper gestures to pick the right dependency. We have similar conditions in NativeAOT integration.

[richlander]

@baronfel -- is this a change we can make?