[Linux] [C#] [0.5.2] Publish/Build Binaries with Microsoft.ML.OnnxRuntimeGenAI.Cuda on Linux containing unnecessary large DLL files

[jiaxuwu2021]

Describe the bug
When I build or publish linux-x64 binary with Microsoft.ML.OnnxRuntimeGenAI.Cuda 0.5.2, the final output folder will contain many unnecessary DLLs (onnxruntime*.dll), especially onnxruntime_providers_cuda.dll which is very large (>600MB)

Could you please fix to stop publishing these onnxruntime*.dll files for linux-x64 when dotnet build/dotnet publish, otherwise the final binaries size will be double (both cuda.so and cuda.sll >600MB)

To Reproduce
Steps to reproduce the behavior:

• test.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <PublishAot>true</PublishAot>
    <IsPackable>false</IsPackable>
    <SelfContained>true</SelfContained>
  </PropertyGroup>

  <!-- OnnxRuntime for Linux -->
  <ItemGroup>
    <PackageReference Include="Microsoft.ML.OnnxRuntimeGenAI.Managed" Version="0.5.2" />
    <PackageReference Include="Microsoft.ML.OnnxRuntimeGenAI.Cuda" Version="0.5.2" />
  </ItemGroup>
</Project>

• Program.cs

namespace MyTest;

using Microsoft.ML.OnnxRuntime;

public class Program
{
    public static async Task Main(string[] args)
    {
        const string modelPath = "/home/jiaxuwu/.aitk/models/onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32.onnx";
        using var session = new InferenceSession(modelPath);
    }
}

• dotnet publish test.csproj --runtime linux-x64
• ll -h bin/Release/net8/linux-x64/publish/onnxruntime*.dll
• ll -h bin/Release/net8/linux-x64/onnxruntime*.dll

Expected behavior
Microsoft.ML.OnnxRuntimeGenAI.Cuda shouldn't publish any unnecessary very large files on Linux

Screenshots

Desktop (please complete the following information):

• OS: WLS(Windows Subsystem for Linux)

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.5 LTS
Release:        22.04
Codename:       jammy

• .NET

$ dotnet --list-sdks
8.0.110 [/usr/lib/dotnet/sdk]

• Nuget Package Version: Microsoft.ML.OnnxRuntimeGenAI.Cuda 0.5.2

Additional context
Another package Microsoft.ML.OnnxRuntimeGenAI 0.5.2 has the similar issue on MacOS
MacOS final output binaries will also contain several unnecessary DLL files with Microsoft.ML.OnnxRuntimeGenAI 0.5.2

[jiaxuwu2021]

A temporary workaround to our team before this bug fixed

<PropertyGroup>
  <IsWindows Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))' == 'true'">true</IsWindows>
</PropertyGroup>

<!-- Remove once ONNX team fix the bug -->
<Target Name="CustomAfterBuild" AfterTargets="AfterBuild" Condition="('$(IsWindows)'!='true')">
  <Message Importance="High" Text="Delete unnecessary onnx dlls" />
  <ItemGroup>
    <FilesToDeleteInBuild Include="$(TargetDir)onnxruntime*.dll"/>
  </ItemGroup>
  <Delete Files="@(FilesToDeleteInBuild)" />
</Target>

<!-- Remove once ONNX team fix the bug -->
<Target Name="CustomAfterPublish" AfterTargets="AfterPublish" Condition="('$(IsWindows)'!='true')">
  <Message Importance="High" Text="Delete unnecessary onnx dlls" />
  <ItemGroup>
    <FilesToDeleteInPublish Include="$(PublishDir)onnxruntime*.dll"/>
  </ItemGroup>
  <Delete Files="@(FilesToDeleteInPublish)" />
</Target>

[skyline75489]

Hi! Thanks for reporting this. I don't really think we control how ORT DLLs are published, though. We basically refer to ORT NuGet as a dependency and use their package just like everyone else. I'll take a look tomorrow.

[jiaxuwu2021]

Make some more investigations, found something:

• Microsoft.ML.OnnxRuntimeGenAI.Cuda depending on Microsoft.ML.OnnxRuntime.Gpu
• Microsoft.ML.OnnxRuntime.Gpu depending on Microsoft.ML.OnnxRuntime.Gpu.Windows

It seems that this file Microsoft.ML.OnnxRuntime.Gpu.Windows.props is trying to copy DLL files for all x64 platforms including linux-x64.
So, it's a bug, right?

[skyline75489]

@jiaxuwu2021 I think it's a bug, but I'm not certain. It is beyond the scope of this repo, though. Will need to contact the packaging person of ORT itself.

[jiaxuwu2021]

@skyline75489 Thanks for your help! let me create a linked issue into ORT
microsoft/onnxruntime#23077

[Webreaper]

A temporary workaround to our team before this bug fixed

Thanks for this. Unfortunately it doesn't work for single-file publishing, because the single file is created before the custom build action executes.

I'll just need to wait for MSFT to fix this issue.