Skip to content

Commit 1a71a5f

Browse files
tianleiwutianleiwu
authored
Fix NuGet DLL Loading on Linux and macOS (#27266)
## Summary This PR addresses persistent native library loading issues in the ONNX Runtime NuGet package, specifically on macOS and Linux, by implementing a robust DllImportResolver. It also includes necessary pipeline and packaging adjustments to ensure required macOS artifacts are correctly located and validated during CI. ## Problem #27263 reports that `Unable to load shared library 'onnxruntime.dll' or one of its dependencies`. It was caused by #26415 since the commit hard-coded onnxruntime.dll even for Linux and MacOS (The correct filename shall be libonnxruntime.so for Linux, and libonnxruntime.dylib for MacOS). The Nuget test pipeline has been broken for a while, so we also need fix the pipeline to test our change. It has the following issues: * MacOS nuget is for arm64, but the vmImage `macOS-15` is x64. * MacOS nuget test need libcustom_op_library.dylib, but it is not copied from artifacts to test environment. * MacOS artifact contains libonnxruntime.dylib and libonnxruntime.1.24.1.dylib, where libonnxruntime.dylib is symlink. It causes issue since the later is excluded by nuspec. * MacOS nuget test use models from onnx repo. However, latest onnx has some models with data types like float8 that are not supported by C#, so those model test failed. * Linux nuget test uses a docker Dockerfile.package_ubuntu_2404_gpu, but docker build failed due to libnvinfer-headers-python-plugin-dev and libnvinfer-win-builder-resource10 version. ## Changes ### 1. Robust C# DLL Resolution The DllImportResolver has been enhanced to handle various deployment scenarios where standard .NET resolution might fail: - **Platform-Specific Naming**: Maps extension-less library names (`onnxruntime`, `ortextensions`) to appropriate filenames (`onnxruntime.dll`, `libonnxruntime.so`, `libonnxruntime.dylib`) based on the OS. - **Multi-Stage Probing**: 1. **Default Loading**: Attempts `NativeLibrary.TryLoad` with the mapped name. 2. **NuGet `runtimes` Probing**: If the above fails, it probes the `runtimes/{rid}/native/` subdirectories relative to the assembly location, covering common RIDs (`win-x64`, `linux-arm64`, `osx-arm64`, etc.). 3. **Base Directory Fallback**: As a final attempt, it looks in `AppContext.BaseDirectory`. - **Case-Sensitivity Handling**: Ensures lowercase extensions are used on Windows to prevent lookup failures on case-sensitive filesystems. ### 2. macOS CI/Packaging Improvements - **Templates (test_macos.yml)**: - Updated to extract artifacts from TGZ files. - Ensures `libcustom_op_library.dylib` is placed in the expected location (`testdata/testdata`) for end-to-end tests. - Initializes the ONNX submodule to provide required test data. - **Node.js**: - Restored the Node.js macOS test stage in c-api-noopenmp-test-pipelines.yml, configured to run on the ARM64 pool (`AcesShared`). - Updated test_macos.yml template to support custom agent pools (similar to the NuGet template). - **Pipeline Config**: Adjusted agent pool selection and demands for macOS jobs to ensure stable execution. - **Binary Robustness**: The `copy_strip_binary.sh` script now ensures `libonnxruntime.dylib` is a real file rather than a symlink, improving NuGet packaging reliability. ### 3. Test Refinements - **Inference Tests**: Skips a specific set of pretrained-model test cases on macOS that are currently known to be flaky or unsupported in that environment, preventing noise in the CI results. ## Verification ### Pipelines - [x] Verified in `NuGet_Test_MacOS`. - [x] Verified in `NuGet_Test_Linux`. - [x] Verified in Windows test pipelines. ### Net Effect The C# bindings are now significantly more resilient to different deployment environments. The CI process for macOS is also more robust, correctly handling the artifacts required for comprehensive NuGet validation.
1 parent d7f32e2 commit 1a71a5f

File tree

9 files changed

+243
-23
lines changed

9 files changed

+243
-23
lines changed

‎csharp/src/Microsoft.ML.OnnxRuntime/NativeMethods.shared.cs‎

Lines changed: 132 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
// Licensed under the MIT License.
33

44
using System;
5+
using System.Reflection;
56
using System.Runtime.InteropServices;
67
using static Microsoft.ML.OnnxRuntime.NativeMethods;
78

@@ -474,6 +475,12 @@ internal static class NativeMethods
474475

475476
static NativeMethods()
476477
{
478+
#if !NETSTANDARD2_0 && !__ANDROID__ && !__IOS__
479+
// Register a custom DllImportResolver to handle platform-specific library loading.
480+
// Replaces default resolution specifically on Windows for case-sensitivity.
481+
NativeLibrary.SetDllImportResolver(typeof(NativeMethods).Assembly, DllImportResolver);
482+
#endif
483+
477484
#if NETSTANDARD2_0
478485
IntPtr ortApiBasePtr = OrtGetApiBase();
479486
OrtApiBase ortApiBase = (OrtApiBase)Marshal.PtrToStructure(ortApiBasePtr, typeof(OrtApiBase));
@@ -847,7 +854,7 @@ static NativeMethods()
847854
api_.CreateSyncStreamForEpDevice,
848855
typeof(DOrtCreateSyncStreamForEpDevice));
849856

850-
OrtSyncStream_GetHandle =
857+
OrtSyncStream_GetHandle =
851858
(DOrtSyncStream_GetHandle)Marshal.GetDelegateForFunctionPointer(
852859
api_.SyncStream_GetHandle,
853860
typeof(DOrtSyncStream_GetHandle));
@@ -872,11 +879,127 @@ internal class NativeLib
872879
// Define the library name required for iOS
873880
internal const string DllName = "__Internal";
874881
#else
875-
// Note: the file name in ONNX Runtime nuget package must be onnxruntime.dll instead of onnxruntime.DLL(Windows filesystem can be case sensitive)
876-
internal const string DllName = "onnxruntime.dll";
882+
// For desktop platforms (including .NET Standard 2.0), we use the simple name
883+
// to allow .NET's automatic platform-specific resolution (lib*.so, lib*.dylib, *.dll).
884+
// For .NET Core 3.0+, case-sensitivity on Windows is handled by DllImportResolver.
885+
internal const string DllName = "onnxruntime";
877886
#endif
878887
}
879888

889+
#if !NETSTANDARD2_0 && !__ANDROID__ && !__IOS__
890+
/// <summary>
891+
/// Custom DllImportResolver to handle platform-specific library loading.
892+
/// On Windows, it explicitly loads the library with a lowercase .dll extension to handle
893+
/// case-sensitive filesystems.
894+
/// </summary>
895+
private static IntPtr DllImportResolver(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
896+
{
897+
if (libraryName == NativeLib.DllName || libraryName == OrtExtensionsNativeMethods.ExtensionsDllName)
898+
{
899+
string mappedName = null;
900+
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
901+
{
902+
// Explicitly load with .dll extension to avoid issues where the OS might try .DLL
903+
mappedName = libraryName + ".dll";
904+
}
905+
else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
906+
{
907+
// Explicitly load with .so extension and lib prefix
908+
mappedName = "lib" + libraryName + ".so";
909+
}
910+
else if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
911+
{
912+
// Explicitly load with .dylib extension and lib prefix
913+
mappedName = "lib" + libraryName + ".dylib";
914+
}
915+
916+
if (mappedName != null)
917+
{
918+
// 1. Try default loading (name only)
919+
if (NativeLibrary.TryLoad(mappedName, assembly, searchPath, out IntPtr handle))
920+
{
921+
return handle;
922+
}
923+
924+
// 2. Try relative to assembly location (look into runtimes subfolders)
925+
string assemblyLocation = null;
926+
try { assemblyLocation = assembly.Location; } catch { }
927+
if (!string.IsNullOrEmpty(assemblyLocation))
928+
{
929+
string assemblyDir = System.IO.Path.GetDirectoryName(assemblyLocation);
930+
string rid = RuntimeInformation.RuntimeIdentifier;
931+
932+
// Probe the specific RID first, then common fallbacks for the current OS
933+
string[] ridsToTry;
934+
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
935+
{
936+
ridsToTry = new[] { rid, "win-x64", "win-arm64" };
937+
}
938+
else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
939+
{
940+
ridsToTry = new[] { rid, "linux-x64", "linux-arm64" };
941+
}
942+
else if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
943+
{
944+
// We no longer provide osx-x64 in official package since 1.24.
945+
// However, we keep it in the list for build-from-source users.
946+
ridsToTry = new[] { rid, "osx-arm64", "osx-x64" };
947+
}
948+
else
949+
{
950+
ridsToTry = new[] { rid };
951+
}
952+
953+
foreach (var tryRid in ridsToTry)
954+
{
955+
string probePath = System.IO.Path.Combine(assemblyDir, "runtimes", tryRid, "native", mappedName);
956+
if (System.IO.File.Exists(probePath) && NativeLibrary.TryLoad(probePath, assembly, searchPath, out handle))
957+
{
958+
LogLibLoad($"[DllImportResolver] Loaded {mappedName} from: {probePath}");
959+
return handle;
960+
}
961+
}
962+
}
963+
964+
// 3. Try AppContext.BaseDirectory as a fallback
965+
string baseDir = AppContext.BaseDirectory;
966+
if (!string.IsNullOrEmpty(baseDir))
967+
{
968+
string probePath = System.IO.Path.Combine(baseDir, mappedName);
969+
if (NativeLibrary.TryLoad(probePath, assembly, searchPath, out handle))
970+
{
971+
LogLibLoad($"[DllImportResolver] Loaded {mappedName} from: {probePath}");
972+
return handle;
973+
}
974+
975+
string rid = RuntimeInformation.RuntimeIdentifier;
976+
probePath = System.IO.Path.Combine(baseDir, "runtimes", rid, "native", mappedName);
977+
if (NativeLibrary.TryLoad(probePath, assembly, searchPath, out handle))
978+
{
979+
LogLibLoad($"[DllImportResolver] Loaded {mappedName} from: {probePath}");
980+
return handle;
981+
}
982+
}
983+
984+
LogLibLoad($"[DllImportResolver] Failed loading {mappedName} (RID: {RuntimeInformation.RuntimeIdentifier}, Assembly: {assemblyLocation})");
985+
986+
}
987+
}
988+
989+
// Fall back to default resolution
990+
return IntPtr.Zero;
991+
}
992+
993+
private static void LogLibLoad(string message)
994+
{
995+
System.Diagnostics.Trace.WriteLine(message);
996+
if (!string.IsNullOrEmpty(Environment.GetEnvironmentVariable("ORT_LOADER_VERBOSITY")))
997+
{
998+
Console.WriteLine(message);
999+
}
1000+
}
1001+
#endif
1002+
8801003
[DllImport(NativeLib.DllName, CharSet = CharSet.Ansi)]
8811004
#if NETSTANDARD2_0
8821005
public static extern IntPtr OrtGetApiBase();
@@ -2644,7 +2767,7 @@ public delegate void DOrtAddKeyValuePair(IntPtr /* OrtKeyValuePairs* */ kvps,
26442767
byte[] /* const char* */ value);
26452768

26462769
/// <summary>
2647-
/// Get the value for the provided key.
2770+
/// Get the value for the provided key.
26482771
/// </summary>
26492772
/// <returns>Value. Returns IntPtr.Zero if key was not found.</returns>
26502773
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
@@ -2767,7 +2890,7 @@ out IntPtr /* OrtSyncStream** */ stream
27672890
// Auto Selection EP registration and selection customization
27682891

27692892
/// <summary>
2770-
/// Register an execution provider library.
2893+
/// Register an execution provider library.
27712894
/// The library must implement CreateEpFactories and ReleaseEpFactory.
27722895
/// </summary>
27732896
/// <param name="env">Environment to add the EP library to.</param>
@@ -2952,9 +3075,10 @@ internal static class OrtExtensionsNativeMethods
29523075
#elif __IOS__
29533076
internal const string ExtensionsDllName = "__Internal";
29543077
#else
2955-
// For desktop platforms, explicitly specify the DLL name with extension to avoid
2956-
// issues on case-sensitive filesystems. See NativeLib.DllName for detailed explanation.
2957-
internal const string ExtensionsDllName = "ortextensions.dll";
3078+
// For desktop platforms, use the simple name to allow .NET's
3079+
// automatic platform-specific resolution (lib*.so, lib*.dylib, *.dll).
3080+
// Case-sensitivity on Windows is handled by DllImportResolver.
3081+
internal const string ExtensionsDllName = "ortextensions";
29583082
#endif
29593083

29603084
[DllImport(ExtensionsDllName, CharSet = CharSet.Ansi,

‎csharp/test/Microsoft.ML.OnnxRuntime.Tests.NetCoreApp/InferenceTest.netcore.cs‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -601,6 +601,29 @@ private static Dictionary<string, string> GetSkippedModels(DirectoryInfo modelsD
601601
skipModels["VGG 16-fp32"] = "bad allocation";
602602
}
603603

604+
// The following models are from onnx repo and fail on MacOS nuget test pipeline.
605+
if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
606+
{
607+
var macOSSkips = new[]
608+
{
609+
"test_castlike_FLOAT_to_STRING_expanded",
610+
"test_castlike_FLOAT_to_BFLOAT16_expanded",
611+
"test_castlike_BFLOAT16_to_FLOAT",
612+
"test_cast_FLOAT_to_STRING",
613+
"test_castlike_FLOAT_to_BFLOAT16",
614+
"test_castlike_STRING_to_FLOAT_expanded",
615+
"test_castlike_STRING_to_FLOAT",
616+
"test_cast_STRING_to_FLOAT",
617+
"test_castlike_BFLOAT16_to_FLOAT_expanded",
618+
"test_cast_BFLOAT16_to_FLOAT",
619+
"test_castlike_FLOAT_to_STRING"
620+
};
621+
foreach (var model in macOSSkips)
622+
{
623+
skipModels[model] = "Skipped on macOS due to flakes or lack of support";
624+
}
625+
}
626+
604627
return skipModels;
605628
}
606629

@@ -934,6 +957,7 @@ public void TestPretrainedModelsWithOrtValue(string opsetDir, string modelName)
934957
[MemberData(nameof(GetSkippedModelForTest), Skip = "Skipped due to Error, please fix the error and enable the test")]
935958
private void TestPreTrainedModels(string opsetDir, string modelName, bool useOrtValueAPIs = false)
936959
{
960+
937961
var opsetDirInfo = new DirectoryInfo(opsetDir);
938962
var opset = opsetDirInfo.Name;
939963
string onnxModelFileName = null;

‎tools/ci_build/github/azure-pipelines/c-api-noopenmp-test-pipelines.yml‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -104,9 +104,18 @@ stages:
104104

105105
- template: nuget/templates/test_macos.yml
106106
parameters:
107-
AgentPool: macOS-14
107+
AgentPool: 'AcesShared'
108+
UseHostedVmImage: 'false'
109+
PoolDemands: 'ImageOverride -equals ACES_VM_SharedPool_Sequoia'
108110
ArtifactSuffix: 'CPU'
109111

112+
- template: nodejs/templates/test_macos.yml
113+
parameters:
114+
AgentPool: 'AcesShared'
115+
UseHostedVmImage: 'false'
116+
PoolDemands: 'ImageOverride -equals ACES_VM_SharedPool_Sequoia'
117+
StageSuffix: 'MacOS_ARM64'
118+
110119
- template: nodejs/templates/test_win.yml
111120
parameters:
112121
AgentPool: 'onnxruntime-Win-CPU-VS2022-Latest'
@@ -117,10 +126,6 @@ stages:
117126
AgentPool: 'onnxruntime-Ubuntu2204-AMD-CPU'
118127
StageSuffix: 'Linux_CPU_x64'
119128

120-
- template: nodejs/templates/test_macos.yml
121-
parameters:
122-
StageSuffix: 'macOS_CPU_x64'
123-
124129
- template: nuget/templates/test_win.yml
125130
parameters:
126131
AgentPool: 'onnxruntime-Win2022-GPU-A10'
@@ -225,7 +230,7 @@ stages:
225230
- checkout: self
226231
clean: true
227232
submodules: none
228-
233+
229234
- download: build
230235
artifact: 'Windows_Packaging_tensorrt_build_artifacts'
231236
displayName: 'Download Windows GPU Packages Build'
@@ -246,7 +251,7 @@ stages:
246251
versionSpec: "17"
247252
jdkArchitectureOption: x64
248253
jdkSourceOption: 'PreInstalled'
249-
254+
250255
- task: PythonScript@0
251256
displayName: 'Update CTest Path References'
252257
inputs:

‎tools/ci_build/github/azure-pipelines/nodejs/templates/test.yml‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,20 @@ steps:
66

77

88
- task: PowerShell@2
9-
displayName: 'Move Artifact Directory'
9+
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
10+
displayName: 'Move Artifact Directory (Windows)'
1011
inputs:
1112
targetType: 'inline'
1213
script: |
1314
Move-Item -Path "$(Pipeline.Workspace)/build/NPM_packages" -Destination "$(Build.BinariesDirectory)/nodejs-artifact"
1415
16+
- task: CmdLine@2
17+
condition: and(succeeded(), ne(variables['Agent.OS'], 'Windows_NT'))
18+
displayName: 'Move Artifact Directory (POSIX)'
19+
inputs:
20+
script: |
21+
mv "$(Pipeline.Workspace)/build/NPM_packages" "$(Build.BinariesDirectory)/nodejs-artifact"
22+
1523
- script: mkdir e2e_test
1624
workingDirectory: '$(Build.BinariesDirectory)'
1725

@@ -38,4 +46,4 @@ steps:
3846
npm init -y
3947
npm install $(NpmPackageFilesForTest) --onnxruntime-node-install-cuda=skip
4048
node -p "require('onnxruntime-node')"
41-
workingDirectory: '$(Build.BinariesDirectory)/e2e_test'
49+
workingDirectory: '$(Build.BinariesDirectory)/e2e_test'

‎tools/ci_build/github/azure-pipelines/nodejs/templates/test_macos.yml‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
parameters:
22
StageSuffix: ''
3+
AgentPool : 'macOS-15'
4+
UseHostedVmImage: 'true'
5+
PoolDemands: ''
6+
37
stages:
48
- stage: Nodejs_Test_MacOS_${{ parameters.StageSuffix }}
59
dependsOn:
@@ -11,7 +15,12 @@ stages:
1115
clean: all
1216
timeoutInMinutes: 120
1317
pool:
14-
vmImage: 'macOS-15'
18+
${{ if eq(parameters.UseHostedVmImage, 'true') }}:
19+
vmImage: ${{ parameters.AgentPool }}
20+
${{ else }}:
21+
name: ${{ parameters.AgentPool }}
22+
${{ if ne(parameters.PoolDemands, '') }}:
23+
demands: ${{ parameters.PoolDemands }}
1524

1625
variables:
1726
- name: OnnxRuntimeBuildDirectory

0 commit comments

Comments
 (0)