Unable to load custom librdkafka builds on OSX

[mhowlett]

On OSX, calling Library.Load(path) with an absolute path ignores the absolute path and loads the librdkafka.dylib included in the librdkafka.redist package referenced by Confluent.Kafka. This is the case even if the librdkafka.dylib we want to load is renamed to something else. The only way I've been able to load a custom build of librdkafka is to remove the librdkafka.redist dependency of Confluent.Kafka. Library.Load on OSX uses dlopen. The docs here https://www.unix.com/man-page/osx/3/dlopen/ suggest this behavior is unexpected. I assume the CoreCLR is doing something special here (not investigated).

[mhowlett]

I believe #567 is a manifestation of this issue (but on linux).

[Goffen]

I got same issue in new Mac m1 and librdkafka installed using brew. Then try to use it using Library.Load("/opt/homebrew/lib/librdkafka.dylib");

[Hoffs]

Ran into this as well, that dlopen succeeds but subsequent extern calls that use DllImport with just lib name fail to work. Seems to me that this issue in dotnet runtime repo talks about same behaviour: dotnet/runtime#12028 . The conclusion is that said approach is not valid and ideally it should instead use loaded/returned handle and resolve symbols from that.

This issue mentions using DllImportResolver to maintain using DllImport attributes: dotnet/runtime#36343

More recent issue that talks about same-ish behaviour: dotnet/runtime#58601

[anchitj]

Is this still an issue?

[justinpenguin45]

I am currently running into issues running a Kafka .net client consumer in a Docker container/environment. We have a plugin architecture and the Confluent c# library is loaded using AssemblyLoadContext. When running outside of Docker (from VS 2022 in Windows) it runs fine. Running inside the container though we get an error that librdkafka could not load due to not being able to find libsasl2.so.3

I hacked at this for a day and a half and could not get anywhere