Linking multiple codon libraries together

[cainamisir]

Hi, I am wondering if there is a way in which we can link multiple codon libraries together. Let's say I have a C++ library which cals a codon function x(). Then, in my main c++ code, I call another y() function from a different codon library, as well as the C++ library from above. This ends up in segfaults in main[unclash]

Right now the solution I have is to create a 3rd codon file and just import the 2 libraries there. Then call codon build on this single file. But this can get rather ugly

[arshajii]

Hi @cainamisir -- could you please share more details like what system you're on? Also are you linking the Codon libraries when building the C++ program or are you dlopen()ing them at runtime? I tried to reproduce this locally on macOS but seems to work at least when linking during building.

[cainamisir]

Coming here with more details - 2 of the libraries use the @python decorator. and everything works if I just make a file with imports to all 4 libraries and compile that alone.

[arshajii]

Hi @cainamisir -- it appears that this is due to the runtime library's initialization function (seq_init) being called multiple times, although in theory this shouldn't be an issue. Could you please try the simple example above and let me know if it crashes on your system? I also tried with 4 libraries on x86_64 Linux/Ubuntu and it worked as expected. It's simple to add an "execute-once" flag to the init function but I just want to verify that this is indeed the issue first.

[cainamisir]

That example does not crash for me. maybe it has to do with the @python usage in my project?

[arshajii]

I think that's right... using Python from one of the libraries seems to cause an issue when multiple libraries are linked. I'm currently looking into that...

[arshajii]

(I was able to reproduce the crash by calling Python from the two modules.)

[arshajii]

I found the root cause of this issue after some investigation, which actually has nothing to do with the runtime library initialization. The problem was that the global constructor Codon generated to call the main code when loading a shared library was externally visible, which caused a clash if multiple Codon-compiled libraries were linked as in your example. This is fixed in f21fa9b and will be in the next release!