Kaguya does not work properly with dynamic libraries

[carvalho-paulo]

I found an issue that happens only when using Kaguya from a dynamic library on Linux.

Basically, I have a dynamic library that provides a function (registerInteger) to register a class (Integer) into Lua. In another translation unit, I have a function (triple) that uses that class. The function triple is also register into Lua.

If I build the dynamic library with the option -fvisibility=hidden and export symbols with __attribute__((visibility("default"))) (what is usual), then the function triple will not work from Lua.

You can easily reproduce the issue with gcc or clang with the following example project:

Integer.h:

#pragma once

class __attribute__((visibility("default"))) Integer {
public:
    Integer();
    Integer(int v);
    int get() const;
    void set(int v);
private:
    int value = int{};
};

Integer.cpp:

#include "Integer.h"

Integer::Integer() = default;
Integer::Integer(int v) : value(v) {}
int Integer::get() const { return value; }
void Integer::set(int v) { value = v; }

LuaInteger.h:

#pragma once
#include "kaguya/kaguya.hpp"

void __attribute__((visibility("default"))) registerInteger(kaguya::State &state);

LuaInteger.cpp:

#include "LuaInteger.h"
#include "Integer.h"

void registerInteger(kaguya::State &state)
{
    state["Integer"].setClass(
        kaguya::UserdataMetatable<Integer>()
        .setConstructors<Integer(), Integer(int)>()
        .addFunction("get", &Integer::get)
        .addFunction("set", &Integer::set)
    );
}

main.cpp:

#include <iostream>

#include "kaguya/kaguya.hpp"

#include "Integer.h"
#include "LuaInteger.h"

#define EXPECT(c) do {                                              \
    if (c) {                                                        \
        std::cout << "OK: `" << #c << "` is true" << std::endl;     \
    }                                                               \
    else {                                                          \
        std::cout << "FAIL: `" << #c << "` is false" << std::endl;  \
    }                                                               \
} while(false)

Integer triple(Integer v)
{
    return v.get() * 3;
}

void test_01() {
    kaguya::State state;
    registerInteger(state);
    state["triple"] = &triple;

    state.dostring(R"(        
        v1 = Integer(2)
        v2 = triple(v1)
        v3 = v2:get()
    )");
    
    EXPECT(state["v3"] == 6);
}

int main()
{
    test_01();
    
    return 0;
}

The dynamic library was built with the following command lines:

$ clang++ -fPIC -fvisibility=hidden -Wall -std=c++14 -I. -I/usr/include/lua5.3 -c -o Integer.o Integer.cpp
$ clang++ -fPIC -fvisibility=hidden -Wall -std=c++14 -I. -I/usr/include/lua5.3 -c -o LuaInteger.o LuaInteger.cpp
$ clang++ -o libLuaInteger.so Integer.o LuaInteger.o -shared -Wl,-soname,libLuaInteger.so

The test application was built with the following command lines:

$ clang++ -fPIC -Wall -std=c++14 -I. -I/usr/include/lua5.3 -c -o main.o main.cpp
$ clang++ -o repro main.o -llua5.3 -L. -lLuaInteger

Test:

$ LD_LIBRARY_PATH=. ./repro
maybe...Argument mismatch:Integer	 candidate is:
		Integer,

stack traceback:
	[C]: in function 'triple'
	[string "        ..."]:3: in main chunk
FAIL: `state["v3"] == 6` is false

I used the latest Kaguya with Lua 5.3. I tested the issue with gcc 9.3.0 and clang 9.0.1 on Ubuntu 20.04.

Thanks!