Build error on Ubuntu 20.04.3: requires -fPIC

[BasantaChaulagain]

Hello,

I am trying to install OpenSSE on Ubuntu 20.04.3. I have installed all the dependencies needed, I am able to execute 'cmake', but I get the following error while trying to 'make' it.

..
[ 52%] Building CXX object lib/CMakeFiles/schemes.dir/tethys/tethys_allocator.cpp.o
[ 53%] Building CXX object lib/CMakeFiles/schemes.dir/pluto/rocksdb_store.cpp.o
[ 54%] Linking CXX shared library libschemes.so
/usr/bin/ld: /usr/local/lib/librocksdb.a(concurrent_arena.o): relocation R_X86_64_TPOFF32 against symbol `_ZN7rocksdb15ConcurrentArena9tls_cpuidE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/lib/librocksdb.a(iostats_context.o): relocation R_X86_64_TPOFF32 against symbol `_ZN7rocksdb15iostats_contextE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/lib/librocksdb.a(perf_context.o): relocation R_X86_64_TPOFF32 against `__tls_guard' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/lib/librocksdb.a(perf_level.o): relocation R_X86_64_TPOFF32 against symbol `_ZN7rocksdb10perf_levelE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/lib/librocksdb.a(thread_status_updater.o): relocation R_X86_64_TPOFF32 against symbol `_ZN7rocksdb19ThreadStatusUpdater19thread_status_data_E' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/lib/librocksdb.a(thread_status_util.o): relocation R_X86_64_TPOFF32 against symbol `_ZN7rocksdb16ThreadStatusUtil27thread_updater_initialized_E' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/lib/librocksdb.a(random.o): relocation R_X86_64_TPOFF32 against `_ZZN7rocksdb6Random14GetTLSInstanceEvE12tls_instance' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/lib/librocksdb.a(thread_local.o): relocation R_X86_64_TPOFF32 against symbol `_ZN7rocksdb14ThreadLocalPtr10StaticMeta4tls_E' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/lib/librocksdb.a(db_impl_open.o): relocation R_X86_64_PC32 against symbol `_ZN7rocksdb32kPersistentStatsColumnFamilyNameB5cxx11E' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[2]: *** [lib/CMakeFiles/schemes.dir/build.make:334: lib/libschemes.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:429: lib/CMakeFiles/schemes.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

I tried editing the CMakeLists.txt by adding the missing flag when compiling, but I still get the same error.

set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC" ) 
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fPIC" )

Please let me know if any other information is required.

[rbost]

Hi Basanta,

This is not directly related to my code, but most certainly due to the way you compiled and installed RocksDB.
Being blind with respect to your environment, I would recommend you to use the shared_lib install described in RocksDB Install instructions after removing the previous RocksDB installation artefacts.
Otherwise, could you tell me which CLI commands you used to install RocksDB?

Best,
RB

[BasantaChaulagain]

Hi @rbost,

I used 'make all' to install RocksDB, but I will try it with 'make shared_lib' as you suggested and let you know how it goes.

[BasantaChaulagain]

Hi @rbost, I got rid of the previous error, but now it gives me undefined reference errors. I checked that these programs are installed in the system.

.....
[ 66%] Building CXX object src/CMakeFiles/diana_server.dir/diana_server.cpp.o
[ 67%] Linking CXX executable diana_server
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `snappy::MaxCompressedLength(unsigned long)'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_loadDictHC'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_resetStreamHC'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_freeCCtx'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_setStreamDecode'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_freeStreamDecode'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_createStreamDecode'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `BZ2_bzDecompress'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_freeCDict'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_versionNumber'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_freeStream'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_compressBound'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `BZ2_bzCompressInit'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_compressBound'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_freeStreamHC'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `snappy::GetUncompressedLength(char const*, unsigned long, unsigned long*)'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_createStreamHC'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_createStream'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_decompress_usingDict'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZDICT_trainFromBuffer'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `BZ2_bzDecompressInit'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `BZ2_bzDecompressEnd'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_compress_fast_continue'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `BZ2_bzCompressEnd'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_freeDCtx'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_createDCtx'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_createCCtx'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZDICT_isError'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_compress_usingCDict'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_decompress_safe_continue'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_loadDict'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_compress_usingDict'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `snappy::RawUncompress(char const*, unsigned long, char*)'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `LZ4_compress_HC_continue'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `BZ2_bzCompress'
/usr/bin/ld: ../lib/libschemes.so: undefined reference to `ZSTD_createCDict'
collect2: error: ld returned 1 exit status
make[2]: *** [src/CMakeFiles/diana_server.dir/build.make:137: src/diana_server] Error 1
make[1]: *** [CMakeFiles/Makefile2:460: src/CMakeFiles/diana_server.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

This is the line 137 of src/CMakeFiles/diana_server.dir/build.make::

135 - src/diana_server: src/CMakeFiles/diana_server.dir/link.txt
136 - 	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green --bold --progress-dir=/home/lab301/Projects/opensse-schemes/build/CMakeFiles --progress-num=$(CMAKE_PROGRESS_2) "Linking CXX executable diana_server"
137 - 	cd /home/lab301/Projects/opensse-schemes/build/src && $(CMAKE_COMMAND) -E cmake_link_script CMakeFiles/diana_server.dir/link.txt --verbose=$(VERBOSE)

Line 460 of CMakeFiles/Makefile2
$(MAKE) -f src/CMakeFiles/diana_server.dir/build.make src/CMakeFiles/diana_server.dir/build

Any help will be appreciated.

[rbost]

So first of all, I agree that the instructions in the README file are faulty: after giving it a thought, using make static_lib to build RocksDB induces the issues you encountered. I am definitely the one to blame here and I will fix that quickly.

Now, I built the project in a fresh Ubuntu 20 LTS install, with the latest version of RocksDB, and here are two solutions I found to work. What follows are the installations step for RocksDB:

1. Run make shared_lib and sudo make install => Be sure to run make clean beforehand if you've already built RocksDB. Also, remove the existing artefacts if you ran make install.
2. Install using CMake:

mkdir build && cd build
cmake ..
make
sudo make install

I understand you tried solution 1. Maybe retry after cleaning everything.
Otherwise, use solution 2 (but clean all the artefacts still).

[BasantaChaulagain]

Hi @rbost, Thank you for the response. I tried both the solutions you mentioned, and still got the same error as mentioned above. If you were able to build the project in Ubuntu 20 LTS, it is probably due to my environment. I will try with a fresh installation of Ubuntu.

[BasantaChaulagain]

Hi @rbost,

Can you please tell me which version of rocksdb was originally used to build the project? I am trying to build the opensse in Ubuntu 16, but still getting the same error as above. I tried both of the solutions you mentioned above to install the latest version of RocksDB.

[ 67%] Linking CXX executable diana_server
../lib/libschemes.so: undefined reference to `LZ4_loadDictHC'
../lib/libschemes.so: undefined reference to `LZ4_compress_fast_continue'
../lib/libschemes.so: undefined reference to `LZ4_resetStreamHC'
../lib/libschemes.so: undefined reference to `BZ2_bzCompressEnd'
../lib/libschemes.so: undefined reference to `ZSTD_freeCCtx'
../lib/libschemes.so: undefined reference to `LZ4_setStreamDecode'
../lib/libschemes.so: undefined reference to `LZ4_freeStreamDecode'
../lib/libschemes.so: undefined reference to `ZSTD_freeDCtx'
../lib/libschemes.so: undefined reference to `LZ4_createStreamDecode'
../lib/libschemes.so: undefined reference to `ZSTD_createDCtx'
../lib/libschemes.so: undefined reference to `BZ2_bzDecompress'
../lib/libschemes.so: undefined reference to `ZSTD_createCCtx'
../lib/libschemes.so: undefined reference to `ZSTD_versionNumber'
../lib/libschemes.so: undefined reference to `LZ4_freeStream'
../lib/libschemes.so: undefined reference to `LZ4_compressBound'
../lib/libschemes.so: undefined reference to `ZSTD_compressBound'
../lib/libschemes.so: undefined reference to `BZ2_bzCompressInit'
../lib/libschemes.so: undefined reference to `LZ4_decompress_safe_continue'
../lib/libschemes.so: undefined reference to `LZ4_freeStreamHC'
../lib/libschemes.so: undefined reference to `LZ4_loadDict'
../lib/libschemes.so: undefined reference to `ZSTD_compress_usingDict'
../lib/libschemes.so: undefined reference to `snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)'
../lib/libschemes.so: undefined reference to `snappy::RawUncompress(char const*, unsigned long, char*)'
../lib/libschemes.so: undefined reference to `snappy::GetUncompressedLength(char const*, unsigned long, unsigned long*)'
../lib/libschemes.so: undefined reference to `LZ4_createStreamHC'
../lib/libschemes.so: undefined reference to `LZ4_createStream'
../lib/libschemes.so: undefined reference to `ZSTD_decompress_usingDict'
../lib/libschemes.so: undefined reference to `LZ4_compress_HC_continue'
../lib/libschemes.so: undefined reference to `BZ2_bzDecompressInit'
../lib/libschemes.so: undefined reference to `BZ2_bzCompress'
../lib/libschemes.so: undefined reference to `BZ2_bzDecompressEnd'
../lib/libschemes.so: undefined reference to `snappy::MaxCompressedLength(unsigned long)'
collect2: error: ld returned 1 exit status
src/CMakeFiles/diana_server.dir/build.make:147: recipe for target 'src/diana_server' failed
make[2]: *** [src/diana_server] Error 1
CMakeFiles/Makefile2:435: recipe for target 'src/CMakeFiles/diana_server.dir/all' failed
make[1]: *** [src/CMakeFiles/diana_server.dir/all] Error 2
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2

Could there be any other mis-configured dependencies causing this issue?
P.S: I tried building the program on 'release' mode, but the issue persists.