crash when link both libtbb.so and libtbbmalloc_proxy.so

[xufang-lisa]

Summary

App will crash when linking both libtbb.so and libtbbmalloc_proxy.so
free(): invalid pointer
Aborted (core dumped)

Version

master:fa5ed8e81c1ca088b7627e031b9cb059504edb16

Environment

OS: ubuntu 20.04

Observed Behavior

Document behavior you observe.

Expected Behavior

Document behavior you expect.

Steps To Reproduce

Check that the issue is reproducible with the latest revision
of the master branch. Include all the steps to reproduce the issue.
main.cpp

#    ifndef NOMINMAX
#        define NOMINMAX
#    endif
#    ifndef TBB_PREVIEW_LOCAL_OBSERVER
#        define TBB_PREVIEW_LOCAL_OBSERVER 1
#    endif
#    ifndef TBB_PREVIEW_WAITING_FOR_WORKERS
#        define TBB_PREVIEW_WAITING_FOR_WORKERS 1
#    endif
#    ifndef TBB_PREVIEW_NUMA_SUPPORT
#        define TBB_PREVIEW_NUMA_SUPPORT 1
#    endif
#    ifndef TBB_PREVIEW_TASK_ARENA_CONSTRAINTS_EXTENSION
#        define TBB_PREVIEW_TASK_ARENA_CONSTRAINTS_EXTENSION 1
#    endif

#    include "tbb/blocked_range.h"
#    include "tbb/blocked_range2d.h"
#    include "tbb/blocked_range3d.h"
#    include "tbb/parallel_for.h"
#    include "tbb/parallel_reduce.h"
#    include "tbb/parallel_sort.h"
#    include "tbb/task_arena.h"
#    include "tbb/task_scheduler_observer.h"

int main(int argc, char const* argv[])
{
    auto nodes = tbb::info::numa_nodes();
    printf("numa node size:%d\n", nodes.size());
}

CMakeLists.txt

PROJECT("tbb_test")
SET(SRC_LIST main.cpp)
ADD_EXECUTABLE(tbb_test ${SRC_LIST})
target_include_directories(tbb_test PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/tbb/include")
target_link_libraries(tbb_test PRIVATE
              ${CMAKE_CURRENT_SOURCE_DIR}/tbb/lib/libtbbmalloc_proxy.so
              ${CMAKE_CURRENT_SOURCE_DIR}/tbb/lib/libtbb.so
          )

steps:

• Build hwloc and oneTBB

https://github.com/open-mpi/hwloc/archive/refs/tags/hwloc-2.9.3.zip
https://github.com/uxlfoundation/oneTBB/tree/master(fa5ed8e81c1ca088b7627e031b9cb059504edb16)

cmake -DBUILD_SHARED_LIBS=ON -DTBB_DISABLE_HWLOC_AUTOMATIC_SEARCH=ON -DCMAKE_HWLOC_2_5_LIBRARY_PATH=/home/linux-x86/hwloc-hwloc-2.9.3/install/lib/libhwloc.so -DCMAKE_HWLOC_2_5_INCLUDE_PATH=/home/linux-x86/hwloc-hwloc-2.9.3/install/include -DCMAKE_HWLOC_2_5_DLL_PATH=/home/linux-x86/hwloc-hwloc-2.9.3/install/lib -DTBBMALLOC_PROXY_BUILD=ON -DTBB_STRICT=OFF -DCMAKE_BUILD_TYPE=Release -DTBB_TEST=OFF ..

Place the compiled tbb and hwloc (cp hwloc library to tbb/lib64) in the directory where main.cpp is located.

• Build app

mkdir build
cd build
cmake ..
cmake --build .

• Set the environment

export LD_LIBRARY_PATH=../tbb/lib64:${LD_LIBRARY_PATH}

• Run app

./tbb_test

[isaevil]

@xufang-lisa Is presence of HWLOC and TBBBind crucial to reproduce this issue?

[xufang-lisa]

@isaevil Yes, I think so. And this issue and https://jira.devtools.intel.com/browse/TBB-1539 should be duplicates.

[dnmokhov]

The crash occurs because oneTBB dlopens TBBBind with RTLD_DEEPBIND (#426), which causes HWLOC to ignore global symbols, including those replaced by TBBMallocProxy. Libc, however, does not ignore them, so when HWLOC uses libc functions (e.g., asprintf), they allocate using malloc from TBBMallocProxy. The crash occurs when HWLOC tries to release the memory by directly using (non-replaced) free.