Skip to content

ThreadSanitizer false positives from fence-based reference counting in detail::atomic_count (shared_target_executor teardown) #1760

Description

@sketch34

Summary

detail::ref_count_down (asio/detail/atomic_count.hpp) implements reference-count teardown with the standard release-decrement + acquire-fence idiom:

inline bool ref_count_down(atomic_count& a)
{
  if (a.fetch_sub(1, std::memory_order_release) == 1)
  {
    std::atomic_thread_fence(std::memory_order_acquire);
    return true;
  }
  return false;
}

This is correct C++, but ThreadSanitizer does not model standalone std::atomic_thread_fence (google/sanitizers#1352, gcc bug 97868). As a result, whenever the final release of a shared executor target runs on a different thread than a prior decrement, TSan reports a data race between the operator delete in shared_target_executor::impl<...>::~impl() and the previous fetch_sub — a false positive, but one that is indistinguishable from a real race in CI.

This is the runtime cousin of the compile-time -Wtsan warning already tracked for std_fenced_block in #1193 / #1512; the same TSan limitation, but here it produces actual race reports rather than a warning, so it cannot be silenced with -Wno-tsan.

Trigger

Any pattern that lets copies of a shared any_executor target die on different threads. A reliable one, from a C++20-coroutine TCP/WebSocket proxy: an HTTP-session coroutine hops to another strand for a snapshot and back —

co_return co_await asio::co_spawn(
    other_strand_,            // asio::strand<asio::any_io_executor>
    [self]() -> asio::awaitable<Snapshot> { co_return self->snapshot(); },
    asio::use_awaitable);

co_spawn type-erases the strand into the spawned frame's any_io_executor, heap-allocating a shared_target_executor::impl<strand<any_io_executor>>. The spawned frame's copy is destroyed on the strand's thread; the initiation's copy is destroyed wherever the awaiting coroutine resumes. When the strand is busy (completion posted cross-thread rather than run inline), the two destructions race per TSan. Under a modest load (a few hundred concurrent client sessions churning while the endpoint above is polled ~4×/s) the report fires within seconds.

Reproduced byte-for-byte identically on Boost 1.87.0 and Boost 1.91.0 (GCC 15, Linux x86_64, -O2 -fsanitize=thread); nothing in the range fixes or changes the pattern, as expected given the code is correct.

Representative report (Boost 1.91.0, paths trimmed)

WARNING: ThreadSanitizer: data race
  Write of size 8 at 0x7218... by thread T6:
    #0 operator delete(void*, unsigned long)
    #1 execution::detail::shared_target_executor::impl<strand<any_io_executor>>::~impl() boost/asio/execution/any_executor.hpp:477
    #2 execution::detail::shared_target_executor::~shared_target_executor() boost/asio/execution/any_executor.hpp:453
    #3 execution::detail::any_executor_base::destroy_shared(...) boost/asio/execution/any_executor.hpp:772
    #4 execution::detail::any_executor_base::~any_executor_base() boost/asio/execution/any_executor.hpp:579
    #5 any_io_executor::~any_io_executor() boost/asio/impl/any_io_executor.ipp:81
    #6 detail::initiate_co_spawn<any_io_executor>::~initiate_co_spawn() boost/asio/impl/co_spawn.hpp:303
    ...

  Previous atomic write of size 8 at 0x7218... by thread T25:
    #0 std::__atomic_base<long>::fetch_sub(long, std::memory_order)
    #1 detail::ref_count_down(std::atomic<long>&) boost/asio/detail/atomic_count.hpp
    #2 execution::detail::shared_target_executor::~shared_target_executor() boost/asio/execution/any_executor.hpp:453
    #3 execution::detail::any_executor_base::destroy_shared(...) boost/asio/execution/any_executor.hpp:772
    #4 execution::detail::any_executor_base::~any_executor_base() boost/asio/execution/any_executor.hpp:579
    #5 any_io_executor::~any_io_executor() boost/asio/impl/any_io_executor.ipp:81
    #6 detail::awaitable_frame<awaitable_thread_entry_point, any_io_executor>::~awaitable_frame() boost/asio/impl/awaitable.hpp:623
    #7 co_spawn_entry_point boost/asio/impl/co_spawn.hpp:191

Suggestion

libstdc++ solved the identical problem for std::shared_ptr: _Sp_counted_base::_M_release carries _GLIBCXX_TSAN-conditional annotations so its fence-based counting is TSan-clean. Two options that would do the same for asio, in the spirit of the patch already proposed for std_fenced_block in #1193:

  1. Annotate ref_count_down / ref_count_up with __tsan_release / __tsan_acquire (from sanitizer/tsan_interface.h) under __has_feature(thread_sanitizer) / __SANITIZE_THREAD__; or
  2. Fall back to fetch_sub(std::memory_order_acq_rel) in ref_count_down under the same detection — slightly stronger ordering, but only in sanitizer builds where precise performance is already off the table.

Impact

Without one of these, every TSan user whose executors tear down across threads (any multi-threaded io_context + type-erased strands, co_spawn, etc.) has to ship a suppressions file for asio internals, and halt_on_error=1 soak/CI runs die on the false positive before they can surface real races.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions