Skip to content

abort uTP sockets when their UDP socket is torn down (fixes shutdown hang) + teardown regression test - #8552

Open
KSEGIT wants to merge 2 commits into
arvidn:RC_2_1from
KSEGIT:lsd-async-announce
Open

abort uTP sockets when their UDP socket is torn down (fixes shutdown hang) + teardown regression test#8552
KSEGIT wants to merge 2 commits into
arvidn:RC_2_1from
KSEGIT:lsd-async-announce

Conversation

@KSEGIT

@KSEGIT KSEGIT commented Jul 7, 2026

Copy link
Copy Markdown

Problem

qBittorrent on macOS freezes permanently (force-quit required) after a sleep/wake cycle with a VPN active — qbittorrent/qBittorrent#24353, with spindump and live-process sample evidence. Investigation found two distinct defects behind the same user-visible hang, both triggered by a network interface stalling/vanishing across sleep/wake (e.g. a VPN utunN tunnel):

  1. Blocking service sends: synchronous, untimed send_to() in LSD/UPnP/NAT-PMP parks the single io thread in a poll() writability wait that never returns (the spindump attached to the issue shows the io thread in on_lsd_announce → announce_lsd → lsd::announce_impl → poll across all samples). Now fixed upstream by 4138b27 ("fix blocking UDP socket in UPnP") and 38d78f0 ("non-blocking UDP socket in LSD and NAT-PMP") — this PR originally carried equivalent changes and has been rebased to drop them in favor of upstream's.
  2. uTP teardown leak (this PR, second commit): even with non-blocking sends, session shutdown can hang forever on orphaned uTP sockets. Observed live on qBittorrent 5.2.2 / libtorrent 2.0.12.

Commit 1 — add regression test for bounded session teardown with LSD active

test/test_lsd_teardown.cpp (registered explicit in the Jamfile, matching its sibling test_lsd, and added to the Makefile dist list): with LSD enabled and an announcing torrent (driven via force_lsd_announce()), abort() + session_proxy destruction must complete within a bounded time; a watchdog (joined via a scope guard so an exception cannot unwind through a joinable thread) converts any future teardown deadlock into a fast, attributed failure instead of an indefinite test-binary hang.

Honest scope, also stated in the test: a healthy CI interface cannot simulate the stalled-interface trigger, and with no peer connections the test does not exercise the uTP teardown path below — it pins the bounded-teardown invariant. A deterministic simulation test for the uTP stall itself (building on simulation/test_utp.cpp's utp_small_kernel_send_buf backpressure pattern, plus a mid-transfer abort) is a natural follow-up; happy to add it here or separately if preferred.

Commit 2 — abort uTP sockets when their UDP socket is torn down

Observed live (qBittorrent 5.2.2 / libtorrent 2.0.12, macOS): quit initiated, resume data saved, then a permanent hang with the io thread alive — parked in kqueue_reactor::run → kevent and caught mid-session_impl::on_tick re-arming its 100 ms timer — zero open OS sockets, GUI blocked in the teardown join. sample captures attached below.

Root cause chain:

  • utp_stream stores its completion handlers in plain std::function members (aux_/utp_stream.hpp); each handler owns a shared_ptr<peer_connection>, and the utp_stream is itself a member of that peer — a reference cycle (peer → socket → handler → peer) breakable only by utp_socket_impl::cancel_handlers().
  • session_impl::abort() closes the UDP sockets without telling the uTP socket managers — utp_socket_manager::remove_udp_socket(), the function designed for exactly this, had no callers (the abort() code even carries a TODO: "closing the udp sockets here means that the uTP connections cannot be closed gracefully").
  • A uTP socket that stalled on a send (EWOULDBLOCK — e.g. the vanished VPN interface; note the new non-blocking service sends make stalls like this more common, not less) loses its only wake-up when the UDP socket closes: session_impl::on_udp_writeable drops the notification on error (if (ec) return;), and should_delete() requires !m_stalled, so the stalled socket can never be deleted.
  • on_tick() only initiates abort_stage2() (→ m_work.reset()) once m_undead_peers is empty and num_sockets() == 0. An orphaned uTP socket pins its peer in m_undead_peers forever and itself keeps num_sockets() > 0 — so the tick re-arms every 100 ms indefinitely, io_context::run() never returns, and session_proxy::~session_proxy()'s join() blocks forever.

Fix: close each listen socket's UDP socket through a new close_udp_listen_socket() helper — used by both session_impl::abort() and the runtime listen-socket removal path (the latter also stops the same references leaking while the session keeps running) — which notifies the uTP socket managers. remove_udp_socket() first flushes all stalled sockets via the existing writable() (the same unfiltered delivery on_udp_writeable() performs), so sockets stalled on the closed socket fail their sends and tear down (clearing m_stalled, making them deletable), then aborts every uTP socket bound to the closed UDP socket. Its must-be-closed-first precondition is documented at the declaration and definition. A ChangeLog entry is included.

Related recent work in this same teardown region: f6fdabb ("fix issue in uTP shutdown"), 507c600 ("fix use-after-free in uTP failure socket shutdown path") — those fix crashes; this fixes the hang.

Verification

  • test_lsd, test_lsd_teardown (new), test_utp: pass on macOS (arm64) and Linux, on v2.0.12+patch and on RC_2_0 (on top of 4138b27/38d78f022). After rebasing this branch onto current RC_2_1, re-verified test_lsd_teardown and test_utp on macOS (arm64): both pass.
  • On a patched qBittorrent 5.2.0 build on macOS: a 1 ms-interval sample of a quit shows the io thread actively executing abort()/teardown; the app exits in ~2.3 s through the previously-hanging ~session_proxy → join path (Dock-quit AppleEvent path: 5.9 s).
  • Evidence attached below: two sample captures of the live uTP-variant hang (19:56 and 20:08 — the second catches the on_tick re-arm), sanitized timeline (app log + pmset sleep/wake correlation), and lsof output showing zero network sockets at hang time.

Related

qbittorrent/qBittorrent#24353 (primary report) · #4510, #2861, #1199 (session-destructor hangs) · qbittorrent/qBittorrent#23695, #23604, #13012, #19666 (same symptom) · downstream defense-in-depth: qbittorrent/qBittorrent#24649 (bounded default shutdown timeout).

🤖 Generated with Claude Code

https://claude.ai/code/session_01JbRhL98sdGs31KAXE9uaj7

@KSEGIT
KSEGIT marked this pull request as ready for review July 7, 2026 16:59
Copilot AI review requested due to automatic review settings July 7, 2026 16:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR prevents the libtorrent network thread from being indefinitely blocked by LSD (Local Service Discovery) multicast announces by switching the announce send path from a synchronous send_to() to async_send_to(). It also adds a regression test intended to ensure session teardown remains bounded when LSD is enabled.

Changes:

  • Convert lsd::announce_impl() to use async_send_to() and add an on_send() completion handler plus a per-interface m_send_pending guard.
  • Cancel LSD retry timer and mark the interface disabled on send errors (while ignoring operation_aborted).
  • Add a new teardown-boundedness test and register it in the Jamfile.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
src/lsd.cpp Reworks LSD announce sending to be asynchronous and adds completion/error handling.
include/libtorrent/lsd.hpp Adds on_send() declaration and tracks m_send_pending state.
test/test_lsd_teardown.cpp Adds a regression test to ensure LSD-enabled session teardown completes within a bound.
test/Jamfile Registers the new LSD teardown test.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/lsd.cpp Outdated
Comment on lines +166 to +190
auto msg = std::make_shared<std::array<char, 200>>();
int const msg_len = render_lsd_packet(msg->data(), int(msg->size())
, listen_port, aux::to_hex(ih).c_str()
, m_cookie, v4 ? v4_address : v6_address);

udp::endpoint const to(v4 ? address(lsd_multicast_addr4) : address(lsd_multicast_addr6)
, lsd_port);
udp::endpoint const to(v4 ? address(lsd_multicast_addr4) : address(lsd_multicast_addr6)
, lsd_port);

#ifndef TORRENT_DISABLE_LOGGING
debug_log("==> LSD: ih: %s port: %d [iface: %s]", aux::to_hex(ih).c_str()
, listen_port, m_listen_address.to_string().c_str());
debug_log("==> LSD: ih: %s port: %d [iface: %s]", aux::to_hex(ih).c_str()
, listen_port, m_listen_address.to_string().c_str());
#endif

m_socket.send_to(boost::asio::buffer(msg, static_cast<std::size_t>(msg_len))
, to, {}, ec);
if (ec)
{
m_disabled = true;
#ifndef TORRENT_DISABLE_LOGGING
if (should_log())
{
debug_log("*** LSD: failed to send message: (%d) %s", ec.value()
, ec.message().c_str());
}
#endif
}
}
// Send asynchronously. A synchronous send_to() blocks the single
// io_context thread in-kernel when the outbound interface has stalled
// across sleep/wake (e.g. a VPN utunN tunnel that vanished). That wedges
// the whole network thread (static transfer rates) and prevents the
// session from being aborted/joined on shutdown, requiring a force quit
// (qbittorrent/qBittorrent#24353). async_send_to keeps the io thread in
// its run loop so abort() and per-interface error handling can proceed.
ADD_OUTSTANDING_ASYNC("lsd::on_send");
m_send_pending = true;
m_socket.async_send_to(boost::asio::buffer(msg->data()
, static_cast<std::size_t>(msg_len)), to
, std::bind(&lsd::on_send, self(), msg, _1, _2));

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is no longer part of this PR — the branch was rebased and the LSD changes were dropped in favor of upstream 38d78f0/4138b27d5; the current diff does not touch src/lsd.cpp. On the substance (which predates both): the cast is unreachable-safe here. The format string is pure-ASCII %s/%d/%x, so snprintf cannot fail with a negative return for encoding reasons, and the maximum rendered length is bounded at 139 bytes (69 literal + 17 for "[ff15::efc0:988f]" + 5 for a 16-bit port + 40 for the hex infohash + 8 for the cookie, which is masked to 31 bits) against the 200-byte buffer, so truncation cannot occur either. A defensive if (msg_len < 0 || msg_len >= int(sizeof(msg))) return; would be reasonable hardening, but it belongs in a separate change against the upstream code.

Comment thread test/test_lsd_teardown.cpp Outdated
Comment on lines +92 to +96
ses.add_torrent(atp, ec);
TEST_CHECK(!ec);

// give the io thread time to emit at least one LSD announce
std::this_thread::sleep_for(milliseconds(1500));

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented in the current revision: the test now drives the announce deterministically via th.force_lsd_announce() (test_lsd_teardown.cpp:131) and the settle sleep is reduced to 500ms.

@KSEGIT
KSEGIT force-pushed the lsd-async-announce branch from a3b56b6 to 7f96312 Compare July 7, 2026 20:20
@KSEGIT KSEGIT changed the title lsd: send announce asynchronously to avoid wedging the io thread put service-discovery sockets in non-blocking mode so sends cannot wedge the io thread Jul 7, 2026
@KSEGIT
KSEGIT force-pushed the lsd-async-announce branch from 7f96312 to 4786a6e Compare July 7, 2026 20:32
@KSEGIT

KSEGIT commented Jul 7, 2026

Copy link
Copy Markdown
Author

Reworked the patch based on further review (force-pushed 4786a6e21):

  • Approach changed from async_send_to to non_blocking(true) + synchronous send, matching the existing udp_socket.cpp precedent. Smaller diff, no handler/buffer lifetime machinery, and a would_block send simply drops the datagram (these are best-effort discovery packets with retry at every layer).
  • Extended the same fix to UPnP (SSDP M-SEARCH) and NAT-PMP sockets — they had the identical blocking-send pattern, so the sleep/wake wedge in Constantly seems to freeze on OSX 26.2 qbittorrent/qBittorrent#24353 could still trigger through them (would_block on the M-SEARCH sockets is treated as transient rather than disabling port forwarding).
  • Test hardening: added a watchdog so a future teardown deadlock fails fast with attribution instead of hanging the test binary; the LSD announce is now driven deterministically via force_lsd_announce() (addresses the inline comment about the fixed sleep). The other inline comment concerned the snprintf length cast — that code is back to its upstream form, untouched by this PR.

Verification re-run after the rework: test_lsd + test_lsd_teardown pass on macOS (arm64) and Linux on this branch.

@KSEGIT
KSEGIT force-pushed the lsd-async-announce branch from 4786a6e to 06e31aa Compare July 8, 2026 16:13
@KSEGIT KSEGIT changed the title put service-discovery sockets in non-blocking mode so sends cannot wedge the io thread lsd, upnp, natpmp: use non-blocking sends to avoid wedging the io thread Jul 8, 2026
@KSEGIT
KSEGIT marked this pull request as draft July 8, 2026 22:20
@KSEGIT KSEGIT changed the title lsd, upnp, natpmp: use non-blocking sends to avoid wedging the io thread fix session shutdown hangs when a network interface stalls (LSD/UPnP/NAT-PMP sends + uTP teardown) Jul 8, 2026
@KSEGIT
KSEGIT force-pushed the lsd-async-announce branch from ed0b381 to c2af746 Compare July 9, 2026 10:45
@KSEGIT KSEGIT changed the title fix session shutdown hangs when a network interface stalls (LSD/UPnP/NAT-PMP sends + uTP teardown) abort uTP sockets when their UDP socket is torn down (fixes shutdown hang) + teardown regression test Jul 9, 2026
@KSEGIT
KSEGIT force-pushed the lsd-async-announce branch from c2af746 to 77f4775 Compare July 9, 2026 14:22
@KSEGIT
KSEGIT changed the base branch from RC_2_0 to RC_2_1 July 28, 2026 12:27
@KSEGIT
KSEGIT force-pushed the lsd-async-announce branch 2 times, most recently from f2adc7b to a49e744 Compare August 3, 2026 14:04
KSEGIT and others added 2 commits August 3, 2026 15:07
With LSD enabled and an announcing torrent, abort() + session_proxy
destruction must complete within a bounded time. Guards against the
shutdown wedges reported in qbittorrent/qBittorrent#24353 (fixed by
4138b27, 38d78f0 and the following commit). A watchdog (joined via
a scope guard, so an exception cannot unwind through a joinable
thread) converts any future teardown deadlock into a fast, attributed
failure instead of an indefinite test hang. Registered as explicit,
like test_lsd, and added to the Makefile dist list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JbRhL98sdGs31KAXE9uaj7
utp_stream stores its completion handlers in plain std::function
members; each handler owns a shared_ptr to the peer_connection, and the
utp_stream is itself a member of that peer_connection — a reference
cycle (peer -> socket -> handler -> peer) that only
utp_socket_impl::cancel_handlers() can break.

session_impl::abort() closes the UDP sockets without telling the uTP
socket managers: utp_socket_manager::remove_udp_socket() — the function
designed for exactly this — had no callers. Worse, a uTP socket that
had stalled on a send (EWOULDBLOCK, e.g. on an interface that went away
across a sleep/wake cycle) loses its only wake-up when the UDP socket
closes, because session_impl::on_udp_writeable drops the notification
on error; and should_delete() requires !m_stalled, so the stalled
socket can never be deleted.

The result is a permanent shutdown hang: session_impl::on_tick() only
initiates abort_stage2() once m_undead_peers is empty and
m_utp_socket_manager.num_sockets() reaches zero. An orphaned uTP socket
pins its peer_connection in m_undead_peers forever (and itself keeps
num_sockets() > 0), so the tick re-arms every 100ms indefinitely,
io_context::run() never returns, and session_proxy::~session_proxy()'s
join() blocks forever. Observed live in qbittorrent/qBittorrent#24353:
after a sleep/wake cycle with a VPN active, quitting qBittorrent hangs
with the io thread ticking in kevent, zero open sockets, and the GUI
blocked in the session teardown join.

Fix: close each listen socket's UDP socket through a new
close_udp_listen_socket() helper (used by both abort() and the runtime
listen-socket removal path) which tells the uTP socket managers the
socket is gone. remove_udp_socket() first flushes all stalled sockets
via writable() — the same unfiltered delivery on_udp_writeable()
performs — so sockets stalled on the closed socket fail their sends
and tear down (clearing m_stalled, making them deletable), then aborts
every uTP socket bound to the closed UDP socket. remove_udp_socket()
requires the UDP socket to be closed first; this precondition is
documented at the declaration and definition.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JbRhL98sdGs31KAXE9uaj7
@KSEGIT
KSEGIT force-pushed the lsd-async-announce branch from a49e744 to dc64304 Compare August 3, 2026 14:27
@KSEGIT
KSEGIT marked this pull request as ready for review August 4, 2026 21:39
@KSEGIT
KSEGIT requested a lite review from Copilot August 4, 2026 21:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Ready to approve

The changes are cohesive and correctly route UDP listen-socket teardown through uTP manager notification, with CI/build updates and a regression test to prevent future teardown hangs.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details
  • Files reviewed: 9/9 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

@KSEGIT

KSEGIT commented Aug 4, 2026

Copy link
Copy Markdown
Author

Ready to merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants