Skip to content

HeartbeatManager logs normal shutdown as thread crash #110

Description

@kyle-li-dev

Summary

HeartbeatManager.stop() can log a normal worker shutdown as an error:

HeartbeatManager thread crashed
Traceback (most recent call last):
  File ".../site-packages/litestar_saq/heartbeat.py", line 180, in _run
    self._loop.run_until_complete(self._manager_loop())
  File ".../asyncio/base_events.py", line 723, in run_until_complete
    raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.

This happens during process shutdown / Ctrl-C, after the worker is being stopped normally.

Environment

  • litestar-saq: 0.8.0
  • saq: 0.26.4
  • litestar: 2.23.0
  • Python: 3.13.13
  • Platform: macOS arm64

What appears to be happening

HeartbeatManager.stop() sets the stop flag and then stops the event loop from another thread:

self._stop_event.set()
self._loop.call_soon_threadsafe(self._loop.stop)

But the heartbeat thread is running:

self._loop.run_until_complete(self._manager_loop())

If the loop is stopped while _manager_loop() is sleeping / still pending, run_until_complete() raises:

RuntimeError: Event loop stopped before Future completed.

The generic exception handler then logs this expected shutdown path as:

logger.exception("HeartbeatManager thread crashed")

Expected behavior

A normal worker shutdown should not emit an error-level "thread crashed" log.

Possible approaches:

  • Let _manager_loop() wake up and return naturally instead of calling loop.stop() directly.
  • Track/cancel the manager task explicitly and treat cancellation during shutdown as expected.
  • At minimum, suppress or downgrade RuntimeError("Event loop stopped before Future completed") when _stop_event.is_set().

Actual behavior

The worker appears to stop, but shutdown logs an error traceback, which makes normal Ctrl-C / graceful process termination look like a runtime crash.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions