-
-
Notifications
You must be signed in to change notification settings - Fork 98
Description
Taskiq version
0.11.17
Python version
Python 3.11
OS
macOS
What happened?
When encountering a task/a message that can't be found, the TaskIQ receiver logs a warning and then returns (None), see receiver.py
However, this leaves the task/message unacked and forever pending in limbo.
Clearly, if the task can't be found this is not going anywhere and either the broker and/or the result backend should in one way or another be informed.
In our case: we use RabbitMQ as broker and had a handful of "not found" tasks (the reason has been fixed, doesn't really matter) around. They were no longer in queue (due to pre-fetch), so inspecting the message queue or purging it did not help. Terminating the consumer only put those messages back into the queue very briefly, before the re-spawned consumer picked them up again and nothing was gained. (For RabbitMQ specifically, these messages should have been nacked. )
Independent of the exact broker and implementation however this behaviour is problematic: it leaves messages in an undefined state.
Relevant log output
WARNING:receiver.py:119: task "project:dont_ask_me_how_this_happened_task" is not found. Maybe you forgot to import it?Broker initialization code
broker = AioPikaBroker(
url=broker_config.to_url(
vhost=app_config.name
),
queue_name=app_config.name,
delayed_message_exchange_plugin=True,
max_priority=5,
qos=3,
)