Skip to content

Publish Retries grow the queue without bound #25

@mmmries

Description

@mmmries

If you run the script below without a local copy of rabbitmq running you will get an out of memory error.

require "active_publisher"
require "pry"
::ActivePublisher::Configuration.configure_from_yaml_and_cli({})
::ActivePublisher::Async.publisher_adapter = ::ActivePublisher::Async::InMemoryAdapter.new()

::ActivePublisher.publish_async("wat", "yo yo yo", "myexchange")

loop do
  queue_size = ::ActivePublisher::Async.publisher_adapter.async_queue.size
  puts "queue size is #{queue_size}"
  sleep 0.5
end

You will get output like:

queue size is 1
queue size is 0
queue size is 168
queue size is 828
queue size is 1716
queue size is 3276
queue size is 5336
queue size is 7344
queue size is 10304
queue size is 13764
queue size is 17724
queue size is 21252
queue size is 26112
queue size is 31472
queue size is 36120
queue size is 42380
queue size is 49140
queue size is 54908
queue size is 62568
queue size is 70748
queue size is 77616
queue size is 86676
queue size is 96236
queue size is 104244
queue size is 114704
queue size is 125664
queue size is 134792
queue size is 146652
queue size is 159012
queue size is 169260
queue size is 182520
queue size is 193488
queue size is 207648
queue size is 219336
queue size is 234396

The queue grows unbounded until you run out of memory and die. This is because of how we are re-queuing failed messages.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions