SemaphoreFullException in PublisherConfirmationTracking during BasicPublishAsync (MassTransit + RabbitMQ.Client)

[Cloutout]

Describe the bug

We are encountering an exception when publishing messages to RabbitMQ using MassTransit with publisher confirmations enabled.

The exception occurs inside RabbitMQ.Client.Impl.Channel.MaybeEndPublisherConfirmationTrackingAsync, and is wrapped in an InvalidOperationException with the message:

"BUG FOUND - please report this exception"

The inner exception is:

System.Threading.SemaphoreFullException: Adding the specified count to the semaphore would cause it to exceed its maximum count.

This suggests a potential concurrency issue or bug in the publisher confirmation tracking mechanism of the RabbitMQ .NET client.

Reproduction steps

Unfortunately, the issue is intermittent and not easily reproducible on demand.

Observed behavior:

1. A message is published using MassTransit
2. Publisher confirmations are enabled (awaiting ACK)
3. Under production load, the following exception is thrown during publish

❗ Actual Result
Publish operation fails with an exception
Message delivery status becomes uncertain
MassTransit logs an S-FAULT event

Expected behavior

Message should be published successfully
ACK should be received without any exception
No internal semaphore or confirmation tracking errors should occur

Additional context

stack trace

[
     at RabbitMQ.Client.Impl.Channel.MaybeEndPublisherConfirmationTrackingAsync(PublisherConfirmationInfo publisherConfirmationInfo, CancellationToken cancellationToken)
     at RabbitMQ.Client.Impl.Channel.BasicPublishAsync[TProperties](String exchange, String routingKey, Boolean mandatory, TProperties basicProperties, ReadOnlyMemory`1 body, CancellationToken cancellationToken)
     at MassTransit.RabbitMqTransport.RabbitMqChannelContext.<>c__DisplayClass12_1.<<BasicPublishAsync>g__WaitAck|1>d.MoveNext() in /_/src/Transports/MassTransit.RabbitMqTransport/RabbitMqTransport/RabbitMqChannelContext.cs:line 54
  --- End of stack trace from previous location ---
     at MassTransit.RabbitMqTransport.ScopeChannelContext.BasicPublishAsync(String exchange, String routingKey, Boolean mandatory, BasicProperties basicProperties, Byte[] body, Boolean awaitAck, CancellationToken cancellationToken) in /_/src/Transports/MassTransit.RabbitMqTransport/RabbitMqTransport/ScopeChannelContext.cs:line 40
     at MassTransit.RabbitMqTransport.SharedChannelContext.BasicPublishAsync(String exchange, String routingKey, Boolean mandatory, BasicProperties basicProperties, Byte[] body, Boolean awaitAck, CancellationToken cancellationToken) in /_/src/Transports/MassTransit.RabbitMqTransport/RabbitMqTransport/SharedChannelContext.cs:line 35
     at MassTransit.RabbitMqTransport.SharedChannelContext.BasicPublishAsync(String exchange, String routingKey, Boolean mandatory, BasicProperties basicProperties, Byte[] body, Boolean awaitAck, CancellationToken cancellationToken) in /_/src/Transports/MassTransit.RabbitMqTransport/RabbitMqTransport/SharedChannelContext.cs:line 35
     at MassTransit.RabbitMqTransport.RabbitMqSendTransportContext.Send[T](ChannelContext transportContext, SendContext`1 sendContext) in /_/src/Transports/MassTransit.RabbitMqTransport/RabbitMqTransport/RabbitMqSendTransportContext.cs:line 168
     at MassTransit.Transports.SendTransport`1.SendPipe`1.Send(TContext context) in /_/src/MassTransit/Transports/SendTransport.cs:line 93,
     at System.Threading.SemaphoreSlim.Release(Int32 releaseCount)
     at RabbitMQ.Client.Impl.Channel.MaybeEndPublisherConfirmationTrackingAsync(PublisherConfirmationInfo publisherConfirmationInfo, CancellationToken cancellationToken)
]

[lukebakken]

Please ensure you're using version 7.2.1 of this library which should have all fixes for this issue. If you provide proof of that, I will re-open this issue.

[lukebakken]

What is that screenshot from?

[Cloutout]

I realized I'm using version 7.1.2

[lukebakken]

OK, cool, if you can upgrade and verify you don't see the issue, that would be helpful as well!