Why does IAmACommandProcessor.PostAsync go through Outbox pipeline?

[jekingry]

I thought that PostAsync (and Post) should be for posting events to the queue where you don't need transactional behavior, and therefore any kind of Outbox support. However I've found, surprisingly that the Outbox is still involved, including if you are using the default in-memory outbox, the default EntryLimit and EntryTimeToLive.

This has led to the exception of Message with Id 4d74d624-c36e-4433-91e9-79df3424c11b not found in the Outbox when the producer system is producing a lot of messages.

[lillo42]

Brighter uses an outbox pattern under the hood during Post or PostAsync to guarantee message delivery. Since brokers like RabbitMQ and Kafka are inherently asynchronous (even when using their synchronous clients), a confirmation callback is required to verify the broker has successfully received the message.

[iancooper]

To supplement @lillo42's answer. You should run a sweeper as a background service to clear the in-memory outbox if you are not using a persistent outbox. This is because you may not see transient errors that led to your message not being sent when using middleware like RabbitMQ or Kafka, which Brighter can address for you with a retry via the Sweeper. Obviously, an in-memory outbox will not survive restarts of the application, but message loss is a risk for you, you should be using a persistent outbox, as in-memory cannoy survive an application restart

[iancooper]

In addition, we have an issue to add a fix in V10, to allow you greater control over the amount of compaction we do when removing older messages from an in-memory outbox. You should, however, set the capacity of the in-memory outbox sufficiently high to deal with a high throughput of messages. If you are receiving this message, there has likely been aggressive cache clearing due to throughput