RabbitMQ 4.2.4

RabbitMQ 4.2.4 is a maintenance release in the 4.2.x release series.

It is strongly recommended that you read 4.2.0 release notes
in detail if upgrading from a version prior to 4.2.0.

Minimum Supported Erlang Version

RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Server

Bug Fixes

• Classic queue message store could run into an exception when a file was concurrently deleted by a compaction operation.

  GitHub issue: #15411

• rabbit_quorum_queue:stat/2 could fail when the quorum queue leader was undefined.

  GitHub issues: #15400, #15423

• The amq.rabbitmq.og exchange is now declared unconditionally at node boot,
  avoiding a potential deadlock between feature flag enablement and logging to the exchange sink.

  GitHub issue: #15320

MQTT Plugin

Bug Fixes

• Variable interpolation in topic permissions now.
  escapes the special regular expression characters before interpolation.

  GitHub issue: #15442

Stream Plugin

Bug Fixes

• Single Active Consumer coordinator
  could deadlock when a consumer being deactivated would lose its connection, leaving the consumer group
  without an active consumer.

  GitHub issue: #15353

Management Plugin

Bug Fixes

• The Policies page in the management UI incorrectly displayed quorum queue arguments
  twice on the declaration form in place of stream queue arguments.

  GitHub issue: #15335

• The clustering listener now correctly reports its TLS enablement status.

  GitHub issue: #15399

Enhancements

• GET /api/overview and GET /api/nodes now report crypto library (such as OpenSSL) version.

  GitHub issue: #15468

• RabbitMQ and Erlang/OTP versions are now included in GET /api/nodes responses.

  GitHub issue: #15454

• Login page usability improvements: autofocus on the username field and autocomplete attributes
  for browser password manager integration.

  GitHub issue: #15398

CLI Tools

Enhancements

• rabbitmq-upgrade has_reached_target_cluster_size is a new command that checks whether
  the cluster has reached its target size, as provided in rabbitmq.conf.

  This is useful for automating rolling upgrades, particularly on Kubernetes.

  GitHub issue: #15404

Shovel Plugin

Bug Fixes

• Stopping and deleting a shovel could fail with a badmatch exception
  when the underlying worker process no longer existed.

  GitHub issue: #15408

Trust Store Plugin

Enhancements

• HTTPS endpoint requests now have a configurable timeout (20 seconds by default),
  preventing the plugin from hanging indefinitely when a provider fails to respond.

  GitHub issues: #15308, #15310

AWS Peer Discovery Plugin

Bug Fixes

• Nodes on EC2 instances in non-running states (stopping, stopped, shutting-down, terminated)
  are now filtered out from peer discovery results.

  GitHub issue: #15388

Enhancements

• Multiple hostname resolution paths can now be configured. This is an upgrade pathway
  for scenarios where resolution hostnames have to change.

  GitHub issue: #14705

Tracing Plugin

Bug Fixes

• The plugin now uses UTF-8 encoding for traced messages.

  GitHub issue: #15439

Dependency Changes

• ra was upgraded to 2.17.2

RabbitMQ 4.2.3

RabbitMQ 4.2.3 is a maintenance release in the 4.2.x release series.

It is strongly recommended that you read 4.2.0 release notes
in detail if upgrading from a version prior to 4.2.0.

Minimum Supported Erlang Version

RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Server

Bug Fixes

• Default queue type handling is now more defensive, avoiding an issue where
  attempts to declare a queue in a virtual host without any DQT set would result
  in a PRECONDITION_FAILED exception instead of falling back to the classic queue type
  for DQT.

  GitHub issues: #11541, #12109, #12821, #13837

• Classic queue recovery on Windows could fail due to an OS-specific file locking behavior.
  Such failures are now retried in two more contexts.

  GitHub issue: #15136

• Certain exchange update operations in Khepri could cause severe contention.

  GitHub issue: #15236

• Topic exchange binding deletions could leave orphaned trie edges in the Khepri projection,
  potentially causing a slowly creeping memory leak.

  GitHub issue: #15025

• When a client that owns an exclusive queue disconnects and immediately reconnects and redeclares the same queue
  (under the same name), RabbitMQ node could delete the new queue depending on the timing of events.

  Previously the only workaround was to use exclusive server-named queues for clients that exhibited
  this behavior. Now RabbitMQ nodes take internal owner identity into account and will remove
  the original queue but not the new one.

  GitHub issue: #15276

• "Other ETS" and "Other system" memory metrics could end up being reported as negative values
  due to concurrent ETS table updates, deletion and creation on the node.

  GitHub issue: #15289

• mirrored_supervisor:child/2 could fail with a badmatch exception
  when a significant enough number of shovels (or federation links) was removed.

  GitHub issue: #15229

• Quorum queue could run into a logging exception during node drain.

  GitHub issue: #15212

Enhancements

• Topic binding deletion in Khepri is now significantly more efficient.

  Deleting 100k topic bindings could previously take up to 30 minutes and now takes
  some 3-4 seconds.

  GitHub issue: #15214

• It is now possible to rabbitmq-queues shrink only a subset of quorum queues on
  a node using a pattern (a regex).

  GitHub issue: #15021

• CRL (Certificate Revocation List) cache can now be configured via rabbitmq.conf.

  GitHub issues: #2338, #15160

Federation Plugins

Bug Fixes

• Restored exchange federation compatibility in mixed 4.2.x/4.1.x multi-node clusters.

  GitHub issue: #15252

Enhancements

• Federation links and their connections are now stopped in parallel.

  This significantly improves shutdown time for nodes with many (into thousands) federation links.

  GitHub issue: #15271

• Federation links no longer restart during plugin or node shutdown.

  For nodes with hundreds or thousands of federation links, link recovery could
  significantly delay node shutdown.

  GitHub issue: #15258

• Federation links use AMQP 0-9-1 connections to remote nodes (clusters).

  Previously the timeout used when those connections are closed was fixed to 10 seconds.
  Now it is configurable via rabbitmq.conf in milliseconds:

  # 3 seconds for exchange federation
  federation.exchanges.connection_close_timeout = 3000
  
  # 3 seconds for queue federation
  federation.queues.connection_close_timeout = 3000

  The maximum supported value is 5 seconds (5000 ms).

  GitHub issue: #15268

CLI Tools

Enhancements

• rabbitmq-streams grow and rabbitmq-streams shrink are new commands that mirror existing
  rabbitmq-queues grow and rabbitmq-queues shrink functionality for streams.

  GitHub issue: #15189

MQTT Plugin

Enhancements

• A new configuration option, mqtt.disconnect_on_unauthorized, controls whether
  MQTT connections are closed upon authorization failures (for example, an attempt to
  consume from a topic the client has no permission for).

  When set to false, the connection remains open and an appropriate protocol-level
  response is sent to the client instead.

  The default value is true, same as RabbitMQ MQTT implementation's historic behavior.

  GitHub issue: #15201

Management Plugin

Bug Fixes

• OAuth 2: preferred_auth_mechanism and strict_auth_mechanism are no longer validated
  when not used (configured).

  GitHub issue: #15148

etcd Peer Discovery Plugin

Bug Fixes

• When credentials were provided, the password was double-encrypted, resulting in
  etcd authentication failures.

  GitHub issue: #15191

HTTP Auth Backend Plugin

Bug Fixes

• The customize_hostname_check TLS option was unintentionally ignored.

  GitHub issue: #15184

Dependency Changes

• khepri was upgraded to 0.17.4
• khepri_mnesia_migration was upgraded to 0.8.1
• osiris was upgraded to 1.10.3

RabbitMQ 4.1.8

RabbitMQ 4.1.8 is a maintenance release in the 4.1.x release series.

It is strongly recommended that you read 4.1.0 release notes
in detail if upgrading from a version prior to 4.1.0.

Minimum Supported Erlang Version

RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Server

Bug Fixes

• Classic queue recovery on Windows could fail due to an OS-specific file locking behavior.
  Such failures are now retried in two more contexts.

  GitHub issue: #15136

• Default queue type handling is now more defensive, avoiding an issue where
  attempts to declare a queue in a virtual host without any DQT set would result
  in a PRECONDITION_FAILED exception instead of falling back to the classic queue type
  for DQT.

GitHub issues: #11541, #12109, #12821, #13837

• Topic exchange binding deletions could leave orphaned trie edges in the Khepri projection,
  potentially causing a slowly creeping memory leak.

  GitHub issue: #15025

• When a client that owns an exclusive queue disconnects and immediately reconnects and redeclares the same queue
  (under the same name), RabbitMQ node could delete the new queue depending on the timing of events.

  Previously the only workaround was to use exclusive server-named queues for clients that exhibited
  this behavior. Now RabbitMQ nodes take internal owner identity into account and will remove
  the original queue but not the new one.

  GitHub issue: #15276

• "Other ETS" and "Other system" memory metrics could end up being reported as negative values
  due to concurrent ETS table updates, deletion and creation on the node.

  GitHub issue: #15289

• mirrored_supervisor:child/2 could fail with a badmatch exception
  when a significant enough number of shovels (or federation links) was removed.

  GitHub issue: #15229

Federation Plugins

Bug Fixes

• Restored exchange federation compatibility in mixed 4.2.x/4.1.x multi-node clusters.

  GitHub issue: #15252

Enhancements

• Federation links and their connections are now stopped in parallel.

  This significantly improves shutdown time for nodes with many (into thousands) federation links.

  GitHub issue: #15271

• Federation links no longer restart during plugin or node shutdown.

  For nodes with hundreds or thousands of federation links, link recovery could
  significantly delay node shutdown.

  GitHub issue: #15258

• Federation links use AMQP 0-9-1 connections to remote nodes (clusters).

  Previously the timeout used when those connections are closed was fixed to 10 seconds.
  Now it is configurable via rabbitmq.conf in milliseconds:

  # 3 seconds for exchange federation
  federation.exchanges.connection_close_timeout = 3000
  
  # 3 seconds for queue federation
  federation.queues.connection_close_timeout = 3000

  The maximum supported value is 5 seconds (5000 ms).

  GitHub issue: #15268

MQTT Plugin

Enhancements

• A new configuration option, mqtt.disconnect_on_unauthorized, controls whether
  MQTT connections are closed upon authorization failures (for example, an attempt to
  consume from a topic the client has no permission for).

  When set to false, the connection remains open and an appropriate protocol-level
  response is sent to the client instead.

  The default value is true, same as RabbitMQ MQTT implementation's historic behavior.

  GitHub issue: #15201

RabbitMQ 4.2.2

RabbitMQ 4.2.2 is a maintenance release in the 4.2.x release series.

It is strongly recommended that you read 4.2.0 release notes
in detail if upgrading from a version prior to 4.2.0.

Minimum Supported Erlang Version

RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Server

Bug Fixes

• Fixes a binding deletion-related memory leak in Khepri.

  This leak primarily affected systems with high binding churn.

  GitHub issue: #15024

• Corrected a performance regression around metric collection that could have a meaningful impact on quorum queues.

  GitHub issue: rabbitmq/seshat#16

• AMQP 1.0 link error frames did not have settlement set in certain failure scenarios,
  namely when attaching a link failed due to a missing source (queue or stream).

  GitHub issue: #15004

• AMQP 1.0 clients that publish messages without a body now get a clearer error message.
  Messages without a body explicitly violate the AMQP 1.0 specification.

  GitHub issue: #15048

• Default operator policy pre-configuration in rabbitmq.conf were unintentionally missing a key,
  default_policies.operator.$id.apply_to, for controlling what objects the policy applies to.

  GitHub issue: #14988

• Definition import is now more defensive when importing exchanges.

  GitHub issue: #15128

Enhancements

• More efficient Khepri (metadata store) querying.

  GitHub issue: rabbitmq/khepri#349

• Removed a cool down delay between stream chunk redelivery attempts.

  GitHub issue: #14766

• Stream data directory can now be configured via rabbitmq.conf.

  GitHub issue: #15014

• Stream replication: IP address family (IPv4 or IPv6) can now be configured via rabbitmq.conf.

  GitHub issue: #15012

OAuth 2 Plugin

Bug Fixes

• When multiple resource server IDs are configured, management UI login screen now provides a way to pre-select a specific resource ID
  before logging in.

  As part of this change, when multiple resource server IDs are configured, their order stored internally now reflects
  that in rabbitmq.conf.

  GitHub issue: #15044

Management Plugin

Bug Fixes

• GET /api/deprecated-features/used no longer returns an empty state field.

  GitHub issue: #14340

• Feature flag icons now contrast well when the dark mode is used in the UI.

  GitHub issue: #14620

• Corrected an invalid apply_to value (classic_queue instead of classic_queues)
  on the policy declaration form.

  Declaring a policy "directly" via the HTTP API was not affected.

  GitHub issue: #15054

Enhancements

• Queue declaration form fields were reordered so that the queue type dropdown comes before
  the virtual host.

  GitHub issue: #15017

• The order of configured OAuth 2 resource servers is now preserved in the management UI.
  Additionally, a resource server ID can now be pre-selected during service-provider-initiated logons.

  GitHub issue: #15044

Prometheus Plugin

Enhancements

• GET /metrics/per-object responses on systems with thousands of objections now takes 20-30% less time.

  GitHub issue: #15035

• Prometheus scraping endpoint responses are now streamed instead of being fully buffered before transmission.
  This reduces memory peak footprint and improves response times for nodes with a larger number of objects.

  GitHub issue: #15018

Bug Fixes

• Fixed a dependency loading issue with ddskerl (a Prometheus library dependency)
  that could cause stop_app to fail.

  GitHub issue: #15027

CLI Tools

Bug Fixes

• rabbitmq-queues quorum_status could run into an exception when a quorum queue member dies before recording any metrics.

  GitHub issue: #15003

Web MQTT Plugin

Bug Fixes

• The plugin now supports the mqttv3.1 WebSocket subprotocol in addition to mqtt,
  improving compatibility with older MQTT clients.

  GitHub issue: #15009

AWS Peer Discovery Plugin

Bug Fixes

• The plugin now uses IPv6 discovery endpoints in IPv6-only environments.

  GitHub issue: #14974

Tracing Plugin

Bug Fixes

• Trace files served via the management UI now include proper UTF-8 charset specification.

  GitHub issue: #14970

Trust Store Plugin

Bug Fixes

• The plugin now merges user-provided TLS options with the options it needs in a safer manner.

  GitHub issue: #15116

Dependency Changes

• khepri was upgraded to 0.17.3
• seshat was upgraded to 1.0.1

RabbitMQ 4.1.7

RabbitMQ 4.1.7 is a maintenance release in the 4.1.x release series.

It is strongly recommended that you read 4.1.0 release notes
in detail if upgrading from a version prior to 4.1.0.

Minimum Supported Erlang Version

RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Server

Enhancements

• Stream replication: IP address family (IPv4 or IPv6) can now be configured via rabbitmq.conf.

  GitHub issue: #15013

Management Plugin

Bug Fixes

• GET /api/deprecated-features/used no longer returns an empty state field.

  GitHub issue: #15059

• Feature flag icons now contrast well when the dark mode is used in the UI.

  GitHub issue: #15070

Web MQTT Plugin

Bug Fixes

• The plugin now supports the mqttv3.1 WebSocket subprotocol in addition to mqtt,
  improving compatibility with older MQTT clients.

  GitHub issue: #15010

AWS Peer Discovery Plugin

Bug Fixes

• The plugin now uses IPv6 discovery endpoints in IPv6-only environments.

  GitHub issue: #14987

Tracing Plugin

Bug Fixes

• Trace files served via the management UI now include proper UTF-8 charset specification.

  GitHub issue: #14972

RabbitMQ 4.2.1

RabbitMQ 4.2.1 is a maintenance release in the 4.2.x release series.

It is strongly recommended that you read 4.2.0 release notes
in detail if upgrading from a version prior to 4.2.0.

Minimum Supported Erlang Version

RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Server

Bug Fixes

• Quorum queue at-most-once dead lettering for the overflow behaviour drop-head now happens in the correct order.

  GitHub issue: #14926

• Feature flag state in the registry and on disk were not consistent for a period of time during node boot.

  GitHub issue: #14943

Stream Plugin

Bug Fixes

• stream.read_ahead is a new setting that controls how much data is prefetched from disk
  for stream reads (consumption).

  GitHub issue: #14948

• Stream deletion is now more resilient and can handle certain mid-deletion failure scenarios.

  GitHub issue: #14852

Grafana Dashboards

Enhancements

• The dashboards were updated for RabbitMQ 4.2.x.

  GitHub issue: #14849

Management Plugin

Enhancements

• GET /api/queues/{vhost} requests no longer perform unnecessary virtual host permission checks
  and log less (at debug level) as a result.

  GitHub issue: #14923

Shovel Plugin

Bug Fixes

• Improved target node resource alarm handling for AMQP 1.0 and local shovels.

  GitHub issue: #14886

• Local shovels could run into an exception that would cause a shovel restart.

  GitHub issue: #14872

• AMQP 1.0 shovels ignored the sasl URI parameter.

  GitHub issue: #14867

OAuth 2 Plugin

Bug Fixes

• A usability improvement allows the plugin to automatically load the trusted system x.509 (TLS) certificates.

  GitHub issue: #14927

LDAP Plugin

Bug Fixes

• A usability improvement allows the plugin to automatically load the trusted system certificates
  when the user only enables TLS for the LDAP client but does not configure any other settings.

  GitHub issue: #14937

Dependency Changes

• cuttlefish was upgraded to 3.6.0

RabbitMQ 4.1.6

RabbitMQ 4.1.6 is a maintenance release in the 4.1.x release series.

It is strongly recommended that you read 4.1.0 release notes
in detail if upgrading from a version prior to 4.1.0.

Minimum Supported Erlang Version

RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Server

Bug Fixes

• Enabling the khepri_db feature flag while the Log Exchange
  was enabled could cause a RabbitMQ node to run out of memory and crash.

  GitHub issues: #14069, #14796

• Feature flag state in the registry and on disk were not consistent for a period of time during node boot.

  GitHub issue: #14943

CLI Tools

Bug Fixes

• rabbitmqctl export_definitions could incorrectly serialize policy and operator policy
  definitions.

  GitHub issue: #14800

MQTT Plugin

Bug Fixes

• Resource alarm handling now uses more context: it is aware of individual resources.
  When a cluster had multiple resource alarms (namely for memory footprint and free disk space)
  in effect, the blocking state was prematurely cleared when only one resource alarm was.

  GitHub issue: #14795

STOMP Plugin

Bug Fixes

• Resource alarm handling now uses more context: it is aware of individual resources.
  When a cluster had multiple resource alarms (namely for memory footprint and free disk space)
  in effect, the blocking state was prematurely cleared when only one resource was [cleared].

  GitHub issue: #14795

Web MQTT Plugin

Bug Fixes

• Resource alarm handling now uses more context: it is aware of individual resources.
  When a cluster had multiple resource alarms (namely for memory footprint and free disk space)
  in effect, the blocking state was prematurely cleared when only one resource was [cleared].

  GitHub issue: #14795

RabbitMQ 4.2.0

RabbitMQ 4.2.0 is a new feature release.

Breaking Changes and Compatibility Notes

Default value for AMQP 1.0 durable field.

Starting with RabbitMQ 4.2, if a sending client omits the header section, RabbitMQ assumes the durable field to be false complying with the AMQP 1.0 spec:

<field name="durable" type="boolean" default="false"/>

AMQP 1.0 apps or client libraries must set the durable field of the header section to true to mark the message as durable.

Team RabbitMQ recommends client libraries to send messages as durable by default.
All AMQP 1.0 client libraries maintained by Team RabbitMQ send messages as durable by default.

Mandatory flag in Direct Reply-To

Starting with RabbitMQ 4.2, if an AMQP 0.9.1 Direct Reply-To responder (RPC server) publishes with the mandatory flag set, then amq.rabbitmq.reply-to.* is treated as a queue.
Whether the requester (RPC client) is still there to consume the reply is not checked at routing time.
In other words, if the responder publishes to only this queue name, then the message will be considered "routed" and RabbitMQ will therefore not send a basic.return.

Very Rarely Used *.cacerts Settings are Removed from rabbitmq.conf

*.cacerts (not to be confused with cacertfile) settings in rabbitmq.conf did not have the expected effect and were removed
to eliminate confusion.

Quorum Queue Metric Changes

Metrics emitted for Ra-based components (quorum queues, Khepri, Stream Coordinator)
have changed. Some metrics were removed, many were added, some changed their names.
Users relying on Prometheus metrics starting with rabbitmq_raft or rabbitmq_detailed_raft
will need to update their dashboards and/or alerts. If you are using the
RabbitMQ-Quorum-Queues-Raft dashboard,
please update it to the latest version for RabbitMQ 4.2 compatibility.

Release Highlights

SQL Filter Expression for Streams

AMQP 1.0 clients can now define SQL-like filter expressions when consuming from streams, enabling server-side message filtering.
RabbitMQ will only dispatch messages that match the provided filter expression, reducing network traffic and client-side processing overhead.
SQL filter expressions are a more powerful alternative to the AMQP Property Filter Expressions introduced in RabbitMQ 4.1.

RabbitMQ implements a subset of AMQP Filter Expressions Version 1.0 Committee Specification Draft 01 Section 6 including support for:

• Comparison operators (=, !=, <>, >, <, >=, <=)
• Logical operators (AND, OR, NOT)
• Arithmetic operators (+, -, *, /, %)
• Special operators (LIKE, IN, IS NULL)
• UTC function
• Access to the properties and application-properties sections

Examples

Simple expression:

header.priority > 4

Complex expression:

order_type IN ('premium', 'express') AND
(customer_region LIKE 'EU-%' OR customer_region = 'US-CA') AND
UTC() < properties.absolute-expiry-time AND
NOT cancelled

To learn more, check out the new documentation guide on Stream Filtering.

Pull Request: #14184

Direct Reply-To for AMQP 1.0

RabbitMQ 4.2 adds Direct Reply-To support for AMQP 1.0, alongside the existing AMQP 0.9.1 implementation.
It also works across protocols (e.g., AMQP 1.0 requester with AMQP 0.9.1 responder, or vice versa).

For more information, read our updated documentation on Direct Reply-To.

Pull Request: #14474

New Tooling for More Automated Blue-Green Deployment Migrations from 3.13.x Clusters to 4.2.x

Blue-Green Deployment migration from RabbitMQ 3.13.x
to 4.2.0 is now easier to automate thanks to a new set of commands provided by rabbitmqadmin v2.

Incoming and Outgoing Message Interceptors for Native Protocols (AMQP 1.0, AMQP 0-9-1, MQTTv3, MQTTv5)

Incoming and outgoing messages can now be intercepted on the broker.
This works for AMQP 1.0, AMQP 0.9.1, MQTTv3, and MQTTv5.

What the interceptor does is entirely up to its implementation, for example it can validate message metadata, add annotations, or perform arbitrary side effects.
Custom interceptors can be developed and integrated via plugins.

Two new optional built-in interceptors were added to RabbitMQ:

1. Timestamps for outgoing messages
2. Setting client ID of publishing MQTT client

Detailed information can be found in the Message Interceptor documentation.

Khepri Enabled by Default for New Clusters

RabbitMQ supports two databases to store
metadata such as virtual hosts,
topology, runtime parameters, policies, internal users and so on: Mnesia and
Khepri. That metadata store is also at the heart of clustering in RabbitMQ. As
of RabbitMQ 4.2.0, Khepri is the default metadata store for new
deployments.

Khepri is based on the
same Raft consensus algorithm used by quorum queues and streams. The goal is to
have a consistent well defined behaviour around all queries and updates of
metadata across an entire cluster, especially when the cluster suffers
increased latency or network issues for instance. It also comes with increased
performance in several use cases, even though this was not a goal.

A new RabbitMQ 4.2.0+ node will use Khepri by default. If you upgrade an
existing node or cluster, it will continue to use whatever metadata store it
was using so far.

If you did not enable Khepri yet, it is recommended that you enable it:

rabbitmqctl enable_feature_flag khepri_db

Khepri will become mandatory in a future minor version. Mnesia support will be
dropped in a future major version. These exact versions are to be decided.

Local Shovels

In addition to AMQP 0-9-1 and AMQP 1.0, Shovels
now support a new "protocol" option called local.

These specialized shovels are internally based on AMQP 1.0 but instead of
separate TCP connections, use the intra-cluster connections
between cluster nodes and the internal API for consumption, publishing
and AMQP 1.0 credit flow.

Such shovels can only be used for consuming and publishing
within the same cluster, not across clusters, but can offer
higher throughput and use fewer resources per connections
than their AMQP 0-9-1 and AMQP 1.0 counterparts.

Upgrading to 4.2.0

Documentation guides on upgrades

See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.

This release series supports upgrades from 4.1.x, 4.0.x and 3.13.x.

If upgrading from a 3.13.x cluster that uses classic mirrored queues,
take a look at what modern CLI tools can offer for such migrations away from classic mirrored queues
via Blue/Green deployments.

Blue/Green Deployment-style upgrades are available for migrations
from RabbitMQ 3.12.x series.

New Required Feature Flags

None. The required feature flag set is the same as in 4.1.x and 4.0.x.

Mixed version cluster compatibility

RabbitMQ 4.2.0 nodes can run alongside 4.1.x and 4.0.x nodes. 4.2.x-specific features can only be made available when all nodes in the cluster upgrade to 4.2.0 or a later patch release in the new series.

While operating in mixed version mode, some aspects of the system may not behave as expected.
Once all nodes are upgraded to 4.1.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).

Recommended Post-upgrade Procedures

This version does not require any additional post-upgrade procedures
compared to other versions.

Changes Worth Mentioning

Core Server

Enhancements

• In clusters with a larger number of quorum queues (say, tens of thousands),
  quorum queue leadership transfer is now performed gradually and not all at once.

  Previously tens of thousands of concurrent leader elections
  could result in timeouts and some quorum queues ending up
  without an elected leader.

  GitHub issue: #14401

• Schema data store (Khepri) read concurrency optimizations that can lead to low doublt digit percent
  throughput gains on nodes with larger numbers of cores.

  GitHub issue: #14530

• Two new rabbitmq.conf, settings log.summarize_process_state and log.error_logger_format_depth, can be used
  to significantly reduce the amount of queue member (replica) state logged in case of an abnormal termination.

...

RabbitMQ 4.1.5

RabbitMQ 4.1.5 is a maintenance release in the 4.1.x release series.

It is strongly recommended that you read 4.1.0 release notes
in detail if upgrading from a version prior to 4.1.0.

Minimum Supported Erlang Version

RabbitMQ and Erlang/OTP Compatibility Matrix has more details on Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Server

Bug Fixes

• Classic queues could run into a rare message store exception that resuulted in
  a loss of a few messages.

  Special kudos to the contributors who have spent a very significant amount of time
  reproducing and debugging the issue: @lhoguin @lukebakken @trvrnrth @gomoripeti

  GitHub issues: #14181, #14576

• Messages routed to quorum queues during or immediately before a network partition,
  in some cases were not re-republished internally in certain cases.

  GitHub issue: #14589

• Quorum queues with disabled poison message handling
  (an unlimited number of redeliveries, which is not a recommended practice) could accumulate
  a significant number of Raft log segment files.

  GitHub issues: #14202, #14458

Streams

Bug Fixes

• Streams declaration response was missing a leader field when the stream was declared via AMQP 1.0 or 0.9.1
  (that is, not the RabbitMQ Stream Protocol).

  GitHub issue: #14476

CLI Tools

Bug Fixes

• Fixes JSON formatting of cluster nodes produced by rabbitmq-diagnostics status --formatter json

  GitHub issue: #14509

Shovel Prometheus Plugin

Bug Fixes

• Requesting shovel status via Prometheus at a specific moment resulted in an exception
  because the plugin did not consider a highly transient starting state.

  GitHub issue: #14481

RabbitMQ 4.2.0-rc.1

RabbitMQ 4.2.0-rc.1 is a feature release candidate.

Breaking Changes and Compatibility Notes

Default value for AMQP 1.0 durable field.

Starting with RabbitMQ 4.2, if a sending client omits the header section, RabbitMQ assumes the durable field to be false complying with the AMQP 1.0 spec:

<field name="durable" type="boolean" default="false"/>

AMQP 1.0 apps or client libraries must set the durable field of the header section to true to mark the message as durable.

Team RabbitMQ recommends client libraries to send messages as durable by default.
All AMQP 1.0 client libraries maintained by Team RabbitMQ send messages as durable by default.

Mandatory flag in Direct Reply-To

Starting with RabbitMQ 4.2, if an AMQP 0.9.1 Direct Reply-To responder (RPC server) publishes with the mandatory flag set, then amq.rabbitmq.reply-to.* is treated as a queue.
Whether the requester (RPC client) is still there to consume the reply is not checked at routing time.
In other words, if the responder publishes to only this queue name, then the message will be considered "routed" and RabbitMQ will therefore not send a basic.return.

Very Rarely Used *.cacerts Settings are Removed from rabbitmq.conf

*.cacerts (not to be confused with cacertfile) settings in rabbitmq.conf did not have the expected effect and were removed
to eliminate confusion.

Release Highlights

SQL Filter Expression for Streams

AMQP 1.0 clients can now define SQL-like filter expressions when consuming from streams, enabling server-side message filtering.
RabbitMQ will only dispatch messages that match the provided filter expression, reducing network traffic and client-side processing overhead.
SQL filter expressions are a more powerful alternative to the AMQP Property Filter Expressions introduced in RabbitMQ 4.1.

RabbitMQ implements a subset of AMQP Filter Expressions Version 1.0 Committee Specification Draft 01 Section 6 including support for:

• Comparison operators (=, !=, <>, >, <, >=, <=)
• Logical operators (AND, OR, NOT)
• Arithmetic operators (+, -, *, /, %)
• Special operators (LIKE, IN, IS NULL)
• UTC function
• Access to the properties and application-properties sections

Examples

Simple expression:

header.priority > 4

Complex expression:

order_type IN ('premium', 'express') AND
(customer_region LIKE 'EU-%' OR customer_region = 'US-CA') AND
UTC() < properties.absolute-expiry-time AND
NOT cancelled

To learn more, check out the new documentation guide on Stream Filtering.

Pull Request: #14184

Direct Reply-To for AMQP 1.0

RabbitMQ 4.2 adds Direct Reply-To support for AMQP 1.0, alongside the existing AMQP 0.9.1 implementation.
It also works across protocols (e.g., AMQP 1.0 requester with AMQP 0.9.1 responder, or vice versa).

For more information, read our updated documentation on Direct Reply-To.

Pull Request: #14474

New Tooling for More Automated Blue-Green Deployment Migrations from 3.13.x Clusters to 4.2.x

Blue-Green Deployment migration from RabbitMQ 3.13.x
to 4.2.0 is now easier to automate thanks to a new set of commands provided by rabbitmqadmin v2.

Incoming and Outgoing Message Interceptors for Native Protocols (AMQP 1.0, AMQP 0-9-1, MQTTv3, MQTTv5)

Incoming and outgoing messages can now be intercepted on the broker.
This works for AMQP 1.0, AMQP 0.9.1, MQTTv3, and MQTTv5.

What the interceptor does is entirely up to its implementation, for example it can validate message metadata, add annotations, or perform arbitrary side effects.
Custom interceptors can be developed and integrated via plugins.

Two new optional built-in interceptors were added to RabbitMQ:

1. Timestamps for outgoing messages
2. Setting client ID of publishing MQTT client

Detailed information can be found in the Message Interceptor documentation.

Khepri Enabled by Default for New Clusters

RabbitMQ supports two databases to store
metadata such as virtual hosts,
topology, runtime parameters, policies, internal users and so on: Mnesia and
Khepri. That metadata store is also at the heart of clustering in RabbitMQ. As
of RabbitMQ 4.2.0, Khepri is the default metadata store for new
deployments.

Khepri is based on the
same Raft consensus algorithm used by quorum queues and streams. The goal is to
have a consistent well defined behaviour around all queries and updates of
metadata across an entire cluster, especially when the cluster suffers
increased latency or network issues for instance. It also comes with increased
performance in several use cases, even though this was not a goal.

A new RabbitMQ 4.2.0+ node will use Khepri by default. If you upgrade an
existing node or cluster, it will continue to use whatever metadata store it
was using so far.

If you did not enable Khepri yet, it is recommended that you enable it:

rabbitmqctl enable_feature_flag khepri_db

Khepri will become mandatory in a future minor version. Mnesia support will be
dropped in a future major version. These exact versions are to be decided.

Local Shovels

In addition to AMQP 0-9-1 and AMQP 1.0, Shovels
now support a new "protocol" option called local.

These specialized shovels are internally based on AMQP 1.0 but instead of
separate TCP connections, use the intra-cluster connections
between cluster nodes and the internal API for consumption, publishing
and AMQP 1.0 credit flow.

Such shovels can only be used for consuming and publishing
within the same cluster, not across clusters, but can offer
higher throughput and use fewer resources per connections
than their AMQP 0-9-1 and AMQP 1.0 counterparts.

Upgrading to 4.2.0

Documentation guides on upgrades

See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.

This release series supports upgrades from 4.1.x, 4.0.x and 3.13.x.

If upgrading from a 3.13.x cluster that uses classic mirrored queues,
take a look at what modern CLI tools can offer for such migrations away from classic mirrored queues
via Blue/Green deployments.

Blue/Green Deployment-style upgrades are available for migrations
from RabbitMQ 3.12.x series.

New Required Feature Flags

None. The required feature flag set is the same as in 4.1.x and 4.0.x.

Mixed version cluster compatibility

RabbitMQ 4.2.0 nodes can run alongside 4.1.x and 4.0.x nodes. 4.2.x-specific features can only be made available when all nodes in the cluster upgrade to 4.2.0 or a later patch release in the new series.

While operating in mixed version mode, some aspects of the system may not behave as expected.
Once all nodes are upgraded to 4.1.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).

Recommended Post-upgrade Procedures

This version does not require any additional post-upgrade procedures
compared to other versions.

Changes Worth Mentioning

Core Server

Enhancements

• In clusters with a larger number of quorum queues (say, tens of thousands),
  quorum queue leadership transfer is now performed gradually and not all at once.

  Previously tens of thousands of concurrent leader elections
  could result in timeouts and some quorum queues ending up
  without an elected leader.

  GitHub issue: #14401

• Schema data store (Khepri) read concurrency optimizations that can lead to low doublt digit percent
  throughput gains on nodes with larger numbers of cores.

  GitHub issue: #14530

• Two new rabbitmq.conf, settings log.summarize_process_state and log.error_logger_format_depth, can be used
  to significantly reduce the amount of queue member (replica) state logged in case of an abnormal termination.

  Limiting logging helps avoid memory allocation spikes.

  GitHub issues: #14349, #14523

• When a configured authentication or authorization backend comes from a known
  plugin but the plugin is not enabled, the node will now refuse to start.

  Previously the node would boot but client connections would fail because
  of the missing backend modules.

...