Release v2.12.4-RC.1

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Go Version

• 1.25.5

Dependencies

• github.com/google/go-tpm v0.9.8 (#7696)
• github.com/nats-io/nats.go v1.48.0 (#7696)

Improved

JetStream

• Creating consumer file stores no longer contends on the stream lock, improving consumer create performance on heavily loaded streams (#7700)

Fixed

JetStream

• Raft single node elections now transition into leader state correctly (#7642)
• R1 streams will no longer incorrectly drift last sequence when exceeding limits (#7658)
• Deleted streams are no longer wrongfully revived if stalled on an upper-layer catchup (#7668)
• A panic that could happen when receiving a shutdown signal while JetStream is still starting up has been fixed (#7683)
• JetStream usage stats now correctly reflect purged whole blocks when optimising large purges (#7685)
• Recovering JetStream encryption keys now happens independently of the stream index recovery, fixing some cases where the key could be reset unexpectedly if the index is rebuilt (#7678)
• Non-replicated file-based consumers now detect corrupted state on disk and are deleted automatically (#7691)
• A panic that could occur when a compact and a hash check run concurrently on a store has been fixed (#7697)
• Raft no longer allows a repeat vote for the same term after a stepdown or leadership transfer (#7698)
• Replicated consumers are no longer incorrectly deleted if they become leader just as JetStream is about to shut down (#7699)

Complete Changes

v2.12.3...v2.12.4-RC.1

Release v2.11.12-RC.2

Changelog

Refer to the 2.11 Upgrade Guide for backwards compatibility notes with 2.10.x.

Go Version

• 1.25.5

Dependencies

• github.com/google/go-tpm v0.9.8 (#7696)
• github.com/nats-io/nats.go v1.48.0 (#7696)

Improved

JetStream

• Creating consumer file stores no longer contends on the stream lock, improving consumer create performance on heavily loaded streams (#7700)

Fixed

JetStream

• Raft will no longer allow multiple membership changes to take place concurrently (#7565, #7609)
• Raft will no longer count responses from peer-removed nodes towards quorum (#7589)
• Raft quorum counting has been refactored so the implicit leader ack is now only counted if still a part of the membership (#7600)
• Raft now writes the peer state immediately when handling a peer-remove to ensure the removed peers cannot unexpectedly reappear after a restart (#7602)
• Raft will no longer allow peer-removing the one remaining peer (#7610)
• Add peer operations to Raft can no longer result in disjoint majorities (#7632)
• Raft groups should no longer readmit a previously removed peer if a heartbeat occurs between the peer removal and the leadership transfer (#7649)
• Raft single node elections now transition into leader state correctly (#7642)
• R1 streams will no longer incorrectly drift last sequence when exceeding limits (#7658)
• Deleted streams are no longer wrongfully revived if stalled on an upper-layer catchup (#7668)
• A panic that could happen when receiving a shutdown signal while JetStream is still starting up has been fixed (#7683)
• JetStream usage stats now correctly reflect purged whole blocks when optimising large purges (#7685)
• Recovering JetStream encryption keys now happens independently of the stream index recovery, fixing some cases where the key could be reset unexpectedly if the index is rebuilt (#7678)
• Non-replicated file-based consumers now detect corrupted state on disk and are deleted automatically (#7691)
• Raft no longer allows a repeat vote for the same term after a stepdown or leadership transfer (#7698)
• Replicated consumers are no longer incorrectly deleted if they become leader just as JetStream is about to shut down (#7699)

Complete Changes

v2.11.12-RC.1...v2.11.12-RC.2

Release v2.11.12-RC.1

Changelog

Refer to the 2.11 Upgrade Guide for backwards compatibility notes with 2.10.x.

Go Version

• 1.25.5 (#7604)

Dependencies

• github.com/google/go-tpm v0.9.7 (#7578)
• github.com/nats-io/nkeys v0.4.12 (#7578)
• golang.org/x/crypto v0.45.0 (#7578)
• github.com/klauspost/compress v1.18.2 (#7604)
• github.com/antithesishq/antithesis-sdk-go v0.5.0-default-no-op (#7604)
• golang.org/x/crypto v0.46.0 (#7648)
• golang.org/x/sys v0.39.0 (#7648)

Added

General

• Added WebSocket-specific ping interval configuration with ping_internal in the websocket block (#7614)

Improved

JetStream

• The scan for the last sourced message sequence when setting up a subject-filtered source is now considerably faster (#7553)
• Consumer interest checks on interest-based streams are now significantly faster when there are large gaps in interest (#7656)

MQTT

• Retained messages will now work correctly even when sourced from a different account and has a subject transform (#7636)

Fixed

General

• WebSocket connections will now correctly limit the buffer size during decompression (#7625, thanks to Pavel Kokout at Aisle Research)

JetStream

• A protocol error caused by an invalid transform of acknowledgement reply subjects when originating from a gateway connection has been fixed (#7579)
• The meta layer will now only respond to peer remove requests after quorum has been reached (#7581)
• Invalid subject filters containing non-terminating full wildcard no longer produce unexpected matches (#7585)
• A data race when creating a stream in clustered mode has been fixed (#7586)
• A panic when processing snapshots with missing nodes or assignments has been fixed (#7588)
• When purging whole message blocks, the subject tracking and scheduled messages are now updated correctly (#7593)
• The filestore will no longer unexpectedly lose writes when AsyncFlush is enabled after a process pause (#7594)
• The filestore now will process message removal on disk before updating accounting, which improves error handling (#7595, #7601)
• Raft will no longer allow peer-removing the one remaining peer (#7610)
• A data race has been fixed in the stream health check (#7619)
• Tombstones are now correctly written for recovering the sequences after compacting or purging an almost-empty stream to seq 2 (#7627)
• Combining skip sequences and compactions will no longer overwrite the block at the wrong offset, correcting a corrupt record state error (#7627)
• Compactions that reclaim over half of the available space now use an atomic write to avoid losing messages if killed (#7627)
• Filestore compaction should no longer result in no idx present cache errors (#7634)
• Filestore compaction now correctly adjusts the high and low sequences for a message block, as well as cleaning up the deletion map accordingly (#7634)
• Potential stream desyncs that could happen during stream snapshotting have been fixed (#7655)

MQTT

• The maximum payload size is now correctly enforced for MQTT clients (#7555, thanks to @yixianOu)
• Fixed a panic that could occur when reloading config if the user did not have permission to access retained messages (#7596)
• Fixed account mapping for JetStream API requests when traversing non-JetStream-enabled servers (#7598)
• QoS0 messages are now mapped correctly across account imports/exports with subject mappings (#7605)
• Loading retained messages no longer fails after restarting due to last sequence checks (#7616)
• A bug which could corrupt retained messages in clustered deployments has been fixed (#7622)
• Permissions to $MQTT. subscriptions are now handled implicitly, with the exception of deny ACLs which still permit restriction (#7637)
• A bug where QoS2 messages could not be retrieved after a server restart has been fixed (#7643)

Complete Changes

v2.11.11...v2.11.12-RC.1

Release v2.12.3

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Go Version

• 1.25.5 (#7604)

Dependencies

• github.com/google/go-tpm v0.9.7 (#7578)
• github.com/nats-io/nkeys v0.4.12 (#7578)
• golang.org/x/crypto v0.45.0 (#7578)
• github.com/klauspost/compress v1.18.2 (#7604)
• github.com/antithesishq/antithesis-sdk-go v0.5.0-default-no-op (#7604)
• golang.org/x/crypto v0.46.0 (#7648)
• golang.org/x/sys v0.39.0 (#7648)

Added

General

• Added WebSocket-specific ping interval configuration with ping_internal in the websocket block (#7614)

Improved

JetStream

• The scan for the last sourced message sequence when setting up a subject-filtered source is now considerably faster (#7553)
• The metalayer will now stage and deduplicate recovery operations at startup, instead of rapidly applying and then undoing conflicting assignments (#7540)
• Consumer interest checks on interest-based streams are now significantly faster when there are large gaps in interest (#7656)

MQTT

• Retained messages will now work correctly even when sourced from a different account and has a subject transform (#7636)

Fixed

General

• WebSocket connections will now correctly limit the buffer size during decompression (#7625, thanks to Pavel Kokout at Aisle Research)

JetStream

• A protocol error caused by an invalid transform of acknowledgement reply subjects when originating from a gateway connection has been fixed (#7579)
• The meta layer will now only respond to peer remove requests after quorum has been reached (#7581)
• Invalid subject filters containing non-terminating full wildcard no longer produce unexpected matches (#7585)
• A data race when creating a stream in clustered mode has been fixed (#7586)
• Raft will no longer allow multiple membership changes to take place concurrently (#7565, #7609)
• A panic when processing snapshots with missing nodes or assignments has been fixed (#7588)
• When purging whole message blocks, the subject tracking and scheduled messages are now updated correctly (#7593)
• Raft will no longer count responses from peer-removed nodes towards quorum (#7589)
• The filestore will no longer unexpectedly lose writes when AsyncFlush is enabled after a process pause (#7594)
• The filestore now will process message removal on disk before updating accounting, which improves error handling (#7595, #7601)
• Raft quorum counting has been refactored so the implicit leader ack is now only counted if still a part of the membership (#7600)
• Raft now writes the peer state immediately when handling a peer-remove to ensure the removed peers cannot unexpectedly reappear after a restart (#7602)
• The DiscardNewPerSubject retention policy is now enforced by the leader before proposing rather than by individual replicas, reducing the potential for stream desync (#7607)
• Raft will no longer allow peer-removing the one remaining peer (#7610)
• A data race has been fixed in the stream health check (#7619)
• Tombstones are now correctly written for recovering the sequences after compacting or purging an almost-empty stream to seq 2 (#7627)
• Combining skip sequences and compactions will no longer overwrite the block at the wrong offset, correcting a corrupt record state error (#7627)
• Compactions that reclaim over half of the available space now use an atomic write to avoid losing messages if killed (#7627)
• Filestore message lookups will now expire the cache if a cache error is encountered (#7630)
• Improved error handling within the loading of message blocks if not already cached (#7631)
• Add peer operations to Raft can no longer result in disjoint majorities (#7632)
• Filestore compaction should no longer result in no idx present cache errors (#7634)
• Filestore compaction now correctly adjusts the high and low sequences for a message block, as well as cleaning up the deletion map accordingly (#7634)
• Raft groups should no longer readmit a previously removed peer if a heartbeat occurs between the peer removal and the leadership transfer (#7649)
• Potential stream desyncs that could happen during stream snapshotting have been fixed (#7655)

MQTT

• The maximum payload size is now correctly enforced for MQTT clients (#7555, thanks to @yixianOu)
• Fixed a panic that could occur when reloading config if the user did not have permission to access retained messages (#7596)
• Fixed account mapping for JetStream API requests when traversing non-JetStream-enabled servers (#7598)
• QoS0 messages are now mapped correctly across account imports/exports with subject mappings (#7605)
• Loading retained messages no longer fails after restarting due to last sequence checks (#7616)
• A bug which could corrupt retained messages in clustered deployments has been fixed (#7622)
• Permissions to $MQTT. subscriptions are now handled implicitly, with the exception of deny ACLs which still permit restriction (#7637)
• A bug where QoS2 messages could not be retrieved after a server restart has been fixed (#7643)

Complete Changes

v2.12.2...v2.12.3

Release v2.12.3-RC.5

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Improved

JetStream

• Consumer interest checks on interest-based streams are now significantly faster when there are large gaps in interest (#7656)

Fixed

JetStream

• Potential stream desyncs that could happen during stream snapshotting have been fixed (#7655)

Complete Changes

v2.12.3-RC.4...v2.12.3-RC.5

Release v2.12.3-RC.4

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Dependencies

• golang.org/x/crypto v0.46.0 (#7648)
• golang.org/x/sys v0.39.0 (#7648)

Improved

MQTT

• Retained messages will now work correctly even when sourced from a different account and has a subject transform (#7636)

Fixed

JetStream

• Filestore compaction should no longer result in no idx present cache errors (#7634)
• Filestore compaction now correctly adjusts the high and low sequences for a message block, as well as cleaning up the deletion map accordingly (#7634)
• Raft groups should no longer readmit a previously removed peer if a heartbeat occurs between the peer removal and the leadership transfer (#7649)

MQTT

• Permissions to $MQTT. subscriptions are now handled implicitly, with the exception of deny ACLs which still permit restriction (#7637)
• A bug where QoS2 messages could not be retrieved after a server restart has been fixed (#7643)

Complete Changes

v2.12.3-RC.3...v2.12.3-RC.4

Release v2.12.3-RC.3

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Fixed

General

• WebSocket connections will now correctly limit the buffer size during decompression (#7625, thanks to Pavel Kokout at Aisle Research)

JetStream

• A data race has been fixed in the stream health check (#7619)
• Tombstones are now correctly written for recovering the sequences after compacting or purging an almost-empty stream to seq 2 (#7627)
• Combining skip sequences and compactions will no longer overwrite the block at the wrong offset, correcting a corrupt record state error (#7627)
• Compactions that reclaim over half of the available space now use an atomic write to avoid losing messages if killed (#7627)
• Filestore message lookups will now expire the cache if a cache error is encountered (#7630)
• Improved error handling within the loading of message blocks if not already cached (#7631)
• Add peer operations to Raft can no longer result in disjoint majorities (#7632)

MQTT

• A bug which could corrupt retained messages in clustered deployments has been fixed (#7622)

Complete Changes

v2.12.3-RC.2...v2.12.3-RC.3

Release v2.12.3-RC.2

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Go Version

• 1.25.5 (#7604)

Dependencies

• github.com/klauspost/compress v1.18.2 (#7604)
• github.com/antithesishq/antithesis-sdk-go v0.5.0-default-no-op (#7604)

Added

General

• Added WebSocket-specific ping interval configuration with ping_internal in the websocket block (#7614)

Fixed

JetStream

• The meta layer will now only respond to peer remove requests after quorum has been reached (#7581)
• Invalid subject filters containing non-terminating full wildcard no longer produce unexpected matches (#7585)
• A data race when creating a stream in clustered mode has been fixed (#7586)
• Raft will no longer allow multiple membership changes to take place concurrently (#7565, #7609)
• A panic when processing snapshots with missing nodes or assignments has been fixed (#7588)
• When purging whole message blocks, the subject tracking and scheduled messages are now updated correctly (#7593)
• Raft will no longer count responses from peer-removed nodes towards quorum (#7589)
• The filestore will no longer unexpectedly lose writes when AsyncFlush is enabled after a process pause (#7594)
• The filestore now will process message removal on disk before updating accounting, which improves error handling (#7595, #7601)
• Raft quorum counting has been refactored so the implicit leader ack is now only counted if still a part of the membership (#7600)
• Raft now writes the peer state immediately when handling a peer-remove to ensure the removed peers cannot unexpectedly reappear after a restart (#7602)
• The DiscardNewPerSubject retention policy is now enforced by the leader before proposing rather than by individual replicas, reducing the potential for stream desync (#7607)
• Raft will no longer allow peer-removing the one remaining peer (#7610)

MQTT

• Fixed a panic that could occur when reloading config if the user did not have permission to access retained messages (#7596)
• Fixed account mapping for JetStream API requests when traversing non-JetStream-enabled servers (#7598)
• QoS0 messages are now mapped correctly across account imports/exports with subject mappings (#7605)
• Loading retained messages no longer fails after restarting due to last sequence checks (#7616)

Complete Changes

v2.12.3-RC.1...v2.12.3-RC.2

Release v2.12.3-RC.1

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Go Version

• 1.25.4

Dependencies

• github.com/google/go-tpm v0.9.7 (#7578)
• github.com/nats-io/nkeys v0.4.12 (#7578)
• golang.org/x/crypto v0.45.0 (#7578)

Improved

JetStream

• The scan for the last sourced message sequence when setting up a subject-filtered source is now considerably faster (#7553)
• The metalayer will now stage and deduplicate recovery operations at startup, instead of rapidly applying and then undoing conflicting assignments (#7540)

Fixed

JetStream

• A protocol error caused by an invalid transform of acknowledgement reply subjects when originating from a gateway connection has been fixed (#7579)

MQTT

• The maximum payload size is now correctly enforced for MQTT clients (#7555, thanks to @yixianOu)

Complete Changes

v2.12.2...v2.12.3-RC.1

Release v2.12.2

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Go Version

• 1.25.4 (#7520)

Dependencies

• github.com/klauspost/compress v1.18.1 (#7474)
• github.com/nats-io/nats.go v1.47.0 (#7474)
• golang.org/x/sys v0.38.0 (#7538)
• github.com/minio/highwayhash 070ab1a (#7539)

Added

General

• Support for the PROXY protocol (v1 and v2) for client connections, when proxy_protocol: true is set in the config (#7456)

JetStream

• Added meta_compact and meta_compact_size, advanced JetStream config options to control how many log entries must be present in the metalayer log before snapshotting and compaction takes place (#7484, #7521)
• Added write_timeout option for clients, routes, gateways and leafnodes which controls the behaviour on reaching the write_deadline, values can be default, retry or close (#7513)

Monitoring

• Added expvar /debug/vars endpoint to the monitoring port (#7469)
• Meta cluster snapshot statistics have been added to the /jsz endpoint (#7524)
• The /jsz endpoint can now show direct consumers with the direct-consumers?true flag (#7543)

Improved

General

• Binary stream snapshots are now preferred by default for nodes on new route connections (#7479)
• Reduced allocations in the sublist and subject transforms (#7519)

JetStream

• Improved the logging for observer mode (#7433)
• Improve interest detection when consumers are created or deleted across different servers (#7440)
• Improved the performance of enforcing max_bytes and max_msgs limits (#7455)
• Streams and consumers will no longer unnecessarily snapshot when being removed or scaling down (#7495)
• Streams are now loaded in parallel when enabling JetStream, often reducing the time it takes to start up the server (#7482, #7526)
• Log lines and errors related to offline/unsupported assets are now clearer (#7416, #7425)
• Stream catchups will now use delete ranges more aggressively, speeding up catchups of large streams with many interior deletes (#7512)
• Streams with subject transforms can now implicitly republish based on those transforms by configuring > for both republish source and destination (#7515)
• A race condition where subscriptions may not be set up before catchup requests are sent after a leader change has been fixed (#7518)
• Reduced heap allocations in hash checks (#7539)
• Healthchecks now correctly report when streams are catching up, instead of showing them as unhealthy (#7535)
• Potentially reduced the number of calls to the account resolver for assets that are not resident on that server (#7537)

Fixed

General

• When using message tracing, header corruption when setting the hop header has been fixed (#7443)
• Fixed a bug when validating leafnode proxies (#7444)
• Fixed a regression where token authentication would not be parsed correctly in leafnode URLs (#7452)
• Gateway subscribe/unsubscribe will no longer block on account fetches (#7449)
• Shutting down a server using lame-duck mode should no longer result in max connection exceeded errors (#7527)

JetStream

• Caches will now no longer expire unnecessarily when re-reading the same sequences multiple times in first-matching code paths (#7435)
• Fixed a bug in atomic batching where the incorrect field would be checked when looking for unsupported headers (#7436)
• Fixed a deadlock that could happen when interleaving a direct get request with a batch write (#7458)
• A couple of issues related to header handling have been fixed (#7465)
• No-wait requests now return a 400 No Messages response correctly if the stream is empty (#7466)
• Raft groups will now only report leadership status after a no-op entry on recovery (#7460)
• Fixed a race condition in the filestore that could happen between storing messages and shutting down (#7496)
• A panic that could occur when recovering streams in parallel has been fixed (#7503)
• An off-by-one when detecting holes at the end of a filestore block has been fixed (#7508)
• Indexing a message block in the filestore no longer tries to adjust the cache first sequence (#7508)
• Writing skip message records in the filestore no longer releases and reacquires the lock unnecessarily (#7508)
• Writing message records only overwrites the cache first sequence on the first indexed write (#7508)
• Fixed a bug on metalayer recovery where stream and consumer monitor goroutines for recreated assets would run with the wrong Raft group (#7510)
• Scaling up an asset from R1 now results in an installed snapshot, allowing recovery after restart if interrupted, avoiding a potential desync (#7509)
• Raft groups should no longer report no quorum incorrectly when shutting down (#7522)
• Consumers that existed in a metalayer snapshot but were deleted on recovery will no longer result in failing healthchecks (#7523)
• An off-by-one when detecting holes at the end of a filestore block has been fixed (#7525)
• Fixed a race condition that could happen with shutdown signals when shutting down JetStream (#7536)
• Fixed an issue in the filestore where skipping message sequences did not correctly update the cache accounting (#7534)
• Fixed a deadlock that could occur when purging a stream with mismatched consumer state (#7546)

Complete Changes

v2.12.1...v2.12.2