Skip to content

[FIXED] Prevent duplicate JetStream ingestion subscriptions during concurrent stream updates - #8479

Draft
realMoonBoy wants to merge 1 commit into
nats-io:mainfrom
realMoonBoy:agent/prevent-duplicate-js-ingestion-subscriptions
Draft

[FIXED] Prevent duplicate JetStream ingestion subscriptions during concurrent stream updates#8479
realMoonBoy wants to merge 1 commit into
nats-io:mainfrom
realMoonBoy:agent/prevent-duplicate-js-ingestion-subscriptions

Conversation

@realMoonBoy

Copy link
Copy Markdown

Resolves #7801.

The complete issue trace shows sequences 100-104 spanning two wire-level publishes, with each ingress multiplied independently. A controlled reproduction makes the boundary deterministic: one traced ingress produces eight PubAcks and eight durable stream records.

Concurrent equivalent stream updates could each calculate their subscription delta from the same stale stream configuration. Each update then installed the same internal ingestion subscription. A later protocol publish matched every duplicate subscription, so one ingress was independently accepted multiple times and produced multiple durable records, stream sequences, and PubAcks.

This change serializes updateWithAdvisory per stream, covering the configuration snapshot, delta calculation, and application. It does not add message deduplication or change publish retry, consumer acknowledgement, or redelivery semantics.

The regression test performs eight concurrent equivalent public stream updates, then verifies that one traced ingress produces exactly one PubAck, one durable record/stream sequence, one first delivery, and zero redeliveries.

Validation:

  • Fresh main control (8e54a59546b11dc62bf894b2360c12a01f465cf6): 1 ingress -> 8 PubAcks, 8 records/sequences, 8 first deliveries.
  • Candidate: 1 ingress -> 1 PubAck, 1 record/sequence, 1 first delivery, 0 redeliveries; 20/20 repeated witness runs passed.
  • Targeted PubAck, explicit-ack, retry, deduplication, and redelivery tests passed.
  • Strengthened race run covering the witness and TestJetStreamUpdateStream passed.
  • golangci-lint and Linux amd64/386 builds passed.
  • The full native run recorded two unrelated failures: the known TestLeafNodeSlowConsumer timing flake, and TestOCSPPeerMonitor failing to restart its fixed 8222 monitoring port between subtests. The OCSP failure reproduces identically on the unmodified base; the remainder of the test package passes with that baseline-broken case excluded.

I certify that this contribution is my original work and that I license it to the project under the Apache License 2.0.

Signed-off-by: Tylor M Collins <tmc501@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JetStream message duplication

1 participant