Bind Object Store reads to the backing stream by name for mirrored buckets#976
Open
ggiesen wants to merge 2 commits into
Open
Bind Object Store reads to the backing stream by name for mirrored buckets#976ggiesen wants to merge 2 commits into
ggiesen wants to merge 2 commits into
Conversation
…ckets ObjectStore.get() and ObjectStore.watch() (and therefore list()) located the backing stream by subject lookup. A mirrored bucket's stream binds no subjects, so both paths raised NotFoundError when reading a bucket whose backing stream is a mirror — e.g. an Object Store mirrored from another JetStream domain over a leafnode link. The bucket name already determines the stream name (OBJ_<bucket>), so pass stream= to both subscriptions, matching the Go client, which binds with OrderedConsumer()+BindStream(streamName) in both GetObject and Watch (nats.go object.go). The KV half of this defect was already fixed by nats-io#807; the new HubLeafJetStreamServerTestCase covers cross-domain mirror reads for both KV and Object Store so the whole of nats-io#505 stays fixed. Fixes nats-io#505 Signed-off-by: Gary Giesen <ggiesen+claude@giesen.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bind Object Store reads to the backing stream by name for mirrored buckets
Fixes #505
Problem
ObjectStore.get()andObjectStore.watch()(and thereforelist(), whichconsumes
watch()) located the backing stream by subject lookup(
JetStreamManager.find_stream_name_by_subject). A mirrored bucket's streambinds no subjects, so both paths raised
NotFoundErrorwhen reading abucket whose backing stream is a mirror - e.g. an Object Store mirrored from
another JetStream domain over a leafnode link, the topology #505 describes.
The bucket name already determines the stream name (
OBJ_<bucket>), and theObjectStoreobject already carries it (self._stream) - the lookup was notjust fragile against mirrors, it was unnecessary.
The Key-Value half of #505 was fixed by #807 (included since v2.15.0); the
regression tests added here cover both KV and Object Store so the whole issue
stays fixed.
Fix
Pass
stream=self._streamto the two subscriptions that previously resolvedby subject:
ObjectStore.get()- the chunk-read ordered consumerObjectStore.watch()- the meta watcher (also serveslist())This matches the Go reference client, which binds with
nats.OrderedConsumer(), nats.BindStream(streamName)in both places(
jetstream/object.go-GetandWatch; same pattern in the legacyobject.go). Behavior is specified by ADR-20 (JetStream-based Object Stores);the KV sibling is ADR-8.
Credit: #808 (@JoaoLockem) proposed the same two-site fix; this PR adds the
regression coverage the fix needs to land safely.
Tests
MirroredDomainBucketsTest, on a newHubLeafJetStreamServerTestCasefixture: a hub server and a leaf server with distinct JetStream domains
connected over a leafnode link, mirrors created with
external: { api: "$JS.hub.API" }.The cross-domain topology is load-bearing for the repro: a same-server
mirror would let the subject lookup resolve the origin stream and silently
mask the defect. (A same-server origin-deleted variant can also reproduce it,
but the cross-domain shape is the one users actually run and the one #505
reports.)
test_object_store_get_from_cross_domain_mirror- multi-chunkget()(256 KiB + 17 B across 128 KiB chunks) plus
list()against the mirror.Fails with
NotFoundError(raised frommanager.find_stream_name_by_subject) without the fix; passes with it.test_kv_get_and_keys_from_cross_domain_mirror- KVget()andkeys()(watch path) against a KV-shaped mirror (
max_msgs_per_subjectinheritedfrom the origin, as
nats kv add --mirrordoes). Passes before and after(regression guard for Avoid $JS.API.STREAM.NAMES call since stream name is known #807).
Conformance against the Go client
Same topology, same mirrored buckets on the leaf node, read with the patched
Python client and natscli 0.4.0 (nats.go):
object get artifacts tiny(mirror)c5ec6f90…)object ls artifacts(mirror)tiny['tiny']kv get cfg alpha(mirror)value-alphavalue-alphakv ls cfg(mirror)alpha,beta['alpha', 'beta']Unpatched nats-py 2.15.0 fails the first two with
NotFoundErrorwhere theGo client succeeds.
Validation
uv run pytest nats/tests/test_js.py::MirroredDomainBucketsTest— 2 passednats/testssuite — green (nats-server v2.14.2, CPython 3.13)uv run ruff check/uv run ruff format --check— cleanNote to maintainers: #808 proposed the same two-line fix - happy to rebase this PR to tests-only if you'd prefer to land #808 for the fix itself; the goal is just that #505 ends up fixed with regression coverage either way.