Summary
ScopeManager stores requested scopes keyed by registry host only, while the token ICache is keyed by (partitionId, registry). As a result, scope state is not isolated per partition: in a multi-tenant / shared-client deployment (one Client + one ScopeManager serving many partitions against the same registry host), scopes accumulated on behalf of one partition are merged into, and replayed in, the token requests of every other partition for that host.
We should partition the ScopeManager by partitionId the same way Cache already is, so scope state is isolated per (partitionId, registry).
Current behavior
Two separate structures back auth, and only one of them is partitioned:
-
Token cache (Cache / ICache) — keyed by partition + registry:
GetCacheKey(registry, partitionId) → "ORAS_AUTH_{partitionId}|{registry}" (src/OrasProject.Oras/Registry/Remote/Auth/Cache.cs).
- Value is a
CacheEntry whose Tokens is a Dictionary<scopeKey, token>.
-
Scope accumulator (ScopeManager) — keyed by registry only:
ConcurrentDictionary<string, SortedSet<Scope>> Scopes keyed by registry (src/OrasProject.Oras/Registry/Remote/Auth/ScopeManager.cs).
GetScopesForHost(string registry) / GetScopesStringForHost(string registry) take only registry.
SetActionsForRepository / SetScopeForRegistry write under reference.Registry with no partitionId.
The token request path in Client.SendCoreAsync builds the scope key from the unpartitioned accumulator and then looks up the partitioned cache:
scopeKey = string.Join(" ", ScopeManager.GetScopesStringForHost(host)); // unpartitioned, host-wide
token = Cache.TryGetToken(host, scheme, scopeKey, ..., partitionId); // partitioned
So partitionId isolates the tokens, but the scopeKey that indexes them is derived from host-wide scope state shared across all partitions.
Impact
For a shared single-client deployment (e.g. a proxy or gateway that makes requests on behalf of many downstream callers to a common upstream registry, including large public registries):
- Cross-partition scope bleed (isolation): repo scopes requested for partition A are merged into the shared per-host
SortedSet<Scope> and then included in token requests issued for partition B against the same host. Tenant request scope sets cross-pollinate even though their tokens are partitioned.
- Unbounded scope growth → oversized token requests: because the set is host-wide and never scoped down per partition/operation, it grows with every distinct repo touched on that host. The full set is replayed on every token fetch (
FetchBearerAuthAsync emits one scope= query param per entry), which can blow past token-endpoint URL/header limits and fail auth.
- Cache thrash: the host-wide
scopeKey changes every time any partition touches a new repo, so the per-partition cache key keeps changing → misses → refetch with an ever-larger scope set. (Note MemoryCache SizeLimit/Size=1 counts the outer CacheEntry as 1 and does not bound the inner Tokens dictionary, which also grows per distinct scopeKey.)
Proposed change
Partition ScopeManager state by partitionId, mirroring Cache.GetCacheKey:
- Key scope storage by
(partitionId, registry) instead of registry alone (e.g. reuse the same {partitionId}|{registry} composite key scheme, or nest partitionId -> registry -> SortedSet<Scope>).
- Thread
partitionId through the scope APIs:
GetScopesForHost / GetScopesStringForHost
SetActionsForRepository / SetScopeForRegistry (instance + static helpers)
- the call sites in
BlobStore, ManifestStore, Repository, Registry (they already pass Repository.Options.PartitionId to the cache; pass it to the scope manager too)
- the read/merge in
Client.SendCoreAsync (attemptedKey, GetScopesForHost)
- Keep
partitionId == null behaving exactly as today (default partition) for backwards compatibility.
API note
SetActionsForRepository(IClient, Reference, params Scope.Action[]), SetScopeForRegistry(IClient, string, Scope), and the ScopeManager instance methods are public, so adding a partitionId parameter is a public API surface change. Suggest adding optional string? partitionId = null overloads to stay non-breaking.
Out of scope / related
This issue is specifically about partitioning the scope manager. Separately worth considering as follow-ups (not required here):
- Bounding/evicting per-
(partition, registry) scope sets, and/or scoping token requests to the current operation's repository rather than replaying the full accumulated set (closer to oras-go's context-scoped GetAllScopesForHost), to cap token-request size.
- Bounding the inner
CacheEntry.Tokens dictionary growth.
Acceptance criteria
- Scope state is isolated per
(partitionId, registry); scopes set under one partition are not returned/merged for another partition on the same host.
partitionId == null preserves current behavior.
- Public scope APIs gain non-breaking optional
partitionId overloads, and internal call sites forward PartitionId.
- Tests covering: isolation between two partitions on the same host, and equivalence with existing behavior when
partitionId is null.
Summary
ScopeManagerstores requested scopes keyed by registry host only, while the tokenICacheis keyed by(partitionId, registry). As a result, scope state is not isolated per partition: in a multi-tenant / shared-client deployment (oneClient+ oneScopeManagerserving many partitions against the same registry host), scopes accumulated on behalf of one partition are merged into, and replayed in, the token requests of every other partition for that host.We should partition the
ScopeManagerbypartitionIdthe same wayCachealready is, so scope state is isolated per(partitionId, registry).Current behavior
Two separate structures back auth, and only one of them is partitioned:
Token cache (
Cache/ICache) — keyed by partition + registry:GetCacheKey(registry, partitionId)→"ORAS_AUTH_{partitionId}|{registry}"(src/OrasProject.Oras/Registry/Remote/Auth/Cache.cs).CacheEntrywhoseTokensis aDictionary<scopeKey, token>.Scope accumulator (
ScopeManager) — keyed by registry only:ConcurrentDictionary<string, SortedSet<Scope>> Scopeskeyed byregistry(src/OrasProject.Oras/Registry/Remote/Auth/ScopeManager.cs).GetScopesForHost(string registry)/GetScopesStringForHost(string registry)take onlyregistry.SetActionsForRepository/SetScopeForRegistrywrite underreference.Registrywith nopartitionId.The token request path in
Client.SendCoreAsyncbuilds the scope key from the unpartitioned accumulator and then looks up the partitioned cache:So
partitionIdisolates the tokens, but thescopeKeythat indexes them is derived from host-wide scope state shared across all partitions.Impact
For a shared single-client deployment (e.g. a proxy or gateway that makes requests on behalf of many downstream callers to a common upstream registry, including large public registries):
SortedSet<Scope>and then included in token requests issued for partition B against the same host. Tenant request scope sets cross-pollinate even though their tokens are partitioned.FetchBearerAuthAsyncemits onescope=query param per entry), which can blow past token-endpoint URL/header limits and fail auth.scopeKeychanges every time any partition touches a new repo, so the per-partition cache key keeps changing → misses → refetch with an ever-larger scope set. (NoteMemoryCacheSizeLimit/Size=1counts the outerCacheEntryas 1 and does not bound the innerTokensdictionary, which also grows per distinctscopeKey.)Proposed change
Partition
ScopeManagerstate bypartitionId, mirroringCache.GetCacheKey:(partitionId, registry)instead ofregistryalone (e.g. reuse the same{partitionId}|{registry}composite key scheme, or nestpartitionId -> registry -> SortedSet<Scope>).partitionIdthrough the scope APIs:GetScopesForHost/GetScopesStringForHostSetActionsForRepository/SetScopeForRegistry(instance + static helpers)BlobStore,ManifestStore,Repository,Registry(they already passRepository.Options.PartitionIdto the cache; pass it to the scope manager too)Client.SendCoreAsync(attemptedKey,GetScopesForHost)partitionId == nullbehaving exactly as today (default partition) for backwards compatibility.API note
SetActionsForRepository(IClient, Reference, params Scope.Action[]),SetScopeForRegistry(IClient, string, Scope), and theScopeManagerinstance methods are public, so adding apartitionIdparameter is a public API surface change. Suggest adding optionalstring? partitionId = nulloverloads to stay non-breaking.Out of scope / related
This issue is specifically about partitioning the scope manager. Separately worth considering as follow-ups (not required here):
(partition, registry)scope sets, and/or scoping token requests to the current operation's repository rather than replaying the full accumulated set (closer to oras-go's context-scopedGetAllScopesForHost), to cap token-request size.CacheEntry.Tokensdictionary growth.Acceptance criteria
(partitionId, registry); scopes set under one partition are not returned/merged for another partition on the same host.partitionId == nullpreserves current behavior.partitionIdoverloads, and internal call sites forwardPartitionId.partitionIdis null.