diff --git a/.slopwatch/config.json b/.slopwatch/config.json index 2581472ea..80b1d58cb 100644 --- a/.slopwatch/config.json +++ b/.slopwatch/config.json @@ -14,6 +14,11 @@ "ruleId": "SW004", "pattern": "**/Caching/RedisCacheIntegrationTests.cs", "justification": "Same reason as HybridCacheCoherenceTests, against real Redis: the backplane notification is asynchronous and has no awaitable completion, so the assertion polls to a deadline and exits as soon as the invalidation lands." + }, + { + "ruleId": "SW004", + "pattern": "**/Caching/MemoryCacheManagerTests.cs", + "justification": "One test pins how long an invalidation is remembered, which is the lifetime of FusionCache's tag data. That lifetime is observable only by letting it lapse: any read while it is live is what applies the invalidation and removes the entry, so the test cannot poll. It shortens the lifetime through the consumer hook and waits past it once." } ], "globalSuppressions": [] diff --git a/docs/delivery-caching-analysis.md b/docs/delivery-caching-analysis.md new file mode 100644 index 000000000..179859809 --- /dev/null +++ b/docs/delivery-caching-analysis.md @@ -0,0 +1,629 @@ +# Delivery's caching layer + +Analysis of `Kontent.Ai.Delivery.Caching` (`FusionCacheManager`, the two managers over it, the +builder registrations) and the core's cache path (`CacheKeyBuilder`, `CachePayloadHelper`, +`CachedQueryExecutor`, `DependencyTrackingContext`, the query builders' cache branches). First +written 2026-09-03 against `client-builders` at `dddd4f8fe`; revised the same day against +`18a283435` after a second, independent pass. This is the area `docs/delivery-rich-text-analysis.md` +§8 recorded as never traced. + +> [!NOTE] +> **Status: implemented on `caching-improvements`, 2026-09-03**, in the §6 order, one commit per step, +> every step green in the full Delivery suite with zero warnings. What came out differently from the +> plan, each for a reason found while building: +> +> 1. **2.13 was found during step 3** and outranks everything but 2.1: an invalidation was forgotten +> thirty seconds later for any entry not read in between. Fixed in the same step as 2.7, since +> both are about which entry options each operation runs with. +> 2. **2.1 kept the `IDeliveryCacheManager` shape.** Null from the factory means the origin has no +> value, a throw means it could not be reached; the query builders throw an internal +> `OriginUnavailableException` for an outage and `CachedQueryExecutor` catches it. No three-way +> outcome type was needed. +> 3. **4.1 became `CacheResult.IsStale`**, recorded per call through an `AsyncLocal` the two +> remaining event handlers write into. The five-event bookkeeping, the capped dictionary and +> `IFailSafeStateProvider` are gone. The probe that settled it showed FusionCache raising both +> events with the prefixed key, inline, on the calling context, in memory and hybrid mode alike. +> 4. **4.4 is `CachedItemsFetch`**, and it is where the hybrid fail-safe tests found the raw-JSON +> re-wrap dropping provenance - one place now carries `FromFactory` and `IsStale` across. +> 5. **4.5 also removed a test that tested nothing**: the corrupted-payload seed wrote a key without +> the environment segment, so the entry it planted was never read. An empty cache key now throws. +> 6. **2.5's standalone handle is `DeliveryClient.CacheManager`**, and 2.9's builder is four static +> methods on `DeliveryCacheDependencies` next to the scope constants. + +> [!NOTE] +> Every behavioural claim below was checked against the code, and the ones that could not be settled +> by reading were verified by running throwaway probes against the built assemblies. The first pass +> drove the internal managers directly; the second drove the public surface the way a consumer would - +> `AddDeliveryClient` with `UseMemoryCache` / `UseHybridCache`, `IDeliveryClient`, the keyed +> `IDeliveryCacheManager`, `IDeliveryCachePurger` - over a shared `MemoryCache`, a shared +> `MemoryDistributedCache` standing in for Redis, and stub distributed caches that throw or record keys. +> The probes are listed in §6 so each finding can be re-run or turned into a test. Nothing was changed. + +> [!IMPORTANT] +> **Revision, 2026-09-03.** The second pass kept the first one's structure and most of its findings, +> and changed the following. Each change says why. +> +> - **Added 2.1, fail-safe serving content the origin says is gone.** The first pass recorded that an +> invalidated entry stays eligible for fail-safe (now 2.11) and called it arguably right. Probed +> end to end it is not: with fail-safe on, an item that a webhook evicted and the API then reports +> as `404` is served as a success for up to `FailSafeMaxDuration`. Unpublishing does not take +> effect. That is a correctness defect, so it moved to the top. +> - **Corrected the purge finding (2.4) and the "what is right" bullet it contradicted.** The first +> pass reported purge as isolated over a shared memory cache and leaking only to a fresh node over a +> shared distributed cache. The second pass found it leaking over a shared `IMemoryCache` as well, +> and which of the other client's nodes loses its entries depends on which reads first, not on +> warm versus fresh. The memory setup is the one the caching guide recommends for multi-tenant +> applications, so the severity went up. +> - **Added 2.5 and 2.6**, both found by reading the consumer-facing path rather than the manager: +> the default client's cache manager is reachable only through an internal string and a standalone +> client's not at all, and FusionCache runs with its logger disabled so the outages 2.2 asks it to +> swallow would be swallowed silently. +> - **Added §3 (multi-client isolation, stated once) and §4 (what to simplify).** The question this +> document was asked to answer is whether multi-client setups collide and whether the design carries +> more than it needs; the first pass answered the first in passing and the second not at all. +> - **Extended 2.8** with the wire-format prefix FusionCache would have added and the SDK switched +> off, since it is the same hazard from the other side. +> - Renumbered; the §6 probe table now carries both passes, with the two rows the second pass +> overturned marked. +> - **Added 2.13 during implementation**: an invalidation was forgotten after thirty seconds for any +> entry not read in that window, because the tag data was stored with the write options' default +> duration. Neither pass caught it - every probe read the entry straight after invalidating it. It +> outranks everything but 2.1 and was fixed in the step that restructures the entry options. + +**Verdict: the design is sound where it meets the API - tag invalidation is complete for typed +models and isolated per client and per environment, the backplane is wired the way FusionCache +intends, keys are deterministic and environment-scoped, and stampedes are coalesced. It is unsound +in two places where it meets things it does not own. Fail-safe cannot tell an outage from a +definitive answer, so with it on, unpublished content keeps being served. And FusionCache's own +bookkeeping keys are not namespaced, so a purge on one client empties every client sharing the +store, memory or Redis. Around those, a distributed-cache outage or a size-limited memory cache +throws out of every cached query instead of degrading, the documented FusionCache escape hatch does +not reach the SDK's writes, and a payload-shape change between releases turns every stale Redis hit +into an exception. Each has a small fix, and none moves the public surface except where a fix is +an addition.** + +--- + +## 1. What is right + +Recorded first so the findings read against the whole. + +- **Tag invalidation is isolated per client and per environment.** Dependency tags carry the same + `{KeyPrefix}:{EnvironmentId}:` segment as the keys, so `InvalidateAsync(["item_x"])` on one + client never evicts another's entries - probed over a shared `IMemoryCache`, over a shared + distributed cache, and over a shared backplane channel; in every case the other client kept + serving its own value (probes 1b, 2, 12). +- **The backplane is registered and consumed correctly.** Both FusionCache backplane packages + register `IFusionCacheBackplane` as *transient* (probes 4, 5, 20), so each hybrid manager gets an + instance of its own; `SetupBackplane` is called once per FusionCache; the SDK never subscribes two + caches to one backplane object. Cross-node invalidation with a backplane is pinned by + `HybridCacheCoherenceTests` and the Redis integration test. +- **Disposing a manager does not dispose the application's `IMemoryCache`** (probe 10). FusionCache + disposes only what it created. +- **The preview path never touches the cache.** `DeliveryClient.GetEffectiveCacheManager` returns + null whenever `UsePreviewApi` is on, read per request, so preview and production content cannot + cross in a shared store even under one environment id. +- **Dependency tracking is complete for typed models.** Items, their types, modular-content items + (components filtered by the `workflow`/`workflow_step` signal, their *types* still tracked), linked + items elements, asset elements (id parsed from the URL), rich-text inline images, content links and + inline items, and taxonomy groups - plus the list scope on every listing. The fixture item + `coffee_beverages_explained` yields eight keys: its own item and type, three linked items, two + linked types, one asset, one taxonomy group (probe 15). Type tags on item caches are what makes a + content-type webhook evict item responses, as the interface doc promises. +- **Concurrent misses are coalesced.** Three simultaneous first reads of one key produced one origin + call; the two waiters were reported as `ResponseSource.Cache` (probe 16), which is the right + answer for callers that did not fetch. +- **The `FromFactory` envelope trick is the right answer to eager refresh**, and + `CachedQueryExecutor` reads staleness from the manager rather than from factory locals. + `EnableSyncEventHandlersExecution` is what makes that deterministic. Per-query expiration is set + on the factory context's duplicated options and does not leak into the manager's shared write + options (probe 14). +- **The hybrid tier serializes with the SDK's own options**, so polymorphic content-type elements + survive Redis (`18b964bd0`), and `DistributedCacheKeyModifierMode.None` keeps Redis keys readable + - see 2.8 for what that trades away. +- **Keys are deterministic and order-independent**, filters hashed with 72 bits, element + projections sorted, the model type discriminating hydrated entries and *not* raw-JSON ones so a + distributed cache is shared across model types. The environment id is in the prefix (`9558d0124`). + +## 2. Findings + +Ordered by consequence. Severity is what a consumer experiences, not code size. + +### 2.1 Fail-safe serves content the origin says is gone - **high** + +Every cached query's factory returns `null` from `CacheEntry?` when the API call fails, and +`FusionCacheManager.GetOrSetAsync` turns that null into a thrown sentinel so FusionCache's fail-safe +can serve a stale copy (`FusionCacheManager.cs:293-296`). The factory does not say *why* the call +failed, and the manager cannot ask: a `503` and a `404` arrive as the same null. + +Probed end to end through the public surface (probe 13): cache an item with fail-safe on, invalidate +it as a webhook would, have the origin answer `404` from then on. Every subsequent read returns +`IsSuccess = true` with `ResponseSource.FailSafe`, and the origin is asked on every one of them +(throttle set to zero for the probe) and says `404` every time. The same happens after natural +expiry with no invalidation at all. With fail-safe *off* the same sequence returns the `404` as a +failed result, as it should. So for a consumer who turned fail-safe on, unpublishing an item, deleting +a taxonomy group or removing a type stops taking effect for `FailSafeMaxDuration` - a day by default +- and the webhook that was supposed to make it take effect is what puts the entry into the state +where fail-safe applies (2.11). + +FusionCache's `RemoveByTag` under fail-safe is an *expire*, not a *remove* (its own doc says so), so +this is not a FusionCache bug: fail-safe is doing what it is for. The SDK is feeding it definitive +answers as if they were outages. + +**Fix.** Two parts, both small. First, the factory has to distinguish. `HttpRetryPredicates.IsRetryableStatusCode` +and `IsTransientException` are already compiled into the assembly and are the SDK's own definition of +transient: a transport exception, `408`, `429` or `5xx` is an outage; any other failure is the +origin's answer. Second, the manager has to act on it. Probed against FusionCache directly (probe +21): a factory that sets `ctx.Options.IsFailSafeEnabled = false` before throwing has its exception +propagated rather than a stale value served, but the stale copy stays in the store and the next +call with fail-safe on serves it, so the manager must also `RemoveAsync(key)` after such a failure. + +How the factory tells the manager is a contract choice. The minimal one keeps `IDeliveryCacheManager` +as it is and documents the meaning it already half has: a factory that returns `null` means *the +origin has no value* - do not cache, and drop any stale copy - and a factory that *throws* means it +could not reach the origin, so serve stale if there is one. The query builders then throw an internal +exception for transient failures, carrying the failed result, and catch it after `GetOrSetAsync` +returns; `EnsureApiResult` already covers the coalesced waiters that have no captured result. Custom +managers that just await the factory are unaffected: exceptions propagate through them as they do +today. The more explicit alternative is a three-way outcome type in place of `CacheEntry?` +(`Entry`, `Unavailable`, `Gone`), which is honest about the three cases but changes the interface a +second time in one release. Either way, add the probe as a test in both hosting modes and for both +`RemoveByTag` and natural expiry, and say in `DeliveryCacheOptions.IsFailSafeEnabled` that fail-safe +covers outages, not absence. + +### 2.2 A distributed-cache outage throws out of every cached query - **high** + +`FusionCacheManager.CreateHybrid` sets `ReThrowDistributedCacheExceptions = true` on the write +options every `GetOrSetAsync` passes (`FusionCacheManager.cs:226`). FusionCache's own default is +`false`: on a failed L2 read it logs, falls through to the factory (or the memory tier), and lets +auto-recovery re-sync the distributed tier later. With `true`, the L2 failure is thrown. + +Probed with an `IDistributedCache` that throws (probes 6 and 18): every `GetTypes().ExecuteAsync()` +threw `FusionCacheDistributedCacheException` out of the query, with fail-safe off **and** on - +fail-safe does not help, because the exception is raised before the factory runs. Nothing above the +manager catches it: `CachedQueryExecutor` awaits the fetch unguarded and the query builders return it +to the caller as an exception, not as a failed result - the one place in the SDK where a transport +problem is not a result. So a Redis outage takes down every cached query of every hybrid client, +which is the opposite of what a cache is for, and the guide's "Redis Connection Failures" +troubleshooting entry (`caching-guide.md:1369`) suggests a consumer-side `try/catch` around +`_cache.GetAsync` that does not exist in this SDK, catching a `RedisConnectionException` that +FusionCache has already wrapped. + +The setting dates from `cf142a851` (adopt FusionCache, 2026-02-17) with no recorded reason; the +neighbouring `ReThrowSerializationExceptions = true` has one (a serializer bug should surface) and +this one was most likely set alongside it. `InvalidateAsync` is already the other way round: it +swallows and returns `false`. + +**Fix.** `ReThrowDistributedCacheExceptions = false` on the hybrid write options; keep it `false` on +the invalidate options as it is. Set `DistributedCacheCircuitBreakerDuration` (a few seconds) so a +dead Redis is not retried on every request, and pass FusionCache a logger (2.6) so the degradation +is visible. Add a test with a throwing `IDistributedCache` asserting the factory value is served, +with and without fail-safe. Rewrite the troubleshooting entry to say what actually happens. + +### 2.3 A memory cache with a size limit throws on every write - **high** + +`UseMemoryCache` takes the application's `IMemoryCache` (`AddMemoryCache()` is a `TryAdd`). If the +application configured `MemoryCacheOptions.SizeLimit`, `MemoryCache` requires every entry to declare a +`Size`, and FusionCache sets none unless `FusionCacheEntryOptions.Size` is set. Probed (probes 8 and +19): every `GetOrSetAsync` threw `InvalidOperationException: Cache entry must specify a value for +Size when SizeLimit is set`, on first write and on every retry. The exception propagates to the +caller like 2.2's. + +The caching guide recommends exactly this configuration twice - "Advanced Memory Cache +Configuration" (`caching-guide.md:166`) and the memory-pressure troubleshooting entry (`:1352`) both +show `services.AddMemoryCache(options => options.SizeLimit = 1024)`. A consumer who follows the +guide gets a cache that throws. + +**Fix.** Set `Size = 1` on every entry-options object the manager builds (write, default, invalidate, +and `TagsDefaultEntryOptions`, since tag-expiration entries are memory entries too), and say in +`DeliveryCacheOptions` docs that entries count as one unit each under a size limit. Alternatively let +FusionCache own a private memory tier (`memoryCache: null`) - but the shared `IMemoryCache` is the +documented behaviour and worth keeping. Add a test over a size-limited `MemoryCache`. + +### 2.4 Purge empties every client that shares the store - **high** + +`PurgeAsync` is FusionCache's `Clear`, implemented as a clear timestamp stored under an internal key. +The SDK prefixes its own keys and tags, but FusionCache's internal keys are not prefixed: the probes +recorded the Redis keys a two-client setup produces (probe 2), and next to the namespaced +`a:{env}:cache:types` and `__fc:t:b:{env}:dep:type_article` sits a bare `__fc:t:!` - the clear +marker, shared by every FusionCache instance that shares the store. + +Corrected from the first pass, which reported the leak as distributed-only and as reaching only a +fresh node. Probed through the public surface over one `MemoryDistributedCache` (probes 2, 2b) and +over one `IMemoryCache` (probe 1): after client `a` purges, client `b` goes to the origin too. +Over the distributed cache, which of `b`'s nodes loses its entries depends on which reads first after +the purge - in one ordering the warm node, in the other the fresh one - because the clear marker is +read through the shared tier and the first reader then repopulates it. Over a shared memory cache +both clients lose everything, since the marker is one entry in one `MemoryCache`. Tag invalidation +was isolated in every ordering (probe 1b). + +Why the severity went up: the shared-`IMemoryCache` setup with a prefix per client is what the guide's +"Cache Key Prefixing", "Multi-Tenant Caching" and "Per-Client Caching" sections recommend, and an +operator who purges one tenant after a content-model change empties the others. The effect is extra +origin calls rather than stale content, but it is the kind of cross-tenant coupling the prefixes +exist to rule out. + +**Fix.** Set `FusionCacheOptions.CacheKeyPrefix` to the SDK's prefix segment +(`{KeyPrefix}:{EnvironmentId}:`) and stop prefixing keys and tags by hand - FusionCache then +prefixes its internal keys with it too, and its `Clear` doc says outright that it is designed for +shared caches with a key prefix. Probed (probes 3, 2b, 1): with the prefix set per client, the other +client kept every entry after a purge in both stores, while the purging client's own fresh node +correctly saw the purge and a later tag invalidation. Also put the environment id into `CacheName`, +which names the backplane channel: today two default clients on different environments share +`KontentDelivery.Hybrid.Default` and see each other's notifications (harmless once keys are +namespaced, but noise). Add a two-client purge test over a shared `MemoryCache` and over a shared +`MemoryDistributedCache` with a fresh node. + +### 2.5 The cache manager has no public name for the default client, and none for a standalone one - **medium** + +`RegisterCacheManager` registers the manager as a keyed singleton under the client name and nothing +else (`DeliveryClientBuilderCachingExtensions.cs:110-115`). For a named client that is the documented +route: `GetRequiredKeyedService("production")`, which every invalidation +example in the guide and README uses. For the *default* client - `AddDeliveryClient(delivery => +delivery.UseMemoryCache())`, the most common registration - the key is `NamedClients.Default`, an +internal constant with the value `"Default"`, and the unkeyed `IDeliveryCacheManager` resolves to +nothing (probe 15). Neither the guide nor the README shows the default-client case; a consumer who +follows them with a default client has no way to write the webhook handler other than guessing the +string. + +A client built with `DeliveryClient.Create(delivery => delivery.UseMemoryCache())` is worse off: the +manager lives in the private container and `DeliveryClient` exposes nothing that reaches it, so the +cache can be filled but never invalidated or purged. The only standalone option is `UseCacheManager` +with an instance the caller keeps hold of. + +**Fix.** Register an unkeyed alias for the default client the way `AddClientServices` does for the +client itself - `TryAddSingleton(sp => sp.GetRequiredKeyedService(Default))` +- so `GetRequiredService()` is the default-client route and the string never +appears. Expose the manager on the standalone client, `IDeliveryCacheManager? DeliveryClient.Cache` +or similar, so `Create` users can invalidate. Show both in the guide's invalidation sections next to +the named form. + +### 2.6 FusionCache runs with its logging switched off - **medium** + +Both factories construct FusionCache with `logger: null` (`FusionCacheManager.cs:125` and `:209`). +FusionCache's constructor doc: "if null, logging will be completely disabled". Everything it would +otherwise report - a distributed-cache read that failed and was worked around, a backplane publish +that failed, a background eager-refresh factory that threw, auto-recovery kicking in - is invisible. +The manager's own `ILogger` is used only for the SDK's invalidation messages. + +This matters most once 2.2 is fixed: swallowing the L2 exception is right, swallowing it silently is +not. It also explains why the guide's "Logging" section offers a hand-written wrapper rather than +pointing at FusionCache's own categories. + +**Fix.** Resolve `ILoggerFactory` in the `Use…` extensions and pass `CreateLogger()` to +both factories (`ILogger` is what the constructor takes). Say in the guide which +categories to enable: `ZiggyCreatures.Caching.Fusion.FusionCache` for FusionCache, the manager's for +the SDK. + +### 2.7 `ConfigureFusionCache` does not reach the SDK's writes - **medium** + +Every `GetOrSetAsync` passes `_baseWriteOptions`, an options object the manager builds itself; +`RemoveByTagAsync` and `ClearAsync` pass `_baseInvalidateOptions`. FusionCache uses the options it +is handed and consults `DefaultEntryOptions` only when none are passed (its doc for each method says +so). So anything a consumer sets on `DefaultEntryOptions` through `ConfigureFusionCache` - the +headline example in the XML docs and the guide is +`DefaultEntryOptions.AllowBackgroundBackplaneOperations = true` - has no effect on the SDK's reads, +writes or invalidations. Probed with `DefaultEntryOptions.Size = 1` over a size-limited memory cache +(probes 9 and 19): still threw. `FusionCacheOptions`-level settings (`CacheKeyPrefix`, +`BackplaneChannelPrefix`, `TagsDefaultEntryOptions`) do apply - 2.4's fix was probed through this +very hook - so the hook is not useless, only its documented example is. + +**Fix.** After the callback runs, build the write options by duplicating +`fusionCacheOptions.DefaultEntryOptions` and then applying the SDK's non-negotiables on top (the +tier skips in memory mode, the rethrow flags, the duration and fail-safe policy from +`DeliveryCacheOptions`). Document which settings the SDK pins. A test: set +`DefaultEntryOptions.Size` through the hook and write into a size-limited cache. + +### 2.8 Stored entries carry no format version - **medium, upgrade hazard** + +The distributed tier stores `CacheEnvelope` and the wire models, serialized +by the SDK's options, under keys that carry no format version. `ReThrowSerializationExceptions = +true` is deliberate for the write path, but it also governs reads: probed by writing an entry with +one payload type and reading the same key with a manager expecting another (probe 14, first pass) - +every read threw `FusionCacheSerializationException` instead of treating the entry as a miss. A +Redis outlives a deployment, so after an SDK release that changes `CachedRawItemsPayload`, a wire +model or the envelope, every cached key hit by the new version throws until the entry expires +(`DefaultExpiration`, one hour by default; up to `FailSafeMaxDuration` with fail-safe on). + +Extended from the first pass: the same hazard exists one layer down. FusionCache's default +`DistributedCacheKeyModifierMode.Prefix` puts its wire-format version (`v2:`) into every +distributed key precisely so that a FusionCache upgrade that changes its entry format misses on old +entries rather than failing to read them. The SDK sets `None` for readable Redis keys +(`FusionCacheManager.cs:198`), so a future FusionCache major would read the old format through the +same `ReThrowSerializationExceptions = true`. + +**Fix.** One version segment under the SDK's control in the hybrid key prefix - `v1:` between the +client prefix and the key, bumped whenever a cached type *or the FusionCache wire format* changes +shape (the approval-snapshot habit makes the first visible; the second is a dependency bump to look +at). With 2.4's move to `CacheKeyPrefix` the segment belongs in that prefix, so FusionCache's own +keys are versioned with it. Keep `ReThrowSerializationExceptions = true`; with versioned keys it +only ever reports a real bug. Note it in the release checklist, before 20.0.0 ships so the first +stable line starts at version 1. + +### 2.9 Invalidation keys are case-sensitive while tracking is not - **low** + +`DependencyTrackingContext` dedupes with `OrdinalIgnoreCase`, but tags are formatted verbatim and +FusionCache compares them ordinally. Probed (probe 7): `InvalidateAsync(["ITEM_HERO"])` left an +entry tagged `item_hero` in place; the exact key evicted it. The SDK's own keys are lower-case by +construction (codenames, `Guid` formatting), so this only affects consumer-built keys, but a webhook +handler that upper-cases or copies an id from a payload with different casing silently evicts +nothing. The root cause is that consumers build these strings by hand: `CacheDependencyKeyBuilder`, +which knows the formats, is internal. + +**Fix.** Normalize to lower-case invariant in one place - the tag formatter and the start of +`InvalidateAsync` - and say so on `IDeliveryCacheManager.InvalidateAsync`. Better, make the key +builder public (`DeliveryCacheDependencies.ForItem(codename)` and friends next to the scope +constants), so the webhook handler in the guide composes keys the same way the SDK does and the +casing question does not arise. + +### 2.10 A cold node pays one distributed read per tag on its first hit - **characteristic, document** + +FusionCache verifies an entry's tags against tag-expiration entries on every read; those are cached +in the memory tier, but a node that has never seen them reads each from L2. Probed with an entry +carrying 50 tags (probe 13, first pass): the writing node did 1 L2 read; a fresh node's first hit +did 53; its second hit did 0. A listing of 100 items with their types, assets and taxonomies carries +several hundred tags, so the first hit on every node after a restart - and again after the tag +entries expire - costs that many Redis round trips. Not a bug, and the memory tier makes it a +warm-up cost, but the guide should say it, and `TagsDefaultEntryOptions.Duration` is the knob (set +it through `ConfigureFusionCache`, which does apply at that level). + +### 2.11 Invalidated entries remain eligible for fail-safe - **behaviour, document** + +`RemoveByTag` marks entries logically expired rather than deleting them when fail-safe was on at +write time. Probed (probe 11): after `InvalidateAsync(["item_x"])`, a factory that threw got the +pre-invalidation value served as a fail-safe hit, and so did a factory returning null. That is +FusionCache's intended semantics and the right call during an outage. It is the mechanism behind +2.1, and once 2.1 distinguishes outages from answers it is also the right behaviour after a webhook: +evicted, then served stale only while the origin is unreachable. The guide's purge section already +draws this distinction for `PurgeAsync(allowFailSafe: true)`; the invalidation section should draw +it too. + +### 2.12 Smaller + +- **The environment prefix is read once.** `EnvironmentIdOf` reads `EnvironmentId` when the manager + is created and bakes it into the formatters. The DI path advertises per-request option reads, so a + reload that switches environments keeps caching under the old prefix. Rare; a sentence in the + guide's "changing options at runtime" note is enough (it already says to purge). +- **Hydrated entries are shared instances with the client's options baked in.** A memory hit hands + back the very object the previous caller received - same `IContentItem`, same `Elements` + (probe 15) - with `CustomAssetDomain` and `DefaultRenditionPreset` already applied. A consumer who + mutates a model mutates the cache for everyone, and a runtime change to either option does not + reach cached entries until they expire or are purged. The hybrid path rehydrates per hit and has + neither property. Both are inherent to caching hydrated objects and worth one paragraph in the + guide; the first is the kind of thing a consumer discovers in production. +- **Typed dynamic models track fewer dependencies.** For `TModel` where `ModelTypeHelper.IsDynamic` + is true, `ProcessItemsAsync` skips `CompleteItemAsync`, so element-level dependencies (assets, + taxonomy groups, rich-text links) are not tracked; items, types and the list scope are. The + untyped `GetItems()` bypasses the cache entirely, so this only reaches a caller that names a dynamic + model type explicitly. Worth a line in the coverage section. +- **`ItemsListParams` with both `Elements` and `ExcludeElements`** puts only `Elements` in the key. + If the API accepts both, two different projections share an entry; if the SDK forbids the + combination, nothing to do. +- **Assets are missing from the invalidation matrix.** `asset_{guid}` is tracked from asset + elements and rich-text images and listed in the `IDeliveryCacheManager` remarks, but the guide's + "Invalidation Matrix" (`caching-guide.md:638`) has rows for items, types and taxonomies only, so + an asset webhook has no documented mapping. +- **Guide drift.** "Keys have NO prefix: item:homepage" and "`EnvironmentId` ... [is] not part of + query cache keys" (Key Prefixing, `:491-535`) predate `9558d0124`; every key now starts with the + environment id, and the "No prefix (explicit)" example (`KeyPrefix = ""`) merges a named client + into the default client's namespace on the same environment without saying so. The Redis + troubleshooting entry (2.2) and the size-limit advice (2.3) are wrong in the direction that hurts. + The `ConfigureFusionCache` example (2.7) is a no-op. The hybrid-cache note (`:103`) attributes + per-hit rehydration to a FusionCache limitation; it follows from the SDK's own `RawJson` storage + mode, and "negligible" undersells a JSON parse plus element mapping that includes the rich-text + HTML parse. The "Monitor Cache Performance" and "Logging" wrappers (`:1106`, `:1279`) detect a miss + with a local flag set inside the factory, which is exactly the pattern `CacheResult.FromFactory` + exists to replace because eager refresh runs the factory for a different call; they should read + `result.FromFactory`. + +### 2.13 An invalidation is forgotten after thirty seconds unless the entry is read - **high, found during implementation** + +Found while implementing 2.7, which is where the entry options are built. `RemoveByTag` and `Clear` +write a tag-expiration entry that every later read of a tagged entry checks against, and FusionCache +stores that entry with the options passed to the call. The SDK passed `_baseInvalidateOptions`, a +`FusionCacheEntryOptions` that named no `Duration`, so the tag data lived for FusionCache's entry +default of thirty seconds. An entry not read within thirty seconds of the webhook that invalidated it +was served afterwards for the rest of its own expiration, as if the webhook had never arrived. + +Probed against FusionCache directly (probe 22): `RemoveByTag` with `Duration = 1s` on the options was +forgotten 1.5 seconds later; with `null` options, which selects `TagsDefaultEntryOptions` and its +ten-day default, it held. Then through the SDK (probe 23): cache the type listing, invalidate its list +scope, wait 32 seconds without reading, read - served from cache, origin never asked. A purge held in +the same probe, because `Clear` also keeps its timestamp in the instance; on a second node reading the +shared marker it would not. + +None of the probes in either pass caught this because every one of them read the entry immediately +after invalidating it. The first pass's 2.10 even noted that tag entries expire without asking when. + +**Fix.** Pass no options to `RemoveByTag` and `Clear`, so FusionCache uses `TagsDefaultEntryOptions`, +and configure that object with the SDK's pinned flags at construction; its duration stays FusionCache's +ten days and is the consumer's knob through `ConfigureFusionCache`. Document that it must exceed the +longest expiration in use. The test that pins it shortens the tag duration through the hook and shows +the invalidation lapsing with it, which is the only way to observe the lifetime without waiting. + +## 3. Multi-client isolation, stated once + +The namespace every key and tag lives in is `{KeyPrefix}:{EnvironmentId}:`, where `KeyPrefix` +defaults to the client name for a named client and to nothing for the default one +(`ResolveCacheKeyPrefix`). So the default client caches under `{env}:…`, a named client under +`{name}:{env}:…`, and hybrid keys add `cache:` / `dep:` after that. Everything else the key needs - +language, depth, projection, pagination, ordering, filters, and in memory mode the model type - is in +the key itself. + +| Setup | Isolated? | By what | +|---|---|---| +| Two named clients, any environments, one `IMemoryCache` or one Redis | Keys and tags: yes | Client name in the prefix | +| Two applications, default clients, different environments, one Redis | Keys and tags: yes | Environment id in the prefix | +| Two applications, default clients, the *same* environment, one Redis | Shared, on purpose | Same content; hybrid stores raw JSON so each app rehydrates with its own asset options | +| Production and preview clients on one environment | Preview never caches | `GetEffectiveCacheManager` | +| Any of the above | **Purge: no** (2.4) | FusionCache's clear marker is outside the prefix | +| Default clients across applications, one Redis backplane | Notifications shared | `CacheName` lacks the environment id (2.4) | +| A named client with `KeyPrefix = ""` | Merged with the default client on that environment | The guide offers this without the consequence (2.12) | +| Two named clients given the same explicit `KeyPrefix` on one environment | Merged | Consumer's choice, but nothing warns | + +Two things are global rather than per client and are fine as long as they are known: `ITypeProvider` +is one unkeyed registration shared by every client in a container, and the hybrid managers of one +container each open their own backplane instance (transient), so two hybrid clients mean two Redis +subscriptions unless the backplane is configured with a shared multiplexer. + +Invalidation, checked against what a content change actually touches: an item webhook evicts the +item, every listing that contained it and every item whose modular content or rich text referenced +it; the list scope covers listings the new item should now appear in; a type webhook evicts the type +definition, type listings, and every item response containing an item of that type; a taxonomy +webhook evicts the group and every item whose taxonomy element draws from it; an asset webhook evicts +every item whose asset element or inline image is that asset. Components ride on their owning item. +Languages, used-in lookups, element lookups, the feed and the untyped queries are not cached, so +nothing to evict. The one hole is 2.1: with fail-safe on, an eviction followed by a definitive +answer does not stick. + +## 4. What to simplify + +The layer is not over-built for what it does; the two storage modes, the envelope, and the tag +plumbing each earn their place. What follows is where the same behaviour can be had with less, and +one place where the current shape is a liability. + +### 4.1 Fail-safe classification: put it in the result, per call + +`FusionCacheManager` subscribes to five FusionCache events (`FailSafeActivate`, `FactorySuccess`, +`Hit`, `Remove`, `Memory.Eviction`) to maintain `_failSafeActiveKeys`, a process-wide dictionary of +formatted keys with a 10,000-entry cap that is cleared wholesale when reached (`:37`, `:477-503`). +`CachedQueryExecutor` then asks the manager, through the internal `IFailSafeStateProvider`, whether +the key it just read is in that set (`CachedQueryExecutor.cs:70-71`), and the manager re-formats the +key to look it up (`:422-423`). This is a side channel answering a per-call question with global +state: two concurrent reads of one key, one served stale and one fresh, can classify each other, and +the cap is a "clear everything" bounded structure. + +The events are needed - FusionCache does not return metadata from `GetOrSetAsync`, and the throttled +stale hit is only observable through `Hit` with `IsStale` - but the *state* does not have to be +global. With `EnableSyncEventHandlersExecution` the handlers run inline on the calling thread, so a +per-call slot (an `AsyncLocal` set around the `GetOrSetAsync` call, checked by the handlers for the +matching key) records "this call observed a stale hit or a fail-safe activation for this key" and +nothing else. The manager then returns it as `CacheResult.IsStale` next to `FromFactory`, the +executor reads the result instead of probing, and `IFailSafeStateProvider`, the dictionary, the +cap, the unformatted-key removals (`:309`) and the double lookup all go. `CacheResult` gains one +init-only property, which a custom manager is free to set. + +### 4.2 The two managers are one manager + +`MemoryCacheManager` and `HybridCacheManager` are 47 and 59 lines that forward every member to a +`FusionCacheManager` built by `CreateMemory` or `CreateHybrid`. They exist to give `UseMemoryCache` +and `UseHybridCache` a type to construct and the tests a type to assert on. Registering +`FusionCacheManager.CreateMemory(...)` and `CreateHybrid(...)` directly removes both classes and +changes nothing observable except the logger category, which moves from +`Kontent.Ai.Delivery.Caching.MemoryCacheManager` to `…FusionCacheManager` - acceptable in a +prerelease, and the category can be kept by naming the logger explicitly if it matters. The +`IDeliveryCacheManager`, `IDeliveryCachePurger` and `IDisposable` surface is unchanged. + +### 4.3 Let FusionCache own the prefix + +With 2.4's `CacheKeyPrefix`, `_cacheKeyFormatter` and `_dependencyTagFormatter` become two constant +strings, `ComposeKeyPrefix` moves into the options, and the `cache:` / `dep:` segments can go: they +exist to keep keys and tags apart in Redis, but FusionCache already stores tag data under +`__fc:t:{tag}`, so a key and a tag cannot collide whatever they are called. The version segment +from 2.8 takes their place. This changes Redis key shapes, which is one more reason to do it before +20.0.0 rather than after. + +### 4.4 One cached-fetch helper for items and listings + +`ItemQuery` and `ItemsQuery` each carry a raw-JSON branch and a hydrated branch +(`ItemQuery.cs:116-189`, `ItemsQuery.cs:161-234`) that differ only in the payload factory and the +rehydrator. The mode split itself should stay: hydrated objects cannot cross a process boundary, +and rehydrating on every memory hit would put a JSON parse, element mapping and an AngleSharp parse +of every rich-text element on the hot path of the fastest cache tier. But the two branches are one +generic method with two delegates, and `TypeQuery`, `TaxonomyQuery` and the listings already show +what the single-branch shape looks like. + +### 4.5 Defensive code nothing reaches + +- `GetOrSetAsync`'s empty-key path (`FusionCacheManager.cs:264-275`) runs the factory uncached. + `CacheKeyBuilder` cannot produce an empty key and no SDK code passes one; the path is pinned by a + unit test and reached by nothing else. +- Dependencies are deduplicated case-insensitively in `DependencyTrackingContext` and again in the + manager (`:270-273`, `:301-304`), and the type and taxonomy listing builders dedupe a third time + in their `BuildDependencies`. One place is enough; the manager is the boundary, so keep that one. +- `_failSafeActiveKeys.TryRemove(cacheKey, …)` on the unformatted key (`:309`) removes nothing that + was ever added; it goes with 4.1. + +### 4.6 Not to change + +The envelope-identity `FromFactory` test, `EnableSyncEventHandlersExecution`, storing the +dependency keys inside the envelope so a hit can surface them, the model-type discriminator in +memory mode only, and the `Distinct`-then-`ConvertAll` tag shaping inside the factory are each the +plain answer to a real constraint, and the first pass's §3 found no defect in any of them. + +## 5. What was looked for and not found + +- Duplicate keys for one logical query: none. Order-independence holds for filters and projections; + hydrated and raw-JSON modes are keyed differently on purpose; `WaitForLoadingNewContent` bypasses + both lookup and store; the auto-added type filter for typed models is part of the hash consistently. +- Cross-client tag collisions: none, in memory or distributed, with or without a backplane, in + either pass. +- Per-query expiration leaking across calls: none; the factory context's options are a duplicate + (probe 14). +- Backplane misuse: none - transient instances, one `SetupBackplane` per cache, synchronous publish + by default so an invalidation has reached the channel before `InvalidateAsync` returns. +- Disposal: the manager disposes its FusionCache; FusionCache disposes only what it created (not the + application's memory cache, not the `IDistributedCache`); the container disposes the manager. + +## 6. Suggested order + +1. **2.1** first, on its own: it is the one finding where a consumer sees wrong content rather than + a slow or failing call, and it decides the factory contract that 4.1 then builds on. +2. **2.2, 2.3 and 2.6** together - the first two turn infrastructure conditions into thrown + exceptions in every cached query and are one flag each plus a test; 2.6 is what makes 2.2's + swallowing observable, and 2.3's `Size` wants 2.7's options plumbing. +3. **2.7**, because it decides how 2.3's `Size` and the consumer's own settings flow into the + SDK's entry options. +4. **2.4, 2.8 and 4.3** as one change to the key scheme: `CacheKeyPrefix` with the environment and + a version segment in it, `CacheName` with the environment id, the `cache:`/`dep:` segments + dropped, two-tenant purge tests over both stores with a fresh node. Before 20.0.0, since it + changes Redis keys. +5. **2.5** - the unkeyed alias and the standalone client's handle; two registrations and a property. +6. **2.9** - normalization, and the public key builder. +7. **4.1, 4.2, 4.4, 4.5** - internal, no snapshot movement except `CacheResult.IsStale`. +8. **Docs**: the guide's key-prefixing section, the Redis and memory-pressure troubleshooting + entries, the escape-hatch example, the default-client and standalone invalidation routes, the + asset row in the matrix, the cold-node cost, fail-safe after invalidation, the shared-instance + and baked-options paragraph, dynamic models, and the two wrapper samples that should read + `FromFactory`. + +Everything is behind the existing public surface except the additions in 2.5, 2.9 and 4.1 and the +doc wording; the approval snapshots move only for those. Every item is a candidate for the release +candidate line rather than a later major: 2.1, 2.2, 2.3 and 2.4 are the kind of thing a first +production deployment finds. + +## 7. Probes + +The first pass ran each as a throwaway xUnit fact in `Kontent.Ai.Delivery.Tests` against the +internal managers; the second ran a throwaway console program against the built assemblies through +the public surface only. Both were deleted afterwards. Re-run any of them by rebuilding from its +description. Rows the second pass overturned are marked. + +| # | Pass | Setup | Result | +|---|---|---|---| +| 1 | both | Two named clients over one `MemoryCache`; `a` purges | First pass: isolated. **Overturned** by the second: `b` went to the origin too (probe 1); a control run without the purge kept `b` on cache; with `CacheKeyPrefix` per client, `b` kept its entry (2.4) | +| 1b | second | Same, `a` invalidates `scope_types_list` | `a` refetched, `b` served from cache - tag invalidation isolated | +| 2 | both | Two clients over one `MemoryDistributedCache`; `a` purges; `b` read from its warm node and from a fresh node | First pass: warm serves, fresh misses. **Refined**: whichever of `b`'s nodes reads first after the purge misses and repopulates; the other then hits (2.4). Recorded L2 keys: `a:{env}:cache:types`, `__fc:t:{prefix}dep:…`, and a bare `__fc:t:!` | +| 2b | second | Probe 2 with the fresh node reading first | Fresh `b` missed, warm `b` then hit; with `CacheKeyPrefix` both hit, one origin call for `b` in total | +| 3 | both | Probe 2 with `FusionCacheOptions.CacheKeyPrefix` set per client | `b` unaffected in every ordering; `a`'s own fresh node sees purge and invalidation (fix for 2.4) | +| 4 | first | `AddFusionCacheMemoryBackplane`, resolved twice; one instance handed to two managers | Transient; both managers work; invalidation isolated | +| 5 | first | `AddFusionCacheStackExchangeRedisBackplane` descriptor lifetime | Transient | +| 6 | first | Hybrid manager over a throwing `IDistributedCache`, fail-safe off and on | Every `GetOrSetAsync` throws `FusionCacheDistributedCacheException`; `InvalidateAsync` returns true (2.2) | +| 7 | first | `InvalidateAsync(["ITEM_HERO"])` against a tag `item_hero` | Not evicted; exact case evicts (2.9) | +| 8 | first | Memory manager over `MemoryCache { SizeLimit = 10000 }` | Every write throws (2.3) | +| 9 | first | Same as 8 with `ConfigureFusionCache(f => f.DefaultEntryOptions.Size = 1)` | Still throws (2.7) | +| 10 | first | Manager disposed; application's `MemoryCache` used afterwards | Still usable | +| 11 | first | Fail-safe on; invalidate; factory throws / returns null | Pre-invalidation value served as fail-safe (2.11) | +| 12 | first | Two default clients on different environments, shared L2 and backplane channel | Invalidate isolated; purge leaks to the other (2.4) | +| 13 | second | Default client, `UseMemoryCache` with fail-safe on and zero throttle; item cached; `InvalidateAsync(["item_…"])`; origin then answers `404` | Every read: `IsSuccess = true`, `ResponseSource.FailSafe`, and the origin asked each time. Same after natural expiry. With fail-safe off: the `404` as a failed result (2.1) | +| 13b | first | Entry with 50 tags; counting `IDistributedCache`; fresh node reads twice | Write path 1 read; cold first hit 53; warm second hit 0 (2.10) | +| 14 | both | First pass: entry written as `CacheEnvelope`, read as `CacheEnvelope`. Second pass: `_baseWriteOptions.Duration` before and after a `WithCacheExpiration` query | `FusionCacheSerializationException` on read (2.8). Duration unchanged at one hour: per-query expiration does not leak | +| 15 | second | Default client, memory: same item read twice; unkeyed `IDeliveryCacheManager` resolved | Same `IContentItem` and `Elements` instance on the hit; eight dependency keys for the fixture item; unkeyed manager resolves to nothing (2.5, 2.12) | +| 16 | second | Three concurrent `GetTypes()` on a cold memory cache, origin delayed | One origin call; sources `Origin`, `Cache`, `Cache` | +| 17 | second | Default client, hybrid over `MemoryDistributedCache`: same item read twice | Fresh instance on the hit, `ResponseSource.Cache` | +| 18 | second | Probe 6 through the public surface: `UseHybridCache` over a throwing `IDistributedCache`, `GetTypes().ExecuteAsync()` | The query itself throws `FusionCacheDistributedCacheException`, not a failed result; fail-safe on or off (2.2) | +| 19 | second | Probe 8 and 9 through the public surface: `AddMemoryCache(o => o.SizeLimit = 1000)` + `UseMemoryCache`, with and without `DefaultEntryOptions.Size` through the hook | Throws either way (2.3, 2.7) | +| 20 | second | Both backplane packages' `IFusionCacheBackplane` descriptor lifetime | Transient | +| 21 | second | FusionCache directly, fail-safe on, entry expired: (a) factory throws; (b) factory sets `ctx.Options.IsFailSafeEnabled = false` and throws; (c) next call with fail-safe on; (d) after `RemoveAsync`, factory throws; (e) after `ExpireAsync`, factory throws | (a) stale served; (b) exception propagates; (c) stale served again - the copy is still there; (d) exception propagates; (e) stale served. The mechanics 2.1's fix needs | +| 22 | implementation | FusionCache directly: `RemoveByTag` with options `Duration = 1s`, with a bare `FusionCacheEntryOptions`, and with `null`; entry read 1.5 s later | 1 s: served again; bare: gone at 1.5 s (but its duration is 30 s); null: gone. `FusionCacheEntryOptions.Duration` defaults to 30 s, `TagsDefaultEntryOptions.Duration` to 10 days (2.13) | +| 23 | implementation | Default client, `UseMemoryCache`: type listing cached, `scope_types_list` invalidated, 32 s without a read, then read; then purge, 32 s, read | Invalidation forgotten - served from cache, origin not asked. Purge held (2.13) | diff --git a/src/delivery/CHANGELOG.md b/src/delivery/CHANGELOG.md index 07cb7475d..049c903fe 100644 --- a/src/delivery/CHANGELOG.md +++ b/src/delivery/CHANGELOG.md @@ -81,6 +81,12 @@ Entries before the move to this monorepo were imported from the GitHub Releases ### Added +- **The cache manager is reachable without knowing the default client's key, and on a standalone client at all.** The manager was registered only as a keyed service under the client's name, which for the default client is an internal constant no document names - so a webhook handler for the common `AddDeliveryClient(delivery => delivery.UseMemoryCache())` setup had no route to `InvalidateAsync` but a guessed string. It now resolves unkeyed as well, `GetRequiredService()`, the way the client itself does; named clients keep their keyed registration. A client built with `DeliveryClient.Create` owns its container and exposed nothing inside it, so its cache could be filled but never invalidated; `DeliveryClient.CacheManager` is the manager it was registered with, or `null` when it caches nothing. + +- **`DeliveryCacheDependencies` composes dependency keys, and invalidation matches them case-insensitively.** A webhook handler had to write `"item_" + codename` by hand, and the SDK compared the result ordinally against tags it had composed lower-case, so a key copied from a payload in another casing evicted nothing and said nothing. `ForItem`, `ForType`, `ForTaxonomy` and `ForAsset` now sit next to the scope constants and produce the exact strings the SDK tags with, trimmed and lower-cased; `InvalidateAsync` normalizes what it is given the same way. + +- **`CacheResult.IsStale` says whether fail-safe served a stale copy.** It is what turns a cache hit into `ResponseSource.FailSafe`, and it belongs with the value: the cache is the only component that knows what it handed back, and the call it handed it to is the only one the answer applies to. The SDK's managers record it per call, on the async context of the read, which replaces a process-wide table of keys in fail-safe that any concurrent read of the same key could overwrite. A custom manager sets it on the results it returns. + - **`ExecuteAsync(continuationToken)` on the feed and used-in queries**, resuming a walk from a persisted cursor. Added as an overload rather than a parameter on the existing method, so `ExecuteAsync(cancellationToken)` keeps compiling. - **`ContinuationToken` on `IDeliveryItemsFeedResponse` and `IDeliveryItemsFeedResponse`**, making the feed's result-based route resumable across a process restart, which `FetchNextPageAsync` cannot be. `HasNextPage` is unchanged and remains equivalent to the token being present. @@ -122,12 +128,28 @@ Entries before the move to this monorepo were imported from the GitHub Releases ### Changed +- **Distributed cache keys carry a format version, and lose the `cache:` and `dep:` segments.** A Redis outlives a deployment, so an entry written by one version of the SDK is read by the next; with nothing in the key to say what shape it has, a payload change between releases made every stale hit throw `FusionCacheSerializationException` until the entry expired. A hybrid client's keys are now `{KeyPrefix}:{EnvironmentId}:v1:{key}`, the version to be bumped whenever a cached type or FusionCache's own entry format changes, so an upgraded node misses on old entries instead. The `cache:` and `dep:` segments are gone with it - FusionCache keeps tag data under keys of its own, so the two could never collide - and the tag data is now stored under the client's prefix as well. Entries written by a previous prerelease are not read; they expire on their own. + +- **The SDK's cache logs under one category, `Kontent.Ai.Delivery.Caching.FusionCacheManager`.** `MemoryCacheManager` and `HybridCacheManager` were two forwarding classes over one implementation, distinguishable only by their logger categories; they are folded into it. The public surface is untouched - both were internal - and `IDeliveryCacheManager.StorageMode` still says which kind a manager is. + - **Refit moves to 15.2.0, and the `Microsoft.Extensions.*` packages to 10.0.11 with it.** Refit 15 adds a keyed registration for source-generated clients, which is the one registration the SDK had to hand-roll and now uses instead; nothing else in the release touches what the SDK uses, and the whole test suite passes on it unchanged. The package's Refit dependency floor moves accordingly, so an application that pins Refit 14 alongside this package must move to 15 as well. - **`Microsoft.Extensions.Configuration` and `Microsoft.Extensions.Configuration.Binder` are no longer direct dependencies, nor is `Microsoft.Extensions.Primitives`.** Nothing in the package used them. `Microsoft.Extensions.Options.ConfigurationExtensions` stays and brings the binder with it, so `Options.BindConfiguration` on the builder works unchanged; an application that built its own `ConfigurationBuilder` on the package's transitive reference must reference `Microsoft.Extensions.Configuration` itself. ### Fixed +- **Fail-safe no longer serves content the API says is gone.** With `IsFailSafeEnabled`, a cached item that a webhook invalidated and the API then answered `404` for was served as a success with `ResponseSource.FailSafe` for up to `FailSafeMaxDuration` - a day by default - so unpublishing did not take effect. The cache could not tell an outage from an answer: every failed fetch reached it as the same "nothing to cache". Now a fetch that got no response, or a status the SDK's own pipeline retries (`408`, `429`, `5xx`), is the outage fail-safe exists for and a stale copy may be served; any other answer is final, the stale copy is dropped with it, and the failure is returned. The `IDeliveryCacheManager` factory contract says the same: return `null` when the origin has no value, throw when it could not be reached. + +- **A distributed-cache outage degrades the cache instead of failing every query.** With `UseHybridCache`, a Redis that could not be reached threw `FusionCacheDistributedCacheException` out of every cached query - the one place in the SDK where a transport problem was an exception rather than a result, and fail-safe did not help because the failure came before the factory ran. The distributed tier is now worked around: the memory tier or the origin answers, a two-second circuit breaker keeps a dead Redis from being retried on every request, and FusionCache re-syncs the tier when it is back. FusionCache's own diagnostics were also silent, its logger never having been wired; they now log under `ZiggyCreatures.Caching.Fusion.FusionCache` whenever logging is registered, so a worked-around outage, a failed backplane publish or a background refresh that threw is visible. + +- **An invalidation is no longer forgotten after thirty seconds.** `InvalidateAsync` records a tag-expiration entry that the next read of each tagged entry checks against, and the SDK stored that entry with the same options as a cached value - options that named no duration, so FusionCache's thirty-second default applied. An entry not read within thirty seconds of the webhook that invalidated it was served again afterwards, for the rest of its own expiration, as if the webhook had never arrived. The tag data is now stored with FusionCache's tag options, whose duration is ten days by default and is adjustable through `ConfigureFusionCache(f => f.TagsDefaultEntryOptions.Duration = …)`; set it above your longest expiration, per-query overrides included. `PurgeAsync` uses the same options. + +- **`ConfigureFusionCache` reaches the SDK's reads and writes.** Whatever the callback set on `DefaultEntryOptions` - the documented example is `AllowBackgroundBackplaneOperations = true` - applied to nothing, because every SDK operation passed entry options it had built itself. The consumer's `DefaultEntryOptions` is now the starting point of those options, so a `Size`, a distributed-cache timeout or the background-operation flags set there take effect. What the SDK decides stays decided: the duration, fail-safe, jitter and eager-refresh policy come from `DeliveryCacheOptions`, serialization failures are thrown, distributed-cache and backplane failures are not. + +- **A purge no longer empties every other client sharing the store.** `PurgeAsync` is FusionCache's `Clear`, which records a purge as a marker key of its own, and that key carried none of the SDK's prefix. Every client sharing the `IMemoryCache` or the Redis read the same marker, so purging one tenant's cache after a content-model change sent the others back to the origin too - the setup the caching guide recommends for multi-tenant applications. The SDK now hands its prefix to FusionCache as its `CacheKeyPrefix`, so every key FusionCache stores for a client carries it, its own markers included, and a purge reaches only the client that asked for it. + +- **A size-limited memory cache no longer refuses every entry.** `services.AddMemoryCache(o => o.SizeLimit = …)`, which the caching guide recommended, made every write throw `Cache entry must specify a value for Size when SizeLimit is set`, because the SDK declared no size. Every entry it writes now counts as one unit, tag entries included, so a limit bounds the number of cached responses. + - **Resolved rich text is encoded by one encoder throughout.** Text nodes used a Unicode-preserving encoder while attribute values and an inline image's `alt` used `HtmlEncoder.Default`, so the same character survived in one position and was escaped in the other — `

café

` next to `alt="café"` in a single document. Both now use the Unicode-preserving encoder. HTML-reserved characters are escaped exactly as before; what changes is that non-ASCII characters in the Basic Multilingual Plane now appear literally in attribute values too. Output that pins the old numeric references character-for-character will differ; rendered output does not. The comment describing that encoder claimed it preserved emojis. It does not: `UnicodeRanges.All` is the Basic Multilingual Plane, and emoji live in a supplementary plane, so they were and remain numeric references. The comment now says what the code does. diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/ApiApproval/PublicApiApprovalTests.PublicApi_ShouldNotChangeUnexpectedly.verified.txt b/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/ApiApproval/PublicApiApprovalTests.PublicApi_ShouldNotChangeUnexpectedly.verified.txt index 217263538..2c84a9b42 100644 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/ApiApproval/PublicApiApprovalTests.PublicApi_ShouldNotChangeUnexpectedly.verified.txt +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/ApiApproval/PublicApiApprovalTests.PublicApi_ShouldNotChangeUnexpectedly.verified.txt @@ -20,6 +20,7 @@ public sealed class CacheEntry`1 : IEquatable> public sealed class CacheResult`1 : IEquatable> .ctor(T Value, IReadOnlyList DependencyKeys) Boolean FromFactory { get; init; } + Boolean IsStale { get; init; } IReadOnlyList DependencyKeys { get; init; } T Value { get; init; } Boolean Equals(CacheResult? other) @@ -39,6 +40,10 @@ public static class DeliveryCacheDependencies const String ItemsListScope = scope_items_list const String TaxonomiesListScope = scope_taxonomies_list const String TypesListScope = scope_types_list + static String ForAsset(Guid id) + static String ForItem(String codename) + static String ForTaxonomy(String codename) + static String ForType(String codename) // Kontent.Ai.Delivery.Abstractions public sealed class DeliveryCacheOptions diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/Caching/CacheDependencyKeyBuilderTests.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/Caching/CacheDependencyKeyBuilderTests.cs deleted file mode 100644 index 9079ba357..000000000 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/Caching/CacheDependencyKeyBuilderTests.cs +++ /dev/null @@ -1,49 +0,0 @@ -namespace Kontent.Ai.Delivery.Abstractions.Tests.Caching; - -public class CacheDependencyKeyBuilderTests -{ - [Theory] - [InlineData(null)] - [InlineData("")] - [InlineData(" ")] - public void BuildItemDependencyKey_WithNullOrWhitespace_ReturnsNull(string? codename) - { - var key = CacheDependencyKeyBuilder.BuildItemDependencyKey(codename); - - Assert.Null(key); - } - - [Fact] - public void BuildItemDependencyKey_WithCodename_ReturnsPrefixedKey() - { - var key = CacheDependencyKeyBuilder.BuildItemDependencyKey("article"); - - Assert.Equal("item_article", key); - } - - [Fact] - public void BuildAssetDependencyKey_ReturnsPrefixedKey() - { - var assetId = Guid.NewGuid(); - - var key = CacheDependencyKeyBuilder.BuildAssetDependencyKey(assetId); - - Assert.Equal($"asset_{assetId}", key); - } - - [Fact] - public void BuildTaxonomyDependencyKey_WithCodename_ReturnsPrefixedKey() - { - var key = CacheDependencyKeyBuilder.BuildTaxonomyDependencyKey("categories"); - - Assert.Equal("taxonomy_categories", key); - } - - [Fact] - public void BuildTypeDependencyKey_WithCodename_ReturnsPrefixedKey() - { - var key = CacheDependencyKeyBuilder.BuildTypeDependencyKey("article"); - - Assert.Equal("type_article", key); - } -} diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/Caching/DeliveryCacheDependenciesTests.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/Caching/DeliveryCacheDependenciesTests.cs new file mode 100644 index 000000000..e40d9f0d4 --- /dev/null +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/Caching/DeliveryCacheDependenciesTests.cs @@ -0,0 +1,38 @@ +namespace Kontent.Ai.Delivery.Abstractions.Tests.Caching; + +public class DeliveryCacheDependenciesTests +{ + [Fact] + public void ForItem_ComposesTheKeyTheSdkTagsWith() + => Assert.Equal("item_article", DeliveryCacheDependencies.ForItem("article")); + + [Fact] + public void ForType_ComposesTheKeyTheSdkTagsWith() + => Assert.Equal("type_article", DeliveryCacheDependencies.ForType("article")); + + [Fact] + public void ForTaxonomy_ComposesTheKeyTheSdkTagsWith() + => Assert.Equal("taxonomy_categories", DeliveryCacheDependencies.ForTaxonomy("categories")); + + [Fact] + public void ForAsset_ComposesTheKeyTheSdkTagsWith() + { + var id = Guid.Parse("A5E1C4B2-1234-5678-9ABC-DEF012345678"); + + Assert.Equal("asset_a5e1c4b2-1234-5678-9abc-def012345678", DeliveryCacheDependencies.ForAsset(id)); + } + + [Theory] + [InlineData("Article", "item_article")] + [InlineData(" article ", "item_article")] + [InlineData("ARTICLE", "item_article")] + public void ForItem_NormalizesTheCodename(string codename, string expected) + => Assert.Equal(expected, DeliveryCacheDependencies.ForItem(codename)); + + [Theory] + [InlineData(null)] + [InlineData("")] + [InlineData(" ")] + public void ForItem_RejectsAMissingCodename(string? codename) + => Assert.ThrowsAny(() => DeliveryCacheDependencies.ForItem(codename!)); +} diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/ContentItems/Processing/DependencyTrackingContextComponentTests.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/ContentItems/Processing/DependencyTrackingContextComponentTests.cs index 5174d39b4..48a76a22b 100644 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/ContentItems/Processing/DependencyTrackingContextComponentTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions.Tests/ContentItems/Processing/DependencyTrackingContextComponentTests.cs @@ -44,7 +44,7 @@ public void TrackItemType_ValidCodename_RecordsTypeDependency(string typeCodenam ctx.TrackItemType(typeCodename); Assert.Single(ctx.Dependencies); - Assert.Contains($"type_{typeCodename}", ctx.Dependencies); + Assert.Contains($"type_{typeCodename.ToLowerInvariant()}", ctx.Dependencies); } [Theory] diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheDependencyKeyBuilder.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheDependencyKeyBuilder.cs deleted file mode 100644 index 02bd442c4..000000000 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheDependencyKeyBuilder.cs +++ /dev/null @@ -1,24 +0,0 @@ -namespace Kontent.Ai.Delivery.Abstractions; - -/// -/// Builds canonical cache dependency keys used for invalidation. -/// -internal static class CacheDependencyKeyBuilder -{ - private const string ItemPrefix = "item_"; - private const string AssetPrefix = "asset_"; - private const string TaxonomyPrefix = "taxonomy_"; - private const string TypePrefix = "type_"; - - public static string? BuildItemDependencyKey(string? codename) => BuildWithPrefix(ItemPrefix, codename); - - public static string BuildAssetDependencyKey(Guid assetId) => $"{AssetPrefix}{assetId}"; - - public static string? BuildTaxonomyDependencyKey(string? taxonomyCodename) => - BuildWithPrefix(TaxonomyPrefix, taxonomyCodename); - - public static string? BuildTypeDependencyKey(string? typeCodename) => - BuildWithPrefix(TypePrefix, typeCodename); - - private static string? BuildWithPrefix(string prefix, string? value) => string.IsNullOrWhiteSpace(value) ? null : $"{prefix}{value}"; -} diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheEntry.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheEntry.cs index 11ec7339f..fcbbe2a8e 100644 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheEntry.cs +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheEntry.cs @@ -10,6 +10,8 @@ namespace Kontent.Ai.Delivery.Abstractions; /// Use standardized key formats (see remarks). /// /// -/// Return null from the factory to signal "don't cache" (e.g., on API failure). +/// Return null from the factory when the origin has no value for the key - an item that does not +/// exist, or is no longer published. Throw when the origin could not be reached, so that a cache manager +/// with fail-safe can serve a stale copy for the outage but never for the answer. /// public sealed record CacheEntry(T Value, IEnumerable Dependencies) where T : class; diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheResult.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheResult.cs index 0d90bf18e..02db6420a 100644 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheResult.cs +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/CacheResult.cs @@ -22,4 +22,11 @@ public sealed record CacheResult(T Value, IReadOnlyList DependencyKey /// which value it handed back. /// public bool FromFactory { get; init; } + + /// + /// Whether the value is a stale copy served by fail-safe, because the origin could not be reached + /// during this call or shortly before it. Only the cache knows: the factory either failed or was + /// not run, and neither says which value came back. + /// + public bool IsStale { get; init; } } diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/DeliveryCacheDependencies.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/DeliveryCacheDependencies.cs index 9996c16a6..ca85f7e03 100644 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/DeliveryCacheDependencies.cs +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/DeliveryCacheDependencies.cs @@ -1,7 +1,9 @@ namespace Kontent.Ai.Delivery.Abstractions; /// -/// Defines standard synthetic dependency keys used by the SDK cache invalidation system. +/// The dependency keys the SDK tags cached responses with, for . +/// The scope constants cover every listing of a kind; the methods compose the key for one entity the way +/// the SDK does, so a webhook handler and the cache agree on the exact string. /// public static class DeliveryCacheDependencies { @@ -22,4 +24,37 @@ public static class DeliveryCacheDependencies /// Invalidating this key clears all cached taxonomy listing queries for the current cache namespace. /// public const string TaxonomiesListScope = "scope_taxonomies_list"; + + /// + /// The key of a content item: item_{codename}. Every cached response that contains the item - + /// as the subject, in a listing, or through modular content - carries it. + /// + public static string ForItem(string codename) => $"item_{Normalize(codename)}"; + + /// + /// The key of a content type: type_{codename}. The cached type definition carries it, and so + /// does every cached response containing an item of that type. + /// + public static string ForType(string codename) => $"type_{Normalize(codename)}"; + + /// + /// The key of a taxonomy group: taxonomy_{codename}. The cached group carries it, and so does + /// every cached response whose items have a taxonomy element drawing from it. + /// + public static string ForTaxonomy(string codename) => $"taxonomy_{Normalize(codename)}"; + + /// + /// The key of an asset: asset_{id}. Every cached response whose items reference the asset, in + /// an asset element or as a rich text image, carries it. + /// + public static string ForAsset(Guid id) => $"asset_{id:D}"; + + // Codenames are lower-case by construction and the SDK compares keys ordinally, so a key composed + // from a differently-cased copy of one - a webhook payload, a hand-written constant - is normalized + // here rather than silently matching nothing. + private static string Normalize(string codename) + { + ArgumentException.ThrowIfNullOrWhiteSpace(codename); + return codename.Trim().ToLowerInvariant(); + } } diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/DeliveryCacheOptions.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/DeliveryCacheOptions.cs index 3df9f11ac..b3e81e804 100644 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/DeliveryCacheOptions.cs +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/DeliveryCacheOptions.cs @@ -17,6 +17,10 @@ namespace Kontent.Ai.Delivery.Abstractions; /// Jitter randomizes the expiration time of cache entries to prevent the /// "thundering herd" problem where many entries expire simultaneously. /// +/// +/// Under a memory cache with a SizeLimit, every entry the SDK writes counts as one unit, so the +/// limit bounds the number of cached responses rather than their bytes. +/// /// public sealed class DeliveryCacheOptions { @@ -40,6 +44,11 @@ public sealed class DeliveryCacheOptions /// When enabled, the cache returns stale (expired) entries when the data source /// is unavailable, providing resilience during API outages. /// + /// + /// An outage is a request that got no response, or a status the SDK's own pipeline retries - a + /// timeout, 429 or 5xx. An answer from the API is never covered: an item that comes + /// back 404 after being unpublished is dropped from the cache, not served stale. + /// /// Defaults to . public bool IsFailSafeEnabled { get; set; } @@ -92,8 +101,13 @@ public sealed class DeliveryCacheOptions /// or other settings not directly exposed by the SDK. /// /// - /// The SDK applies its defaults first, then invokes this callback, allowing you to - /// override or extend any setting. + /// The SDK applies its defaults first, then invokes this callback. Every operation the SDK runs starts + /// from the DefaultEntryOptions the callback leaves behind, so a Size, a distributed-cache + /// timeout or the background-operation flags set there take effect; the duration, fail-safe, jitter + /// and eager-refresh policy come from this object regardless, serialization failures are always thrown, + /// and distributed-cache and backplane failures never are. TagsDefaultEntryOptions is what an + /// invalidation is stored with: its duration, ten days by default, is how long an invalidation is + /// remembered for an entry that has not been read since, and must exceed your longest expiration. /// /// /// Prefer the ConfigureFusionCache extension from Kontent.Ai.Delivery.Caching, which takes diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/IDeliveryCacheManager.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/IDeliveryCacheManager.cs index a9219f4e2..bbbf4411e 100644 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/IDeliveryCacheManager.cs +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions/Caching/IDeliveryCacheManager.cs @@ -17,7 +17,8 @@ namespace Kontent.Ai.Delivery.Abstractions; /// all cache entries that reference them. /// /// -/// Dependency key format conventions: +/// Dependency key format conventions - compose them with rather +/// than by hand: /// /// Content items: item_{codename} (e.g., "item_hero") /// Assets: asset_{guid} (e.g., "asset_a5e1c4b2-...") @@ -58,9 +59,12 @@ public interface IDeliveryCacheManager /// The type of the cached value. /// The unique key identifying the cache entry. /// - /// A factory function invoked on cache miss. Returns a - /// containing the value and its dependency tags, or null to signal "don't cache" - /// (e.g., when an API call fails). + /// A factory function invoked on cache miss. Returns a containing the + /// value and its dependency tags, or null when the origin has no value for this key - nothing + /// is cached, and an implementation that keeps stale copies for fail-safe drops the one it holds, + /// since the answer supersedes it. It throws when the origin could not be reached; an + /// implementation with fail-safe may then serve a stale copy, and one without lets the exception + /// propagate. /// /// /// Optional absolute expiration timespan. If null, the implementation's default expiration is used. @@ -76,8 +80,8 @@ public interface IDeliveryCacheManager /// result in at most one factory invocation (stampede protection). /// /// - /// When the factory returns null, the result should not be cached and - /// null should be returned to the caller. + /// When the factory returns null, nothing is cached, any stale copy of the key is removed, and + /// null is returned to the caller. /// /// Task?> GetOrSetAsync( @@ -90,8 +94,9 @@ public interface IDeliveryCacheManager /// Invalidates all cache entries that depend on the specified dependency keys. /// /// - /// One or more dependency keys to invalidate. All cache entries referencing any of these keys - /// will be removed from the cache. + /// One or more dependency keys to invalidate, composed with . + /// All cache entries referencing any of these keys will be removed from the cache. Keys are matched + /// case-insensitively. /// /// A token to cancel the asynchronous operation. /// diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions/ContentItems/Processing/DependencyTrackingContext.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions/ContentItems/Processing/DependencyTrackingContext.cs index 9db250a7b..6620c9634 100644 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions/ContentItems/Processing/DependencyTrackingContext.cs +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions/ContentItems/Processing/DependencyTrackingContext.cs @@ -74,20 +74,9 @@ public IEnumerable Dependencies /// public void TrackItem(string? codename) { - if (string.IsNullOrWhiteSpace(codename)) + if (!string.IsNullOrWhiteSpace(codename)) { - return; - } - - var dependencyKey = CacheDependencyKeyBuilder.BuildItemDependencyKey(codename); - if (dependencyKey is null) - { - return; - } - - lock (_lock) - { - _dependencies.Add(dependencyKey); + Add(DeliveryCacheDependencies.ForItem(codename)); } } @@ -105,15 +94,9 @@ public void TrackItem(string? codename) /// public void TrackItemType(string? typeCodename) { - var dependencyKey = CacheDependencyKeyBuilder.BuildTypeDependencyKey(typeCodename); - if (dependencyKey is null) - { - return; - } - - lock (_lock) + if (!string.IsNullOrWhiteSpace(typeCodename)) { - _dependencies.Add(dependencyKey); + Add(DeliveryCacheDependencies.ForType(typeCodename)); } } @@ -134,15 +117,7 @@ public void TrackItemType(string? typeCodename) /// Duplicate calls with the same asset ID are ignored. /// /// - public void TrackAsset(Guid assetId) - { - var dependencyKey = CacheDependencyKeyBuilder.BuildAssetDependencyKey(assetId); - - lock (_lock) - { - _dependencies.Add(dependencyKey); - } - } + public void TrackAsset(Guid assetId) => Add(DeliveryCacheDependencies.ForAsset(assetId)); /// /// Tracks a dependency on a taxonomy group by its codename. @@ -165,12 +140,14 @@ public void TrackAsset(Guid assetId) /// public void TrackTaxonomy(string? taxonomyGroup) { - var dependencyKey = CacheDependencyKeyBuilder.BuildTaxonomyDependencyKey(taxonomyGroup); - if (dependencyKey is null) + if (!string.IsNullOrWhiteSpace(taxonomyGroup)) { - return; + Add(DeliveryCacheDependencies.ForTaxonomy(taxonomyGroup)); } + } + private void Add(string dependencyKey) + { lock (_lock) { _dependencies.Add(dependencyKey); diff --git a/src/delivery/Kontent.Ai.Delivery.Abstractions/SharedModels/IDeliveryResult.cs b/src/delivery/Kontent.Ai.Delivery.Abstractions/SharedModels/IDeliveryResult.cs index 1d3676e8d..ae00974fc 100644 --- a/src/delivery/Kontent.Ai.Delivery.Abstractions/SharedModels/IDeliveryResult.cs +++ b/src/delivery/Kontent.Ai.Delivery.Abstractions/SharedModels/IDeliveryResult.cs @@ -54,7 +54,7 @@ public interface IDeliveryResult /// /// Gets a value indicating whether this result was served from the SDK's local cache - /// (MemoryCacheManager or HybridCacheManager). + /// (the memory or hybrid cache). /// Equivalent to is or . /// When true, will be null and properties like /// and contain synthetic values. diff --git a/src/delivery/Kontent.Ai.Delivery.Caching/Extensions/DeliveryCacheOptionsExtensions.cs b/src/delivery/Kontent.Ai.Delivery.Caching/Extensions/DeliveryCacheOptionsExtensions.cs index ea0509dc2..e94be5938 100644 --- a/src/delivery/Kontent.Ai.Delivery.Caching/Extensions/DeliveryCacheOptionsExtensions.cs +++ b/src/delivery/Kontent.Ai.Delivery.Caching/Extensions/DeliveryCacheOptionsExtensions.cs @@ -17,7 +17,12 @@ public static class DeliveryCacheOptionsExtensions /// references it, so the cast belongs here rather than in every caller. /// /// The cache options to configure. - /// Receives the after the SDK's defaults are applied. + /// + /// Receives the after the SDK's defaults are applied. Every SDK + /// operation starts from the it leaves behind, + /// and every invalidation is stored with its ; + /// see for what the SDK pins. + /// /// The same instance, for chaining. /// /// diff --git a/src/delivery/Kontent.Ai.Delivery.Caching/Extensions/DeliveryClientBuilderCachingExtensions.cs b/src/delivery/Kontent.Ai.Delivery.Caching/Extensions/DeliveryClientBuilderCachingExtensions.cs index cde9617ef..363dd137e 100644 --- a/src/delivery/Kontent.Ai.Delivery.Caching/Extensions/DeliveryClientBuilderCachingExtensions.cs +++ b/src/delivery/Kontent.Ai.Delivery.Caching/Extensions/DeliveryClientBuilderCachingExtensions.cs @@ -4,8 +4,10 @@ using Microsoft.Extensions.Caching.Distributed; using Microsoft.Extensions.Caching.Memory; using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.DependencyInjection.Extensions; using Microsoft.Extensions.Logging; using Microsoft.Extensions.Options; +using ZiggyCreatures.Caching.Fusion; namespace Kontent.Ai.Delivery; @@ -90,27 +92,39 @@ private static IDeliveryClientBuilder UseMemoryCacheCore(IDeliveryClientBuilder // Register IMemoryCache if not already registered (shared across all clients) builder.Services.AddMemoryCache(); - return RegisterCacheManager(builder, sp => new MemoryCacheManager( + return RegisterCacheManager(builder, sp => FusionCacheManager.CreateMemory( sp.GetRequiredService(), cacheOptionsFactory(sp), - sp.GetService>(), - EnvironmentIdOf(sp, builder.Name))); + sp.GetService>(), + EnvironmentIdOf(sp, builder.Name), + sp.GetService>())); } private static IDeliveryClientBuilder UseHybridCacheCore(IDeliveryClientBuilder builder, Func cacheOptionsFactory) - => RegisterCacheManager(builder, sp => new HybridCacheManager( + => RegisterCacheManager(builder, sp => FusionCacheManager.CreateHybrid( sp.GetRequiredService(), cacheOptionsFactory(sp), - logger: sp.GetService>(), + logger: sp.GetService>(), // Registered by the consumer the usual FusionCache way, e.g. // services.AddFusionCacheStackExchangeRedisBackplane(...). backplane: sp.GetService(), - environmentId: EnvironmentIdOf(sp, builder.Name))); + environmentId: EnvironmentIdOf(sp, builder.Name), + // FusionCache's own diagnostics: a distributed cache it worked around, a backplane publish + // that failed, a background refresh that threw. Without it those are silent. + fusionCacheLogger: sp.GetService>())); private static IDeliveryClientBuilder RegisterCacheManager(IDeliveryClientBuilder builder, Func createCacheManager) { RemoveExistingCacheManagerRegistration(builder.Services, builder.Name); builder.Services.AddKeyedSingleton(builder.Name, (sp, _) => createCacheManager(sp)); + + // The default client's manager resolves unkeyed as well, the way the client itself does, so a + // webhook handler asks for IDeliveryCacheManager and never learns the default client's key. + if (builder.Name == NamedClients.Default) + { + builder.Services.TryAddSingleton(sp => sp.GetRequiredKeyedService(NamedClients.Default)); + } + return builder; } diff --git a/src/delivery/Kontent.Ai.Delivery.Caching/FusionCacheManager.cs b/src/delivery/Kontent.Ai.Delivery.Caching/FusionCacheManager.cs index 47f3ea1cf..54b966fa7 100644 --- a/src/delivery/Kontent.Ai.Delivery.Caching/FusionCacheManager.cs +++ b/src/delivery/Kontent.Ai.Delivery.Caching/FusionCacheManager.cs @@ -1,4 +1,3 @@ -using System.Collections.Concurrent; using System.Text.Json; using Kontent.Ai.Delivery.Configuration; using Kontent.Ai.Delivery.Logging; @@ -13,64 +12,73 @@ namespace Kontent.Ai.Delivery.Caching; /// -/// Shared FusionCache-backed implementation of SDK cache manager behavior. +/// The SDK's cache manager over FusionCache, in one of two shapes: caches +/// hydrated objects in the application's memory cache, caches raw payloads in +/// a memory tier of its own in front of a distributed cache. /// -internal sealed class FusionCacheManager : IDeliveryCacheManager, IDeliveryCachePurger, IFailSafeStateProvider, IDisposable +internal sealed class FusionCacheManager : IDeliveryCacheManager, IDeliveryCachePurger, IDisposable { private readonly IFusionCache _cache; private readonly CacheStorageMode _storageMode; private readonly TimeSpan _defaultExpiration; - private readonly Func _cacheKeyFormatter; - private readonly Func _dependencyTagFormatter; + private readonly string _keyPrefix; private readonly ILogger? _logger; private readonly FusionCacheEntryOptions _baseWriteOptions; - private readonly FusionCacheEntryOptions _baseInvalidateOptions; - private readonly ConcurrentDictionary _failSafeActiveKeys = new(StringComparer.Ordinal); /// - /// Hard cap for . In hybrid mode (L2-only, no L1 memory cache), - /// memory eviction events never fire, so entries can accumulate if they enter fail-safe but are - /// never re-requested or invalidated. Clearing at this threshold is safe because - /// is metadata-only (affects ResponseSource, - /// not correctness) and stale entries will be re-tracked on the next stale hit. + /// What every entry weighs under a . The application's + /// memory cache may have one, and a cache with a limit refuses any entry that declares no size - so + /// every entry the SDK writes, tag-expiration entries included, declares this one. /// - private const int FailSafeTrackingCapacity = 10_000; + private const long EntrySize = 1; + + /// + /// The shape of what the distributed tier stores, as a key segment. A Redis outlives a deployment, so + /// an entry written by the previous version of the SDK is read by the next one; a key that names the + /// shape lets that read miss instead of failing to deserialize. Bump it whenever a cached type - + /// , CachedRawItemsPayload, a wire model - or FusionCache's own + /// distributed entry format changes. + /// + private const string DistributedFormatVersion = "v1:"; + + /// + /// The call in flight on the current async context. FusionCache raises + /// its events inline (), so a stale + /// hit or a fail-safe activation it reports while a call awaits it belongs to that call, and the + /// handlers record it there. Nothing outlives the call: no shared state, nothing to cap or evict. + /// + private static readonly AsyncLocal CurrentCall = new(); + private readonly EventHandler _failSafeActivateHandler; - private readonly EventHandler _factorySuccessHandler; private readonly EventHandler _hitHandler; - private readonly EventHandler _removeHandler; - private readonly EventHandler _evictionHandler; private int _disposeState; private FusionCacheManager( IFusionCache cache, CacheStorageMode storageMode, TimeSpan defaultExpiration, - Func cacheKeyFormatter, - Func dependencyTagFormatter, + string keyPrefix, ILogger? logger, - FusionCacheEntryOptions baseWriteOptions, - FusionCacheEntryOptions baseInvalidateOptions) + FusionCacheEntryOptions baseWriteOptions) { _cache = cache; _storageMode = storageMode; _defaultExpiration = defaultExpiration; - _cacheKeyFormatter = cacheKeyFormatter; - _dependencyTagFormatter = dependencyTagFormatter; + _keyPrefix = keyPrefix; _logger = logger; _baseWriteOptions = baseWriteOptions; - _baseInvalidateOptions = baseInvalidateOptions; _failSafeActivateHandler = HandleFailSafeActivate; - _factorySuccessHandler = HandleFactorySuccess; _hitHandler = HandleHit; - _removeHandler = HandleRemove; - _evictionHandler = HandleEviction; - SubscribeFailSafeStateEvents(); + _cache.Events.FailSafeActivate += _failSafeActivateHandler; + _cache.Events.Hit += _hitHandler; } /// - /// Builds the segment every cache key and dependency tag is prefixed with. + /// Builds the segment this client's cache lives under. It goes to FusionCache as its + /// , so every key FusionCache stores carries it - the + /// entries, the tag data an invalidation writes, and the marker a purge writes - and two clients + /// sharing one store cannot reach each other's, purges included. /// /// /// The environment id is part of it because a cache store can outlive the process and be shared: two @@ -85,76 +93,50 @@ private static string ComposeKeyPrefix(string? keyPrefix, string? environmentId) return parts.Any() ? $"{string.Join(':', parts)}:" : string.Empty; } + /// + /// Names the FusionCache instance, and through it the backplane channel: clients on different + /// environments must not share one, or every node hears every other environment's notifications. + /// + private static string ComposeCacheName(string tier, string keyPrefix) + => $"KontentDelivery.{tier}.{(keyPrefix.Length == 0 ? "Default" : keyPrefix.TrimEnd(':'))}"; + public static FusionCacheManager CreateMemory( IMemoryCache memoryCache, DeliveryCacheOptions cacheOptions, ILogger? logger = null, - string? environmentId = null) + string? environmentId = null, + ILogger? fusionCacheLogger = null) { ArgumentNullException.ThrowIfNull(memoryCache); ArgumentNullException.ThrowIfNull(cacheOptions); - var effectiveExpiration = cacheOptions.DefaultExpiration; - var keyPrefix = cacheOptions.KeyPrefix; - var prefixSegment = ComposeKeyPrefix(keyPrefix, environmentId); - - var defaultEntryOptions = new FusionCacheEntryOptions - { - AllowBackgroundDistributedCacheOperations = false, - AllowBackgroundBackplaneOperations = false, - ReThrowDistributedCacheExceptions = false, - ReThrowSerializationExceptions = true, - ReThrowBackplaneExceptions = false - }; - ApplyCachePolicy(defaultEntryOptions, cacheOptions, effectiveExpiration); + var keyPrefix = ComposeKeyPrefix(cacheOptions.KeyPrefix, environmentId); var fusionCacheOptions = new FusionCacheOptions { - CacheName = $"KontentDelivery.Memory.{(string.IsNullOrWhiteSpace(keyPrefix) ? "Default" : keyPrefix)}", + CacheName = ComposeCacheName("Memory", keyPrefix), + CacheKeyPrefix = keyPrefix, DistributedCacheKeyModifierMode = CacheKeyModifierMode.None, // Required for deterministic fail-safe source propagation in query builders. EnableSyncEventHandlersExecution = true, - DefaultEntryOptions = defaultEntryOptions + DefaultEntryOptions = EntryDefaults(cacheOptions, memoryOnly: true) }; + ConfigureTagEntries(fusionCacheOptions.TagsDefaultEntryOptions, memoryOnly: true); cacheOptions.ConfigureFusionCacheOptions?.Invoke(fusionCacheOptions); var fusion = new FusionCache( Options.Create(fusionCacheOptions), memoryCache, - logger: null); - - var baseWriteOptions = new FusionCacheEntryOptions - { - SkipDistributedCacheRead = true, - SkipDistributedCacheWrite = true, - ReThrowDistributedCacheExceptions = false, - ReThrowSerializationExceptions = true, - ReThrowBackplaneExceptions = false, - AllowBackgroundBackplaneOperations = false, - AllowBackgroundDistributedCacheOperations = false - }; - ApplyCachePolicy(baseWriteOptions, cacheOptions, effectiveExpiration); + fusionCacheLogger); return new FusionCacheManager( fusion, CacheStorageMode.HydratedObject, - effectiveExpiration, - cacheKey => $"{prefixSegment}{cacheKey}", - dependency => $"{prefixSegment}{dependency}", + cacheOptions.DefaultExpiration, + fusionCacheOptions.CacheKeyPrefix ?? string.Empty, logger, - baseWriteOptions: baseWriteOptions, - baseInvalidateOptions: new FusionCacheEntryOptions - { - IsFailSafeEnabled = false, - SkipDistributedCacheRead = true, - SkipDistributedCacheWrite = true, - ReThrowDistributedCacheExceptions = false, - ReThrowSerializationExceptions = false, - ReThrowBackplaneExceptions = false, - AllowBackgroundBackplaneOperations = false, - AllowBackgroundDistributedCacheOperations = false - }); + WriteOptions(fusionCacheOptions, cacheOptions, memoryOnly: true)); } /// @@ -173,40 +155,38 @@ public static FusionCacheManager CreateHybrid( JsonSerializerOptions? serializerOptions = null, ILogger? logger = null, IFusionCacheBackplane? backplane = null, - string? environmentId = null) + string? environmentId = null, + ILogger? fusionCacheLogger = null) { ArgumentNullException.ThrowIfNull(distributedCache); ArgumentNullException.ThrowIfNull(cacheOptions); - var effectiveExpiration = cacheOptions.DefaultExpiration; - var keyPrefix = cacheOptions.KeyPrefix; - var prefixSegment = ComposeKeyPrefix(keyPrefix, environmentId); - - var defaultEntryOptions = new FusionCacheEntryOptions - { - AllowBackgroundDistributedCacheOperations = false, - AllowBackgroundBackplaneOperations = false, - ReThrowDistributedCacheExceptions = false, - ReThrowSerializationExceptions = true, - ReThrowBackplaneExceptions = false - }; - ApplyCachePolicy(defaultEntryOptions, cacheOptions, effectiveExpiration); + var keyPrefix = ComposeKeyPrefix(cacheOptions.KeyPrefix, environmentId); var fusionCacheOptions = new FusionCacheOptions { - CacheName = $"KontentDelivery.Hybrid.{(string.IsNullOrWhiteSpace(keyPrefix) ? "Default" : keyPrefix)}", + CacheName = ComposeCacheName("Hybrid", keyPrefix), + // The version sits inside the prefix so that FusionCache's own keys carry it too. + CacheKeyPrefix = keyPrefix + DistributedFormatVersion, + // FusionCache would otherwise version the distributed keys itself; DistributedFormatVersion + // covers its wire format as well, and keeps the keys readable. DistributedCacheKeyModifierMode = CacheKeyModifierMode.None, // Required for deterministic fail-safe source propagation in query builders. EnableSyncEventHandlersExecution = true, - DefaultEntryOptions = defaultEntryOptions + // A distributed cache that is down is worked around, not retried on every request: while the + // breaker is open the memory tier and the origin carry the load, and FusionCache re-syncs the + // distributed tier when it comes back. + DistributedCacheCircuitBreakerDuration = TimeSpan.FromSeconds(2), + DefaultEntryOptions = EntryDefaults(cacheOptions, memoryOnly: false) }; + ConfigureTagEntries(fusionCacheOptions.TagsDefaultEntryOptions, memoryOnly: false); cacheOptions.ConfigureFusionCacheOptions?.Invoke(fusionCacheOptions); var fusion = new FusionCache( Options.Create(fusionCacheOptions), memoryCache: null, - logger: null); + fusionCacheLogger); // Falls back to the SDK's own serializer rather than plain defaults. What this tier stores is wire // types, and content type elements are polymorphic: without ContentElementConverter the L2 payload @@ -221,33 +201,97 @@ public static FusionCacheManager CreateHybrid( fusion.SetupBackplane(backplane); } - var baseWriteOptions = new FusionCacheEntryOptions - { - ReThrowDistributedCacheExceptions = true, - ReThrowSerializationExceptions = true, - ReThrowBackplaneExceptions = false, - AllowBackgroundBackplaneOperations = false, - AllowBackgroundDistributedCacheOperations = false - }; - ApplyCachePolicy(baseWriteOptions, cacheOptions, effectiveExpiration); - return new FusionCacheManager( fusion, CacheStorageMode.RawJson, - effectiveExpiration, - cacheKey => $"{prefixSegment}cache:{cacheKey}", - dependency => $"{prefixSegment}dep:{dependency}", + cacheOptions.DefaultExpiration, + fusionCacheOptions.CacheKeyPrefix ?? string.Empty, logger, - baseWriteOptions: baseWriteOptions, - baseInvalidateOptions: new FusionCacheEntryOptions - { - IsFailSafeEnabled = false, - ReThrowDistributedCacheExceptions = false, - ReThrowSerializationExceptions = false, - ReThrowBackplaneExceptions = false, - AllowBackgroundBackplaneOperations = false, - AllowBackgroundDistributedCacheOperations = false - }); + WriteOptions(fusionCacheOptions, cacheOptions, memoryOnly: false)); + } + + /// + /// The entry options the SDK starts from. They are what the consumer's + /// callback sees as + /// , and every write starts from whatever the + /// callback leaves there. + /// + private static FusionCacheEntryOptions EntryDefaults(DeliveryCacheOptions cacheOptions, bool memoryOnly) + { + var options = new FusionCacheEntryOptions + { + Size = EntrySize, + AllowBackgroundDistributedCacheOperations = false, + AllowBackgroundBackplaneOperations = false + }; + + return Pin(options, cacheOptions, memoryOnly); + } + + /// + /// The options every write passes: the consumer's + /// after the callback ran, with what the SDK decides re-applied on top. Anything else set there - a + /// , background backplane operations, a distributed-cache + /// timeout - reaches the SDK's reads and writes. + /// + private static FusionCacheEntryOptions WriteOptions(FusionCacheOptions fusionCacheOptions, DeliveryCacheOptions cacheOptions, bool memoryOnly) + => Pin(fusionCacheOptions.DefaultEntryOptions.Duplicate(), cacheOptions, memoryOnly); + + /// + /// What the SDK decides regardless of the consumer's entry options: the tier skips in memory mode, + /// which failures are thrown, and the timing and fail-safe policy that + /// owns. + /// + /// + /// A distributed-cache failure is worked around - the factory or the memory tier answers and + /// FusionCache logs it - rather than thrown out of every cached query. A serialization failure is + /// still thrown: that is a defect in the SDK's own payloads, and hiding it would hide the defect. + /// + private static FusionCacheEntryOptions Pin(FusionCacheEntryOptions options, DeliveryCacheOptions cacheOptions, bool memoryOnly) + { + options.SkipDistributedCacheRead = memoryOnly; + options.SkipDistributedCacheWrite = memoryOnly; + options.ReThrowDistributedCacheExceptions = false; + options.ReThrowSerializationExceptions = true; + options.ReThrowBackplaneExceptions = false; + + options.Duration = cacheOptions.DefaultExpiration; + options.IsFailSafeEnabled = cacheOptions.IsFailSafeEnabled; + options.FailSafeMaxDuration = cacheOptions.FailSafeMaxDuration; + options.FailSafeThrottleDuration = cacheOptions.FailSafeThrottleDuration; + options.JitterMaxDuration = cacheOptions.JitterMaxDuration; + + if (cacheOptions.EagerRefreshThreshold > 0) + { + options.EagerRefreshThreshold = cacheOptions.EagerRefreshThreshold; + } + + return options; + } + + /// + /// The options RemoveByTag and Clear run with, which is what the tag-expiration entries + /// they write are stored with. + /// + /// + /// The is deliberately left at FusionCache's own + /// default for tag data, ten days: it is how long an invalidation is remembered for an entry that + /// has not been read since, so it has to outlive every entry it could apply to. Passing write options + /// here instead would store the tag data for the entries' duration - with a bare + /// FusionCacheEntryOptions, for thirty seconds - and a webhook's invalidation would be + /// forgotten before a quiet entry was next read. + /// + private static void ConfigureTagEntries(FusionCacheEntryOptions tagOptions, bool memoryOnly) + { + tagOptions.Size = EntrySize; + tagOptions.IsFailSafeEnabled = false; + tagOptions.SkipDistributedCacheRead = memoryOnly; + tagOptions.SkipDistributedCacheWrite = memoryOnly; + tagOptions.ReThrowDistributedCacheExceptions = false; + tagOptions.ReThrowSerializationExceptions = false; + tagOptions.ReThrowBackplaneExceptions = false; + tagOptions.AllowBackgroundDistributedCacheOperations = false; + tagOptions.AllowBackgroundBackplaneOperations = false; } public CacheStorageMode StorageMode => _storageMode; @@ -259,54 +303,45 @@ public static FusionCacheManager CreateHybrid( CancellationToken cancellationToken = default) where T : class { + ArgumentException.ThrowIfNullOrWhiteSpace(cacheKey); + ArgumentNullException.ThrowIfNull(factory); ThrowIfDisposed(); - if (string.IsNullOrWhiteSpace(cacheKey)) - { - var entry = await factory(cancellationToken).ConfigureAwait(false); - if (entry is null) - return null; - - var deps = entry.Dependencies - .Where(d => !string.IsNullOrWhiteSpace(d)) - .Distinct(StringComparer.OrdinalIgnoreCase) - .ToArray(); - return new CacheResult(entry.Value, deps) { FromFactory = true }; - } - - var formattedKey = _cacheKeyFormatter(cacheKey); - // The only reliable way to tell whether the value we get back was produced by *this* call: // FusionCache runs the factory on a background thread for eager refresh while returning the // stale value immediately, so a flag set inside the factory says nothing about which call it // belongs to. The envelope instance does - it comes back only if this invocation produced it. CacheEnvelope? producedHere = null; + var observation = new StaleObservation(cacheKey, _keyPrefix + cacheKey); + CurrentCall.Value = observation; try { var envelope = await _cache.GetOrSetAsync>( - formattedKey, + cacheKey, async (ctx, ct) => { var factoryResult = await factory(ct).ConfigureAwait(false); if (factoryResult is null) { + // The origin has no value for this key. Fail-safe is for an origin that cannot be + // reached, which arrives as a thrown exception; an answer must not be papered over + // with a stale copy, so this call opts out of it and the copy is removed below. + ctx.Options.IsFailSafeEnabled = false; throw new CacheFactoryFailedException(); } // Dependency keys serve two purposes: - // 1. FusionCache tags (formatted via _dependencyTagFormatter) — used for cache invalidation. + // 1. FusionCache tags — used for cache invalidation. // 2. Stored in CacheEnvelope alongside the value — surfaced to consumers via CacheResult. var deps = factoryResult.Dependencies .Where(d => !string.IsNullOrWhiteSpace(d)) .Distinct(StringComparer.OrdinalIgnoreCase) .ToArray(); - ctx.Tags = Array.ConvertAll(deps, _dependencyTagFormatter.Invoke); + ctx.Tags = deps; ctx.Options.Duration = expiration ?? _defaultExpiration; - _failSafeActiveKeys.TryRemove(formattedKey, out var _); - _failSafeActiveKeys.TryRemove(cacheKey, out var _); return producedHere = new CacheEnvelope(factoryResult.Value, deps); }, _baseWriteOptions, @@ -315,36 +350,53 @@ public static FusionCacheManager CreateHybrid( if (envelope is null) return null; + var fromFactory = ReferenceEquals(envelope, producedHere); + return new CacheResult(envelope.Value, envelope.DependencyKeys) { - FromFactory = ReferenceEquals(envelope, producedHere), + FromFactory = fromFactory, + IsStale = !fromFactory && observation.Stale, }; } catch (CacheFactoryFailedException) { - // Factory returned null and no stale entry was available for fail-safe. - _failSafeActiveKeys.TryRemove(formattedKey, out var _); + // Opting out of fail-safe keeps the stale copy in the store, where the next call with + // fail-safe on would find it - so it goes explicitly. + await _cache.RemoveAsync(cacheKey, _baseWriteOptions, cancellationToken).ConfigureAwait(false); return null; } - catch + finally { - // Factory threw and no stale entry was available for fail-safe. - _failSafeActiveKeys.TryRemove(formattedKey, out var _); - throw; + CurrentCall.Value = null; } } /// - /// Sentinel exception thrown inside the FusionCache factory when the upstream - /// factory returns null. This allows FusionCache fail-safe to kick in - /// and serve a stale entry when one is available. The exception never leaves - /// — it is caught immediately after the - /// GetOrSetAsync call. + /// Sentinel thrown inside the FusionCache factory when the upstream factory returns null, so + /// FusionCache abandons the call without storing anything. It never leaves + /// - it is caught immediately after the GetOrSetAsync call. /// #pragma warning disable S3871 // Intentionally private sentinel — never leaves this class private sealed class CacheFactoryFailedException : Exception; #pragma warning restore S3871 + /// + /// What the event handlers record for the call in flight: whether FusionCache served it a stale + /// value. The key is compared in both the form the SDK passed and the form FusionCache stores. + /// + private sealed class StaleObservation(string key, string prefixedKey) + { + public bool Stale { get; private set; } + + public void Observe(string eventKey) + { + if (eventKey == key || eventKey == prefixedKey) + { + Stale = true; + } + } + } + public async Task InvalidateAsync(string[] dependencyKeys, CancellationToken cancellationToken = default) { ThrowIfDisposed(); @@ -354,18 +406,22 @@ public async Task InvalidateAsync(string[] dependencyKeys, CancellationTok return true; } - var validKeys = Array.TrueForAll(dependencyKeys, k => !string.IsNullOrWhiteSpace(k)) - ? dependencyKeys - : dependencyKeys.Where(k => !string.IsNullOrWhiteSpace(k)).ToArray(); + // Tags are compared ordinally and the SDK composes them lower-case; a key that arrives in another + // casing - copied from a webhook payload, say - would otherwise silently match nothing. + var validKeys = dependencyKeys + .Where(k => !string.IsNullOrWhiteSpace(k)) + .Select(k => k.Trim().ToLowerInvariant()) + .ToArray(); if (_logger is not null && validKeys.Length > 0) LoggerMessages.CacheInvalidateStarting(_logger, validKeys.Length); try { + // No options: FusionCache then uses TagsDefaultEntryOptions, which ConfigureTagEntries set up. await _cache.RemoveByTagAsync( - validKeys.Select(_dependencyTagFormatter), - _baseInvalidateOptions, + validKeys, + options: null, cancellationToken) .ConfigureAwait(false); @@ -399,28 +455,9 @@ public async Task PurgeAsync(bool allowFailSafe = false, CancellationToken cance await _cache.ClearAsync( allowFailSafe, - _baseInvalidateOptions, + options: null, cancellationToken) .ConfigureAwait(false); - - // Only clear fail-safe tracking when entries are permanently removed. - // When allowFailSafe is true, entries remain for fail-safe and should - // continue to be reported as ResponseSource.FailSafe. - if (!allowFailSafe) - { - _failSafeActiveKeys.Clear(); - } - } - - bool IFailSafeStateProvider.IsFailSafeActive(string cacheKey) - { - if (string.IsNullOrWhiteSpace(cacheKey)) - { - return false; - } - - return _failSafeActiveKeys.ContainsKey(cacheKey) - || _failSafeActiveKeys.ContainsKey(_cacheKeyFormatter(cacheKey)); } public void Dispose() @@ -428,77 +465,22 @@ public void Dispose() if (Interlocked.Exchange(ref _disposeState, 1) != 0) return; - UnsubscribeFailSafeStateEvents(); + _cache.Events.FailSafeActivate -= _failSafeActivateHandler; + _cache.Events.Hit -= _hitHandler; _cache.Dispose(); } private void ThrowIfDisposed() => ObjectDisposedException.ThrowIf(Volatile.Read(ref _disposeState) != 0, nameof(FusionCacheManager)); - /// - /// Applies fail-safe, jitter, and eager-refresh policy from - /// to a instance. - /// - private static void ApplyCachePolicy( - FusionCacheEntryOptions options, - DeliveryCacheOptions cacheOptions, - TimeSpan duration) - { - options.Duration = duration; - options.IsFailSafeEnabled = cacheOptions.IsFailSafeEnabled; - options.FailSafeMaxDuration = cacheOptions.FailSafeMaxDuration; - options.FailSafeThrottleDuration = cacheOptions.FailSafeThrottleDuration; - options.JitterMaxDuration = cacheOptions.JitterMaxDuration; - - if (cacheOptions.EagerRefreshThreshold > 0) - { - options.EagerRefreshThreshold = cacheOptions.EagerRefreshThreshold; - } - } - - private void SubscribeFailSafeStateEvents() - { - _cache.Events.FailSafeActivate += _failSafeActivateHandler; - _cache.Events.FactorySuccess += _factorySuccessHandler; - _cache.Events.Hit += _hitHandler; - _cache.Events.Remove += _removeHandler; - _cache.Events.Memory.Eviction += _evictionHandler; - } - - private void UnsubscribeFailSafeStateEvents() - { - _cache.Events.FailSafeActivate -= _failSafeActivateHandler; - _cache.Events.FactorySuccess -= _factorySuccessHandler; - _cache.Events.Hit -= _hitHandler; - _cache.Events.Remove -= _removeHandler; - _cache.Events.Memory.Eviction -= _evictionHandler; - } - - private void HandleFailSafeActivate(object? sender, FusionCacheEntryEventArgs eventArgs) - { - if (_failSafeActiveKeys.Count >= FailSafeTrackingCapacity) - _failSafeActiveKeys.Clear(); - - _failSafeActiveKeys[eventArgs.Key] = 1; - } - - private void HandleFactorySuccess(object? sender, FusionCacheEntryEventArgs eventArgs) - => _failSafeActiveKeys.TryRemove(eventArgs.Key, out var _); + private static void HandleFailSafeActivate(object? sender, FusionCacheEntryEventArgs eventArgs) + => CurrentCall.Value?.Observe(eventArgs.Key); - private void HandleHit(object? sender, FusionCacheEntryHitEventArgs eventArgs) + private static void HandleHit(object? sender, FusionCacheEntryHitEventArgs eventArgs) { if (eventArgs.IsStale) { - _failSafeActiveKeys[eventArgs.Key] = 1; - return; + CurrentCall.Value?.Observe(eventArgs.Key); } - - _failSafeActiveKeys.TryRemove(eventArgs.Key, out var _); } - - private void HandleRemove(object? sender, FusionCacheEntryEventArgs eventArgs) - => _failSafeActiveKeys.TryRemove(eventArgs.Key, out var _); - - private void HandleEviction(object? sender, FusionCacheEntryEvictionEventArgs eventArgs) - => _failSafeActiveKeys.TryRemove(eventArgs.Key, out var _); } diff --git a/src/delivery/Kontent.Ai.Delivery.Caching/HybridCacheManager.cs b/src/delivery/Kontent.Ai.Delivery.Caching/HybridCacheManager.cs deleted file mode 100644 index fd39799dd..000000000 --- a/src/delivery/Kontent.Ai.Delivery.Caching/HybridCacheManager.cs +++ /dev/null @@ -1,59 +0,0 @@ -using System.Text.Json; -using Microsoft.Extensions.Caching.Distributed; -using Microsoft.Extensions.Logging; - -namespace Kontent.Ai.Delivery.Caching; - -/// -/// Distributed implementation of backed by FusionCache. -/// -/// -/// FusionCache always has a memory tier in front of the distributed one; there is no distributed-only -/// mode, and this manager uses both tiers. The distributed one is what another node reads from, and a -/// backplane is what keeps the memory tiers in step: without one an invalidation reaches only the node -/// that performed it, so a second instance can go on serving content a webhook already evicted until the -/// entry expires. Multi-node deployments therefore need an . -/// -internal sealed class HybridCacheManager( - IDistributedCache cache, - DeliveryCacheOptions cacheOptions, - JsonSerializerOptions? jsonSerializerOptions = null, - ILogger? logger = null, - IFusionCacheBackplane? backplane = null, - string? environmentId = null) - : IDeliveryCacheManager, IDeliveryCachePurger, IFailSafeStateProvider, IDisposable -{ - private readonly FusionCacheManager _inner = FusionCacheManager.CreateHybrid( - cache, - cacheOptions, - jsonSerializerOptions, - logger, - backplane, - environmentId); - - /// - public CacheStorageMode StorageMode => _inner.StorageMode; - - /// - public Task?> GetOrSetAsync( - string cacheKey, - Func?>> factory, - TimeSpan? expiration = null, - CancellationToken cancellationToken = default) - where T : class - => _inner.GetOrSetAsync(cacheKey, factory, expiration, cancellationToken); - - /// - public Task InvalidateAsync(string[] dependencyKeys, CancellationToken cancellationToken = default) - => _inner.InvalidateAsync(dependencyKeys, cancellationToken); - - /// - public Task PurgeAsync(bool allowFailSafe = false, CancellationToken cancellationToken = default) - => _inner.PurgeAsync(allowFailSafe, cancellationToken); - - bool IFailSafeStateProvider.IsFailSafeActive(string cacheKey) - => ((IFailSafeStateProvider)_inner).IsFailSafeActive(cacheKey); - - /// - public void Dispose() => _inner.Dispose(); -} diff --git a/src/delivery/Kontent.Ai.Delivery.Caching/MemoryCacheManager.cs b/src/delivery/Kontent.Ai.Delivery.Caching/MemoryCacheManager.cs deleted file mode 100644 index 50ab028ec..000000000 --- a/src/delivery/Kontent.Ai.Delivery.Caching/MemoryCacheManager.cs +++ /dev/null @@ -1,47 +0,0 @@ -using Microsoft.Extensions.Caching.Memory; -using Microsoft.Extensions.Logging; - -namespace Kontent.Ai.Delivery.Caching; - -/// -/// In-memory implementation of backed by FusionCache. -/// -internal sealed class MemoryCacheManager( - IMemoryCache memoryCache, - DeliveryCacheOptions cacheOptions, - ILogger? logger = null, - string? environmentId = null) - : IDeliveryCacheManager, IDeliveryCachePurger, IFailSafeStateProvider, IDisposable -{ - private readonly FusionCacheManager _inner = FusionCacheManager.CreateMemory( - memoryCache, - cacheOptions, - logger, - environmentId); - - /// - public CacheStorageMode StorageMode => _inner.StorageMode; - - /// - public Task?> GetOrSetAsync( - string cacheKey, - Func?>> factory, - TimeSpan? expiration = null, - CancellationToken cancellationToken = default) - where T : class - => _inner.GetOrSetAsync(cacheKey, factory, expiration, cancellationToken); - - /// - public Task InvalidateAsync(string[] dependencyKeys, CancellationToken cancellationToken = default) - => _inner.InvalidateAsync(dependencyKeys, cancellationToken); - - /// - public Task PurgeAsync(bool allowFailSafe = false, CancellationToken cancellationToken = default) - => _inner.PurgeAsync(allowFailSafe, cancellationToken); - - bool IFailSafeStateProvider.IsFailSafeActive(string cacheKey) - => ((IFailSafeStateProvider)_inner).IsFailSafeActive(cacheKey); - - /// - public void Dispose() => _inner.Dispose(); -} diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/ApiApproval/PublicApiApprovalTests.PublicApi_ShouldNotChangeUnexpectedly.verified.txt b/src/delivery/Kontent.Ai.Delivery.Tests/ApiApproval/PublicApiApprovalTests.PublicApi_ShouldNotChangeUnexpectedly.verified.txt index 46666412c..3f5600c97 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/ApiApproval/PublicApiApprovalTests.PublicApi_ShouldNotChangeUnexpectedly.verified.txt +++ b/src/delivery/Kontent.Ai.Delivery.Tests/ApiApproval/PublicApiApprovalTests.PublicApi_ShouldNotChangeUnexpectedly.verified.txt @@ -1,5 +1,6 @@ // Kontent.Ai.Delivery public sealed class DeliveryClient : IAsyncDisposable, IDeliveryClient, IDisposable + IDeliveryCacheManager? CacheManager { get; } IAssetUsedInQuery GetAssetUsedIn(String codename) IDynamicEnumerateItemsQuery GetItemsFeed() IDynamicItemQuery GetItem(String codename) diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/ApiApproval/PublicApiApprovalTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/ApiApproval/PublicApiApprovalTests.cs index ca2e5c513..2a1150032 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/ApiApproval/PublicApiApprovalTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/ApiApproval/PublicApiApprovalTests.cs @@ -10,5 +10,5 @@ public Task PublicApi_ShouldNotChangeUnexpectedly() [Fact] public Task CachingPublicApi_ShouldNotChangeUnexpectedly() - => Verify(PublicApiApproval.Surface(typeof(Kontent.Ai.Delivery.Caching.MemoryCacheManager).Assembly)); + => Verify(PublicApiApproval.Surface(typeof(Kontent.Ai.Delivery.DeliveryClientBuilderCachingExtensions).Assembly)); } diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Builders/Configuration/DeliveryClientCreateTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Builders/Configuration/DeliveryClientCreateTests.cs index 51ea9598e..fc2c8d59d 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/Builders/Configuration/DeliveryClientCreateTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Builders/Configuration/DeliveryClientCreateTests.cs @@ -3,7 +3,6 @@ using System.Reflection; using System.Text; using Kontent.Ai.Delivery.Abstractions; -using Kontent.Ai.Delivery.Caching; using Microsoft.Extensions.Caching.Distributed; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; @@ -110,6 +109,46 @@ public async Task Create_FromOptionsInstance_MutatingItAfterwardsDoesNotReachThe _http.VerifyNoOutstandingExpectation(); } + [Fact] + public async Task Create_WithMemoryCache_ExposesTheCacheManager() + { + // A standalone client owns its container, so the client is the only way to reach the cache + // inside it - and a webhook handler has to, to invalidate. + _http.When(HttpMethod.Get, $"https://deliver.kontent.ai/{EnvironmentId}/types") + .Respond("application/json", """{"types":[],"pagination":{"skip":0,"limit":0,"count":0,"next_page":""}}"""); + var attempts = new AttemptCounter(); + + await using var client = CreateClient( + o => + { + o.EnvironmentId = EnvironmentId; + o.EnableResilience = false; + }, + d => + { + d.HttpClient.AddHttpMessageHandler(() => attempts); + d.UseMemoryCache(); + }); + + await client.GetTypes().ExecuteAsync(); + await client.GetTypes().ExecuteAsync(); + Assert.Equal(1, attempts.Count); + + Assert.NotNull(client.CacheManager); + await client.CacheManager.InvalidateAsync([DeliveryCacheDependencies.TypesListScope]); + await client.GetTypes().ExecuteAsync(); + + Assert.Equal(2, attempts.Count); + } + + [Fact] + public async Task Create_WithoutACache_HasNoCacheManager() + { + await using var client = CreateClient(o => o.EnvironmentId = EnvironmentId); + + Assert.Null(client.CacheManager); + } + [Fact] public async Task Create_WithDisabledResilience_DoesNotRetry() { @@ -169,7 +208,7 @@ public async Task Create_WithMemoryCache_ServesARepeatedQueryFromTheCache() o => o.EnvironmentId = EnvironmentId, d => d.UseMemoryCache(opts => opts.DefaultExpiration = TimeSpan.FromMinutes(30))); - Assert.IsType(GetCacheManager(client)); + Assert.Equal(CacheStorageMode.HydratedObject, GetCacheManager(client)!.StorageMode); Assert.Equal(ResponseSource.Origin, (await client.GetItems().ExecuteAsync()).ResponseSource); Assert.Equal(ResponseSource.Cache, (await client.GetItems().ExecuteAsync()).ResponseSource); _http.VerifyNoOutstandingExpectation(); @@ -241,7 +280,7 @@ public async Task Create_WithHybridCache_ServesARepeatedQueryFromTheCache() d.UseHybridCache(opts => opts.DefaultExpiration = TimeSpan.FromHours(1)); }); - Assert.IsType(GetCacheManager(client)); + Assert.Equal(CacheStorageMode.RawJson, GetCacheManager(client)!.StorageMode); Assert.Equal(ResponseSource.Origin, (await client.GetItems().ExecuteAsync()).ResponseSource); Assert.Equal(ResponseSource.Cache, (await client.GetItems().ExecuteAsync()).ResponseSource); _http.VerifyNoOutstandingExpectation(); @@ -278,7 +317,7 @@ public async Task Create_WithAllOptions_AppliesEachOfThem() Assert.True((await client.GetItems().ExecuteAsync()).IsSuccess); Assert.Contains("article", typeProvider.Requested); - Assert.IsType(GetCacheManager(client)); + Assert.Equal(CacheStorageMode.HydratedObject, GetCacheManager(client)!.StorageMode); Assert.Equal("mobile", OwnedServices(client).GetRequiredService>().Get("Default").DefaultRenditionPreset); } @@ -296,7 +335,7 @@ public async Task Create_CallingMemoryCacheAfterHybridCache_UsesLastConfigured() }); Assert.NotNull(client); - Assert.IsType(GetCacheManager(client)); + Assert.Equal(CacheStorageMode.HydratedObject, GetCacheManager(client)!.StorageMode); } [Fact] @@ -313,7 +352,7 @@ public async Task Create_CallingHybridCacheAfterMemoryCache_UsesLastConfigured() }); Assert.NotNull(client); - Assert.IsType(GetCacheManager(client)); + Assert.Equal(CacheStorageMode.RawJson, GetCacheManager(client)!.StorageMode); } [Fact] diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/CachingIntegrationTests.TestDoubles.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/CachingIntegrationTests.TestDoubles.cs index 0eb7f0666..571902dd5 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/CachingIntegrationTests.TestDoubles.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/CachingIntegrationTests.TestDoubles.cs @@ -3,6 +3,7 @@ using System.Text; using Kontent.Ai.Delivery.Abstractions; using Microsoft.Extensions.Caching.Distributed; +using Microsoft.Extensions.Logging; namespace Kontent.Ai.Delivery.Tests.Caching; @@ -48,6 +49,31 @@ public class CachedItem } } + private sealed class CategoryCollectingLoggerProvider : ILoggerProvider + { + private readonly ConcurrentQueue<(string Category, LogLevel Level, string Message)> _entries = new(); + + public IReadOnlyCollection<(string Category, LogLevel Level, string Message)> Entries => _entries.ToArray(); + + public ILogger CreateLogger(string categoryName) => new CategoryCollectingLogger(categoryName, _entries); + + public void Dispose() + { + } + + private sealed class CategoryCollectingLogger( + string categoryName, + ConcurrentQueue<(string Category, LogLevel Level, string Message)> entries) : ILogger + { + public IDisposable? BeginScope(TState state) where TState : notnull => null; + + public bool IsEnabled(LogLevel logLevel) => true; + + public void Log(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func formatter) + => entries.Enqueue((categoryName, logLevel, formatter(state, exception))); + } + } + private class MockDistributedCache : IDistributedCache { private readonly ConcurrentDictionary _cache = new(); diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/CachingIntegrationTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/CachingIntegrationTests.cs index 17a20d00c..dc4eb689f 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/CachingIntegrationTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/CachingIntegrationTests.cs @@ -1,13 +1,11 @@ using System.Net; -using System.Text; -using System.Text.Json; using Kontent.Ai.Delivery.Abstractions; -using Kontent.Ai.Delivery.Api.QueryParams.Items; using Kontent.Ai.Delivery.Caching; using Kontent.Ai.Delivery.Tests.Models.ContentTypes; using Microsoft.Extensions.Caching.Distributed; using Microsoft.Extensions.Caching.Memory; using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; using Microsoft.Extensions.Options; using RichardSzalay.MockHttp; @@ -180,6 +178,137 @@ public async Task MemoryCache_FailSafe_PreservesDependencyKeys() mock.VerifyNoOutstandingExpectation(); } + [Theory] + [InlineData(false)] + [InlineData(true)] + public async Task FailSafe_AnAnswerAfterInvalidation_IsNotServedStale(bool hybrid) + { + // A webhook evicts the item and the API then says it is gone. Fail-safe is for an origin that + // cannot be reached; an answer, however unwelcome, is not an outage, so the stale copy is dropped + // rather than served - otherwise unpublishing would not take effect until FailSafeMaxDuration ran out. + var mock = new MockHttpMessageHandler(); + var itemCodename = "coffee_beverages_explained"; + var fixtureContent = await ReadFixtureAsync($"DeliveryClient{Path.DirectorySeparatorChar}{itemCodename}.json"); + var provider = BuildFailSafeServiceProvider(mock, hybrid); + var client = provider.GetRequiredKeyedService("test"); + var cacheManager = provider.GetRequiredKeyedService("test"); + + mock.Expect($"{BaseUrl}/items/{itemCodename}").Respond("application/json", fixtureContent); + Assert.True((await client.GetItem
(itemCodename).ExecuteAsync()).IsSuccess); + + await cacheManager.InvalidateAsync([$"item_{itemCodename}"]); + + mock.Expect($"{BaseUrl}/items/{itemCodename}") + .Respond(HttpStatusCode.NotFound, "application/json", """{"message":"The requested content item was not found.","error_code":100}"""); + var gone = await client.GetItem
(itemCodename).ExecuteAsync(); + + Assert.False(gone.IsSuccess); + Assert.Equal(HttpStatusCode.NotFound, gone.StatusCode); + Assert.Equal(ResponseSource.Origin, gone.ResponseSource); + + // The stale copy went with the answer: an outage now has nothing to fall back on. + mock.Expect($"{BaseUrl}/items/{itemCodename}") + .Respond(HttpStatusCode.ServiceUnavailable, "application/json", """{"message":"Service unavailable","error_code":503}"""); + var outage = await client.GetItem
(itemCodename).ExecuteAsync(); + + Assert.False(outage.IsSuccess); + Assert.Equal(HttpStatusCode.ServiceUnavailable, outage.StatusCode); + mock.VerifyNoOutstandingExpectation(); + } + + [Theory] + [InlineData(false)] + [InlineData(true)] + public async Task FailSafe_AnOutageAfterInvalidation_IsServedStale(bool hybrid) + { + var mock = new MockHttpMessageHandler(); + var itemCodename = "coffee_beverages_explained"; + var fixtureContent = await ReadFixtureAsync($"DeliveryClient{Path.DirectorySeparatorChar}{itemCodename}.json"); + var provider = BuildFailSafeServiceProvider(mock, hybrid); + var client = provider.GetRequiredKeyedService("test"); + var cacheManager = provider.GetRequiredKeyedService("test"); + + mock.Expect($"{BaseUrl}/items/{itemCodename}").Respond("application/json", fixtureContent); + Assert.True((await client.GetItem
(itemCodename).ExecuteAsync()).IsSuccess); + + await cacheManager.InvalidateAsync([$"item_{itemCodename}"]); + + mock.Expect($"{BaseUrl}/items/{itemCodename}") + .Respond(HttpStatusCode.ServiceUnavailable, "application/json", """{"message":"Service unavailable","error_code":503}"""); + var served = await client.GetItem
(itemCodename).ExecuteAsync(); + + Assert.True(served.IsSuccess); + Assert.Equal(ResponseSource.FailSafe, served.ResponseSource); + Assert.Contains($"item_{itemCodename}", served.DependencyKeys!); + mock.VerifyNoOutstandingExpectation(); + } + + [Fact] + public async Task FailSafe_ATransportFailureAfterInvalidation_IsServedStale() + { + // No response at all is the clearest outage there is. + var mock = new MockHttpMessageHandler(); + var itemCodename = "coffee_beverages_explained"; + var fixtureContent = await ReadFixtureAsync($"DeliveryClient{Path.DirectorySeparatorChar}{itemCodename}.json"); + var provider = BuildFailSafeServiceProvider(mock, hybrid: false); + var client = provider.GetRequiredKeyedService("test"); + var cacheManager = provider.GetRequiredKeyedService("test"); + + mock.Expect($"{BaseUrl}/items/{itemCodename}").Respond("application/json", fixtureContent); + Assert.True((await client.GetItem
(itemCodename).ExecuteAsync()).IsSuccess); + + await cacheManager.InvalidateAsync([$"item_{itemCodename}"]); + + mock.Expect($"{BaseUrl}/items/{itemCodename}").Throw(new HttpRequestException("Connection refused")); + var served = await client.GetItem
(itemCodename).ExecuteAsync(); + + Assert.True(served.IsSuccess); + Assert.Equal(ResponseSource.FailSafe, served.ResponseSource); + mock.VerifyNoOutstandingExpectation(); + } + + [Fact] + public async Task HybridCache_DistributedCacheOutage_QueriesStillSucceed() + { + // The distributed tier is down. The origin answers the first query and the memory tier the + // second; the outage is FusionCache's to log, not the caller's to catch. + var mock = new MockHttpMessageHandler(); + var fixtureContent = await ReadFixtureAsync($"DeliveryClient{Path.DirectorySeparatorChar}types_accessory.json"); + mock.Expect($"{BaseUrl}/types").Respond("application/json", fixtureContent); + var options = new DeliveryOptions { EnvironmentId = _guid.ToString() }; + var provider = BuildNamedHybridCacheServiceProvider(mock, options, new ThrowingDistributedCache()); + var client = provider.GetRequiredKeyedService("test"); + + var first = await client.GetTypes().ExecuteAsync(); + var second = await client.GetTypes().ExecuteAsync(); + + Assert.True(first.IsSuccess); + Assert.Equal(ResponseSource.Origin, first.ResponseSource); + Assert.True(second.IsSuccess); + Assert.Equal(ResponseSource.Cache, second.ResponseSource); + mock.VerifyNoOutstandingExpectation(); + } + + [Fact] + public async Task HybridCache_DistributedCacheOutage_IsLoggedByFusionCache() + { + // Working the outage around is only acceptable if it is visible: FusionCache logs it under its own + // category once it is handed the application's logger. + var mock = new MockHttpMessageHandler(); + var fixtureContent = await ReadFixtureAsync($"DeliveryClient{Path.DirectorySeparatorChar}types_accessory.json"); + mock.When($"{BaseUrl}/types").Respond("application/json", fixtureContent); + var logs = new CategoryCollectingLoggerProvider(); + var services = new ServiceCollection(); + services.AddLogging(logging => logging.AddProvider(logs)); + services.AddSingleton(new ThrowingDistributedCache()); + AddNamedDeliveryClient(services, "test", new DeliveryOptions { EnvironmentId = _guid.ToString() }, mock, d => d.UseHybridCache()); + var client = services.BuildServiceProvider().GetRequiredKeyedService("test"); + + Assert.True((await client.GetTypes().ExecuteAsync()).IsSuccess); + + Assert.Contains(logs.Entries, e => e.Category == "ZiggyCreatures.Caching.Fusion.FusionCache" && e.Level >= LogLevel.Warning); + } + [Fact] public async Task MemoryCache_GetItem_ExpiresAfterTtl_HitsApiAgain() { @@ -1804,36 +1933,6 @@ public async Task HybridCache_GetItems_CacheHitOnSecondCall() mock.VerifyNoOutstandingExpectation(); } - [Fact] - public async Task HybridCache_CorruptedModularContentPayload_FallsBackToApiAndRecaches() - { - var mock = new MockHttpMessageHandler(); - var itemCodename = "coffee_beverages_explained"; - var fixtureContent = await ReadFixtureAsync($"DeliveryClient{Path.DirectorySeparatorChar}{itemCodename}.json"); - - mock.Expect($"{BaseUrl}/items/{itemCodename}") - .Respond("application/json", fixtureContent); - - var options = new DeliveryOptions - { - EnvironmentId = _guid.ToString() - }; - - var mockDistributedCache = new MockDistributedCache(); - SeedCorruptedDistributedItemPayload(mockDistributedCache, "test", itemCodename, fixtureContent); - var serviceProvider = BuildNamedHybridCacheServiceProvider(mock, options, mockDistributedCache); - var client = serviceProvider.GetRequiredKeyedService("test"); - - var result1 = await client.GetItem
(itemCodename).ExecuteAsync(); - var result2 = await client.GetItem
(itemCodename).ExecuteAsync(); - - Assert.True(result1.IsSuccess); - Assert.True(result2.IsSuccess); - Assert.False(result1.IsCacheHit); - Assert.True(result2.IsCacheHit); - mock.VerifyNoOutstandingExpectation(); - } - [Fact] public async Task HybridCache_Invalidation_RefreshesCache() { @@ -2747,7 +2846,7 @@ public async Task UnkeyedCacheManagerRegistration_IsIgnoredByClientCachingPath() services.AddMemoryCache(); services.AddSingleton(sp => - new MemoryCacheManager(sp.GetRequiredService(), new DeliveryCacheOptions())); + FusionCacheManager.CreateMemory(sp.GetRequiredService(), new DeliveryCacheOptions())); services.AddDeliveryClient(options, d => d.HttpClient.ConfigurePrimaryHttpMessageHandler(() => mock)); var serviceProvider = services.BuildServiceProvider(); @@ -2905,6 +3004,44 @@ private static ServiceProvider BuildNamedMemoryCacheServiceProvider( return services.BuildServiceProvider(); } + // Resilience off so a failing origin answers once rather than after the pipeline's retries, and + // no throttle so every call reaches the origin. + private ServiceProvider BuildFailSafeServiceProvider(HttpMessageHandler httpHandler, bool hybrid) + { + var options = new DeliveryOptions + { + EnvironmentId = _guid.ToString(), + EnableResilience = false + }; + + Action failSafe = o => + { + o.IsFailSafeEnabled = true; + o.FailSafeMaxDuration = TimeSpan.FromMinutes(5); + o.FailSafeThrottleDuration = TimeSpan.Zero; + }; + + var services = new ServiceCollection(); + if (hybrid) + { + services.AddSingleton(new MockDistributedCache()); + } + + AddNamedDeliveryClient(services, "test", options, httpHandler, d => + { + if (hybrid) + { + d.UseHybridCache(failSafe); + } + else + { + d.UseMemoryCache(failSafe); + } + }); + + return services.BuildServiceProvider(); + } + private static ServiceProvider BuildNamedHybridCacheServiceProvider( HttpMessageHandler httpHandler, DeliveryOptions options, @@ -2918,33 +3055,5 @@ private static ServiceProvider BuildNamedHybridCacheServiceProvider( return services.BuildServiceProvider(); } - private static void SeedCorruptedDistributedItemPayload( - MockDistributedCache distributedCache, - string clientName, - string itemCodename, - string singleItemResponseJson) - { - using var responseDoc = JsonDocument.Parse(singleItemResponseJson); - var itemJson = responseDoc.RootElement.GetProperty("item").GetRawText(); - - var payload = new CachedRawItemsPayload - { - ItemsJson = [itemJson], - ModularContentJson = new Dictionary(StringComparer.Ordinal) - { - ["broken-linked-item"] = "{ this is not valid json }" - } - }; - - var sdkCacheKey = CacheKeyBuilder.BuildItemKey(itemCodename, new SingleItemParams(), modelType: null); - var distributedCacheKey = $"{clientName}:cache:{sdkCacheKey}"; - var serializedPayload = JsonSerializer.Serialize(payload); - - distributedCache.Set( - distributedCacheKey, - Encoding.UTF8.GetBytes(serializedPayload), - new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5) }); - } - #endregion } diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheCoherenceTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheCoherenceTests.cs index d2371516c..8939dfa8c 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheCoherenceTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheCoherenceTests.cs @@ -78,8 +78,8 @@ public async Task Invalidation_WithoutABackplane_CanFailToReachTheOtherNode() Assert.Equal("v1", fromB!.Value.Value); } - private static HybridCacheManager NewNode(IDistributedCache cache, MemoryBackplane? backplane) => - new(cache, new DeliveryCacheOptions(), backplane: backplane); + private static FusionCacheManager NewNode(IDistributedCache cache, MemoryBackplane? backplane) => + FusionCacheManager.CreateHybrid(cache, new DeliveryCacheOptions(), backplane: backplane); // Both nodes join one in-process channel, standing in for a shared Redis backplane. private static MemoryBackplane NewBackplane(string channel) => diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheFidelityTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheFidelityTests.cs index 982fa55a4..789b1d3a2 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheFidelityTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheFidelityTests.cs @@ -30,7 +30,7 @@ public HybridCacheFidelityTests() _distributedCache = services.BuildServiceProvider().GetRequiredService(); } - private HybridCacheManager Node() => new( + private FusionCacheManager Node() => FusionCacheManager.CreateHybrid( _distributedCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }); @@ -116,8 +116,8 @@ public async Task TwoEnvironmentsSharingOneCache_DoNotSeeEachOthersEntries() // environment. Sharing one Redis between apps pointing at different environments then serves one // app the other's content - the environment has to be part of the key, not just the query. var options = new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }; - using var production = new HybridCacheManager(_distributedCache, options, environmentId: "11111111-1111-1111-1111-111111111111"); - using var staging = new HybridCacheManager(_distributedCache, options, environmentId: "22222222-2222-2222-2222-222222222222"); + using var production = FusionCacheManager.CreateHybrid(_distributedCache, options, environmentId: "11111111-1111-1111-1111-111111111111"); + using var staging = FusionCacheManager.CreateHybrid(_distributedCache, options, environmentId: "22222222-2222-2222-2222-222222222222"); var fromProduction = await production.GetOrSetAsync( "items:article", diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheManagerRealImplementationTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheManagerRealImplementationTests.cs index 83e2071b6..bc4d78003 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheManagerRealImplementationTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheManagerRealImplementationTests.cs @@ -6,13 +6,13 @@ namespace Kontent.Ai.Delivery.Tests.Caching; /// -/// Integration tests for HybridCacheManager using Microsoft's official AddDistributedMemoryCache implementation. +/// Integration tests for FusionCacheManager using Microsoft's official AddDistributedMemoryCache implementation. /// These tests verify that the hybrid cache manager works correctly with a real IDistributedCache implementation, /// not just our custom mock. /// public class HybridCacheManagerRealImplementationTests { - private readonly HybridCacheManager _cacheManager; + private readonly FusionCacheManager _cacheManager; public HybridCacheManagerRealImplementationTests() { @@ -22,7 +22,7 @@ public HybridCacheManagerRealImplementationTests() var serviceProvider = services.BuildServiceProvider(); var distributedCache = serviceProvider.GetRequiredService(); - _cacheManager = new HybridCacheManager(distributedCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }); + _cacheManager = FusionCacheManager.CreateHybrid(distributedCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }); } #region Basic Operations diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheManagerTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheManagerTests.cs index 4f9a8ac93..5371fffcb 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheManagerTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/HybridCacheManagerTests.cs @@ -9,18 +9,18 @@ namespace Kontent.Ai.Delivery.Tests.Caching; /// -/// Comprehensive tests for HybridCacheManager (hybrid L1+L2 cache) implementation. +/// Comprehensive tests for FusionCacheManager (hybrid L1+L2 cache) implementation. /// Tests cover: basic operations, dependency tracking, invalidation, serialization, concurrency, and error handling. /// public class HybridCacheManagerTests { private readonly MockDistributedCache _mockCache; - private readonly HybridCacheManager _cacheManager; + private readonly FusionCacheManager _cacheManager; public HybridCacheManagerTests() { _mockCache = new MockDistributedCache(); - _cacheManager = new HybridCacheManager(_mockCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }); + _cacheManager = FusionCacheManager.CreateHybrid(_mockCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }); } #region Basic Operations Tests @@ -72,6 +72,80 @@ public async Task GetOrSetAsync_FactoryReturnsNull_ReturnsNull() Assert.Null(result); } + [Fact] + public async Task GetOrSetAsync_FactoryReturnsNullAfterInvalidation_DropsTheStaleEntryFromBothTiers() + { + // Null is the origin's answer, not an outage: the stale copy an invalidation left behind for + // fail-safe is not served for it and does not survive it, in memory or in the distributed tier. + var distributedCache = new MockDistributedCache(); + var manager = FusionCacheManager.CreateHybrid(distributedCache, new DeliveryCacheOptions + { + IsFailSafeEnabled = true, + FailSafeMaxDuration = TimeSpan.FromMinutes(5), + FailSafeThrottleDuration = TimeSpan.Zero + }); + + await manager.GetOrSetAsync("gone_key", _ => + Task.FromResult?>(new CacheEntry(new TestCacheValue { Id = 1, Name = "Gone" }, ["item_gone"]))); + await manager.InvalidateAsync(["item_gone"]); + + var answer = await manager.GetOrSetAsync("gone_key", _ => + Task.FromResult?>(null)); + + Assert.Null(answer); + await Assert.ThrowsAsync(() => + manager.GetOrSetAsync("gone_key", _ => throw new InvalidOperationException("Simulated API failure"))); + + var freshNode = FusionCacheManager.CreateHybrid(distributedCache, new DeliveryCacheOptions { IsFailSafeEnabled = true }); + await Assert.ThrowsAsync(() => + freshNode.GetOrSetAsync("gone_key", _ => throw new InvalidOperationException("Simulated API failure"))); + } + + [Fact] + public async Task GetOrSetAsync_DistributedCacheThrows_AnswersFromTheFactoryAndThenFromMemory() + { + // A distributed tier that is down is worked around: the factory answers the first call, the + // memory tier the next, and nothing is thrown out of the query. + var manager = FusionCacheManager.CreateHybrid(new ThrowingDistributedCache(), new DeliveryCacheOptions()); + var factoryCalls = 0; + + var first = await manager.GetOrSetAsync("outage_key", _ => + { + factoryCalls++; + return Task.FromResult?>(new CacheEntry(new TestCacheValue { Id = 1, Name = "Origin" }, ["dep1"])); + }); + var second = await manager.GetOrSetAsync("outage_key", _ => + { + factoryCalls++; + return Task.FromResult?>(null); + }); + + Assert.Equal(1, factoryCalls); + Assert.True(first!.FromFactory); + Assert.False(second!.FromFactory); + Assert.Equal("Origin", second.Value.Name); + } + + [Fact] + public async Task PurgeAsync_OverASharedDistributedCache_LeavesTheOtherClientsEntriesAlone() + { + // The purge marker lives in the shared tier under the purging client's prefix, so the other client + // never reads it - not even on a node of its own that has yet to read anything, which is the node + // that would go to the shared tier for it. + var shared = new MockDistributedCache(); + var a = FusionCacheManager.CreateHybrid(shared, new DeliveryCacheOptions { KeyPrefix = "a" }); + var b = FusionCacheManager.CreateHybrid(shared, new DeliveryCacheOptions { KeyPrefix = "b" }); + await PopulateCache("k", new TestCacheValue { Id = 1, Name = "A" }, ["dep"], a); + await PopulateCache("k", new TestCacheValue { Id = 2, Name = "B" }, ["dep"], b); + + await ((IDeliveryCachePurger)a).PurgeAsync(); + + var freshNodeOfB = FusionCacheManager.CreateHybrid(shared, new DeliveryCacheOptions { KeyPrefix = "b" }); + Assert.False(await IsFactoryCalledAsync("k", freshNodeOfB)); + Assert.False(await IsFactoryCalledAsync("k", b)); + Assert.True(await IsFactoryCalledAsync("k", a)); + } + [Fact] public async Task GetOrSetAsync_OverwritesCachedValue_OnNextMiss() { @@ -102,14 +176,14 @@ public async Task GetOrSetAsync_EmptyDependencies_DoesNotThrow() public void Constructor_WithDefaultExpiration_AcceptsValue() { var expiration = TimeSpan.FromMinutes(30); - var manager = new HybridCacheManager(_mockCache, new DeliveryCacheOptions { DefaultExpiration = expiration }); + var manager = FusionCacheManager.CreateHybrid(_mockCache, new DeliveryCacheOptions { DefaultExpiration = expiration }); Assert.NotNull(manager); } [Fact] public void Constructor_WithNullExpiration_UsesDefaultOneHour() => - Assert.NotNull(new HybridCacheManager(_mockCache, new DeliveryCacheOptions())); + Assert.NotNull(FusionCacheManager.CreateHybrid(_mockCache, new DeliveryCacheOptions())); [Fact] public async Task GetOrSetAsync_WithCustomExpiration_DoesNotThrow() @@ -127,7 +201,7 @@ public async Task GetOrSetAsync_WithCustomExpiration_DoesNotThrow() [Fact] public async Task GetOrSetAsync_ExpirationPassedToCacheEntry() { - var manager = new HybridCacheManager(_mockCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromHours(2) }); + var manager = FusionCacheManager.CreateHybrid(_mockCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromHours(2) }); var value = new TestCacheValue { Id = 1, Name = "Test" }; await manager.GetOrSetAsync("test_key", _ => @@ -147,7 +221,7 @@ await manager.GetOrSetAsync("test_key", _ => public async Task GetOrSetAsync_WithoutCustomExpiration_UsesDefaultExpiration() { var defaultExpiration = TimeSpan.FromMilliseconds(80); - var manager = new HybridCacheManager(_mockCache, new DeliveryCacheOptions { DefaultExpiration = defaultExpiration }); + var manager = FusionCacheManager.CreateHybrid(_mockCache, new DeliveryCacheOptions { DefaultExpiration = defaultExpiration }); var value = new TestCacheValue { Id = 1, Name = "Test" }; await manager.GetOrSetAsync("test_key", _ => @@ -297,7 +371,7 @@ public async Task GetOrSetAsync_WithDependencies_InvalidateRemovesEntry() [Fact] public async Task GetOrSetAsync_SameDependencyWithShorterTtl_StillInvalidatesAllEntries() { - var manager = new HybridCacheManager(_mockCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }); + var manager = FusionCacheManager.CreateHybrid(_mockCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }); var dependency = "dep_shared"; await manager.GetOrSetAsync("long_ttl_key", _ => @@ -512,7 +586,7 @@ await Assert.ThrowsAsync(() => public async Task GetOrSetAsync_WithPrefixedManager_DoesNotLeakToDefaultNamespace() { var value = new TestCacheValue { Id = 1, Name = "Test" }; - var prefixedManager = new HybridCacheManager(_mockCache, new DeliveryCacheOptions { KeyPrefix = "prefixed" }); + var prefixedManager = FusionCacheManager.CreateHybrid(_mockCache, new DeliveryCacheOptions { KeyPrefix = "prefixed" }); await PopulateCache("test_key", value, [], prefixedManager); @@ -585,8 +659,8 @@ public async Task InvalidateAsync_ManyDependencies_Succeeds() public async Task GetOrSetAsync_WithDifferentPrefixes_IsolatesCacheEntries() { var sharedCache = new MockDistributedCache(); - var manager1 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); - var manager2 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); + var manager1 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); + var manager2 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); var value1 = new TestCacheValue { Id = 1, Name = "Client1Value" }; var value2 = new TestCacheValue { Id = 2, Name = "Client2Value" }; @@ -609,8 +683,8 @@ public async Task GetOrSetAsync_WithDifferentPrefixes_IsolatesCacheEntries() public async Task InvalidateAsync_WithDifferentPrefixes_OnlyAffectsOwnEntries() { var sharedCache = new MockDistributedCache(); - var manager1 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); - var manager2 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); + var manager1 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); + var manager2 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); var dependency = "same_dep"; var value1 = new TestCacheValue { Id = 1, Name = "Client1Value" }; @@ -629,8 +703,8 @@ public async Task InvalidateAsync_WithDifferentPrefixes_OnlyAffectsOwnEntries() public async Task GetOrSetAsync_WithDifferentPrefixes_DoesNotCrossContaminate() { var sharedCache = new MockDistributedCache(); - var manager1 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); - var manager2 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); + var manager1 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); + var manager2 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); var value = new TestCacheValue { Id = 1, Name = "OnlyInClient1" }; await PopulateCache("unique_key", value, [], manager1); @@ -643,8 +717,8 @@ public async Task GetOrSetAsync_WithDifferentPrefixes_DoesNotCrossContaminate() public async Task GetOrSetAsync_WithNullPrefix_UsesUnprefixedKeys() { var sharedCache = new MockDistributedCache(); - var managerNoPrefix = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = null }); - var managerWithPrefix = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "prefixed" }); + var managerNoPrefix = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = null }); + var managerWithPrefix = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "prefixed" }); var value1 = new TestCacheValue { Id = 1, Name = "NoPrefix" }; var value2 = new TestCacheValue { Id = 2, Name = "WithPrefix" }; @@ -665,8 +739,8 @@ public async Task GetOrSetAsync_WithNullPrefix_UsesUnprefixedKeys() public async Task InvalidateAsync_WithSharedDependencyName_OnlyInvalidatesOwnPrefix() { var sharedCache = new MockDistributedCache(); - var manager1 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "prod" }); - var manager2 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "preview" }); + var manager1 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "prod" }); + var manager2 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "preview" }); var dependency = "content_type_article"; @@ -687,8 +761,8 @@ public async Task InvalidateAsync_WithSharedDependencyName_OnlyInvalidatesOwnPre public async Task ConcurrentOperations_WithDifferentPrefixes_MaintainsIsolation() { var sharedCache = new MockDistributedCache(); - var manager1 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); - var manager2 = new HybridCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); + var manager1 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); + var manager2 = FusionCacheManager.CreateHybrid(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); var dependency = "shared_dep_name"; @@ -714,8 +788,8 @@ public async Task ConcurrentOperations_WithDifferentPrefixes_MaintainsIsolation( public async Task Constructor_WithKeyPrefix_IsolatesEntries() { var cache = new MockDistributedCache(); - var manager = new HybridCacheManager(cache, new DeliveryCacheOptions { KeyPrefix = "my-prefix" }); - var defaultManager = new HybridCacheManager(cache, new DeliveryCacheOptions()); + var manager = FusionCacheManager.CreateHybrid(cache, new DeliveryCacheOptions { KeyPrefix = "my-prefix" }); + var defaultManager = FusionCacheManager.CreateHybrid(cache, new DeliveryCacheOptions()); await PopulateCache("test", new TestCacheValue { Id = 1 }, [], manager); Assert.False(await IsFactoryCalledAsync("test", manager)); diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/MemoryCacheManagerTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/MemoryCacheManagerTests.cs index 574d9bf56..197ace5a5 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/MemoryCacheManagerTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/MemoryCacheManagerTests.cs @@ -6,18 +6,18 @@ namespace Kontent.Ai.Delivery.Tests.Caching; /// -/// Comprehensive tests for MemoryCacheManager implementation. +/// Comprehensive tests for FusionCacheManager implementation. /// Tests cover: basic operations, dependency tracking, invalidation, concurrency, resource management, and error handling. /// public class MemoryCacheManagerTests : IDisposable { private readonly IMemoryCache _memoryCache; - private readonly MemoryCacheManager _cacheManager; + private readonly FusionCacheManager _cacheManager; public MemoryCacheManagerTests() { _memoryCache = new MemoryCache(new MemoryCacheOptions()); - _cacheManager = new MemoryCacheManager(_memoryCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }); + _cacheManager = FusionCacheManager.CreateMemory(_memoryCache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5) }); } public void Dispose() @@ -121,20 +121,32 @@ await _cacheManager.GetOrSetAsync("test_key", _ => [InlineData(null)] [InlineData("")] [InlineData(" ")] - public async Task GetOrSetAsync_InvalidKey_StillCallsFactory(string? cacheKey) + public async Task GetOrSetAsync_InvalidKey_Throws(string? cacheKey) { - var factoryCalled = false; - var value = new TestCacheValue { Id = 1, Name = "Test" }; + await Assert.ThrowsAnyAsync(() => _cacheManager.GetOrSetAsync(cacheKey!, _ => + Task.FromResult?>(new CacheEntry(new TestCacheValue { Id = 1, Name = "Test" }, [])))); + } - var result = await _cacheManager.GetOrSetAsync(cacheKey!, _ => + [Fact] + public async Task GetOrSetAsync_ReportsAStaleValueAsSuchOnlyToTheCallItWasServedTo() + { + // Fail-safe served this call a stale copy; the classification belongs to this call alone. + using var cache = new MemoryCache(new MemoryCacheOptions()); + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions { - factoryCalled = true; - return Task.FromResult?>( - new CacheEntry(value, [])); + IsFailSafeEnabled = true, + FailSafeMaxDuration = TimeSpan.FromMinutes(5), + FailSafeThrottleDuration = TimeSpan.Zero }); - Assert.True(factoryCalled); - Assert.NotNull(result); + await PopulateCache(manager, "stale_key", new TestCacheValue { Id = 1, Name = "Fresh" }, ["dep"]); + var hit = await manager.GetOrSetAsync("stale_key", _ => throw new InvalidOperationException("not expected")); + await manager.InvalidateAsync(["dep"]); + var stale = await manager.GetOrSetAsync("stale_key", _ => throw new InvalidOperationException("Simulated API failure")); + + Assert.False(hit!.IsStale); + Assert.True(stale!.IsStale); + Assert.False(stale.FromFactory); } [Fact] @@ -223,6 +235,24 @@ public async Task PurgeAsync_WithAllowFailSafe_ExpiresEntriesAndDoesNotAffectNew Assert.False(await IsFactoryCalledAsync("k3")); } + [Fact] + public async Task PurgeAsync_OverASharedMemoryCache_LeavesTheOtherClientsEntriesAlone() + { + // The purge marker is a key of FusionCache's own. It carries the client's prefix like everything + // else FusionCache stores for that client, so another client sharing the memory cache never reads it. + using var sharedCache = new MemoryCache(new MemoryCacheOptions()); + using var a = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "a" }); + using var b = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "b" }); + + await PopulateCache(a, "k", new TestCacheValue { Id = 1, Name = "A" }, ["dep"]); + await PopulateCache(b, "k", new TestCacheValue { Id = 2, Name = "B" }, ["dep"]); + + await ((IDeliveryCachePurger)a).PurgeAsync(); + + Assert.True(await IsFactoryCalledAsync(a, "k")); + Assert.False(await IsFactoryCalledAsync(b, "k")); + } + [Fact] public async Task PurgeAsync_DoesNotAffectEntriesCreatedAfterPurge() { @@ -242,7 +272,7 @@ public async Task ExpiredDependencyEntry_DoesNotBreakFutureInvalidation() { ExpirationScanFrequency = TimeSpan.FromMilliseconds(10) }); - using var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions()); + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions()); var dependency = "dep1"; var expiration = TimeSpan.FromMilliseconds(50); @@ -406,6 +436,18 @@ public async Task InvalidateAsync_WithCaseVariantKeys_RemovesAllMatchingEntries( Assert.True(await IsFactoryCalledAsync("key")); } + [Fact] + public async Task InvalidateAsync_MatchesKeysCaseInsensitively() + { + // The SDK tags lower-case; a webhook handler that copies a codename from a payload in another + // casing must still evict. + await PopulateCache("hero", new TestCacheValue { Id = 1, Name = "Hero" }, [DeliveryCacheDependencies.ForItem("hero")]); + + await _cacheManager.InvalidateAsync(["ITEM_HERO"]); + + Assert.True(await IsFactoryCalledAsync("hero")); + } + #endregion #region Concurrency Tests @@ -490,7 +532,7 @@ public async Task ReverseIndex_ConcurrentCleanupAndSet_PreservesDependencyInvali { ExpirationScanFrequency = TimeSpan.FromMilliseconds(10) }); - using var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions()); + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions()); const string dependency = "dep_race"; const string expiringKey = "expiring_key"; @@ -531,7 +573,7 @@ public async Task ReverseIndex_ConcurrentCleanupAndSet_PreservesDependencyInvali public async Task Dispose_DisposesResources() { var cache = new MemoryCache(new MemoryCacheOptions()); - var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions()); + var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions()); await PopulateCache(manager, "test_key", new TestCacheValue { Id = 1, Name = "Test" }, ["dep1"]); @@ -546,7 +588,7 @@ await Assert.ThrowsAsync(() => public void Dispose_CalledMultipleTimes_DoesNotThrow() { var cache = new MemoryCache(new MemoryCacheOptions()); - var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions()); + var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions()); var exception = Record.Exception(() => { @@ -562,7 +604,7 @@ public void Dispose_CalledMultipleTimes_DoesNotThrow() public async Task Dispose_WithPendingOperations_CleansUpCorrectly() { var cache = new MemoryCache(new MemoryCacheOptions()); - var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions()); + var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions()); await PopulateCache(manager, "test_key", new TestCacheValue { Id = 1, Name = "Test" }, ["dep1", "dep2", "dep3"]); @@ -666,8 +708,8 @@ await _cacheManager.GetOrSetAsync("complex_key", _ => public async Task GetOrSetAsync_WithDifferentPrefixes_IsolatesCacheEntries() { var sharedCache = new MemoryCache(new MemoryCacheOptions()); - using var manager1 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); - using var manager2 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); + using var manager1 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); + using var manager2 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); var key = "same_key"; var value1 = new TestCacheValue { Id = 1, Name = "Client1Value" }; @@ -693,8 +735,8 @@ public async Task GetOrSetAsync_WithDifferentPrefixes_IsolatesCacheEntries() public async Task InvalidateAsync_WithDifferentPrefixes_OnlyAffectsOwnEntries() { var sharedCache = new MemoryCache(new MemoryCacheOptions()); - using var manager1 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); - using var manager2 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); + using var manager1 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); + using var manager2 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); var key = "same_key"; var dependency = "same_dep"; @@ -714,8 +756,8 @@ public async Task InvalidateAsync_WithDifferentPrefixes_OnlyAffectsOwnEntries() public async Task GetOrSetAsync_WithDifferentPrefixes_DoesNotCrossContaminate() { var sharedCache = new MemoryCache(new MemoryCacheOptions()); - using var manager1 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); - using var manager2 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); + using var manager1 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); + using var manager2 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); var key = "unique_key"; await PopulateCache(manager1, key, new TestCacheValue { Id = 1, Name = "OnlyInClient1" }, []); @@ -730,8 +772,8 @@ public async Task GetOrSetAsync_WithDifferentPrefixes_DoesNotCrossContaminate() public async Task GetOrSetAsync_WithNullPrefix_UsesUnprefixedKeys() { var sharedCache = new MemoryCache(new MemoryCacheOptions()); - using var managerNoPrefix = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = null }); - using var managerWithPrefix = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "prefixed" }); + using var managerNoPrefix = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = null }); + using var managerWithPrefix = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "prefixed" }); var key = "test_key"; await PopulateCache(managerNoPrefix, key, new TestCacheValue { Id = 1, Name = "NoPrefix" }, []); @@ -752,8 +794,8 @@ public async Task GetOrSetAsync_WithNullPrefix_UsesUnprefixedKeys() public async Task InvalidateAsync_WithSharedDependencyName_OnlyInvalidatesOwnPrefix() { var sharedCache = new MemoryCache(new MemoryCacheOptions()); - using var manager1 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "prod" }); - using var manager2 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "preview" }); + using var manager1 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "prod" }); + using var manager2 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "preview" }); var dependency = "content_type_article"; @@ -776,8 +818,8 @@ public async Task InvalidateAsync_WithSharedDependencyName_OnlyInvalidatesOwnPre public async Task ConcurrentOperations_WithDifferentPrefixes_MaintainsIsolation() { var sharedCache = new MemoryCache(new MemoryCacheOptions()); - using var manager1 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); - using var manager2 = new MemoryCacheManager(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); + using var manager1 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client1" }); + using var manager2 = FusionCacheManager.CreateMemory(sharedCache, new DeliveryCacheOptions { KeyPrefix = "client2" }); var dependency = "shared_dep_name"; @@ -809,7 +851,7 @@ public async Task ConcurrentOperations_WithDifferentPrefixes_MaintainsIsolation( public async Task FailSafe_Enabled_ServesStaleEntryAfterExpiration() { using var cache = new MemoryCache(new MemoryCacheOptions()); - using var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMilliseconds(100), IsFailSafeEnabled = true, @@ -847,10 +889,12 @@ public async Task FailSafe_Enabled_ServesStaleEntryAfterExpiration() } [Fact] - public async Task FailSafe_Enabled_ServesStaleEntryWhenFactoryReturnsNull() + public async Task FailSafe_Enabled_FactoryReturnsNullAfterExpiration_DropsTheStaleEntry() { + // Null is the origin's answer that there is no value, not an outage: the stale copy must not be + // served for it, and must not be left for the next outage to serve either. using var cache = new MemoryCache(new MemoryCacheOptions()); - using var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMilliseconds(100), IsFailSafeEnabled = true, @@ -861,37 +905,78 @@ public async Task FailSafe_Enabled_ServesStaleEntryWhenFactoryReturnsNull() var value = new TestCacheValue { Id = 99, Name = "StaleFromNull" }; await PopulateCache(manager, "failsafe_null_key", value, ["dep1"]); - // Poll until TTL expires (factory is called) and factory returns null. - TestCacheValue? failSafeResult = null; + CacheResult? resultAfterExpiry = null; var expired = await WaitUntilAsync( async () => { var factoryCalled = false; - var r = await manager.GetOrSetAsync("failsafe_null_key", _ => + resultAfterExpiry = await manager.GetOrSetAsync("failsafe_null_key", _ => { factoryCalled = true; return Task.FromResult?>(null); }); - if (factoryCalled) - { - failSafeResult = r?.Value; - } return factoryCalled; }, timeout: TimeSpan.FromSeconds(2), pollInterval: TimeSpan.FromMilliseconds(20)); Assert.True(expired, "Cache entry did not expire within timeout"); - Assert.NotNull(failSafeResult); - Assert.Equal(99, failSafeResult.Id); - Assert.Equal("StaleFromNull", failSafeResult.Name); + Assert.Null(resultAfterExpiry); + + await Assert.ThrowsAsync(() => + manager.GetOrSetAsync("failsafe_null_key", _ => throw new InvalidOperationException("Simulated API failure"))); + } + + [Fact] + public async Task FailSafe_Enabled_FactoryReturnsNullAfterInvalidation_DropsTheStaleEntry() + { + // The webhook case: invalidation expires the entry rather than removing it, so a stale copy is + // there to serve. The origin's answer supersedes it; an outage right after does not resurrect it. + using var cache = new MemoryCache(new MemoryCacheOptions()); + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions + { + IsFailSafeEnabled = true, + FailSafeMaxDuration = TimeSpan.FromMinutes(5), + FailSafeThrottleDuration = TimeSpan.Zero + }); + + await PopulateCache(manager, "invalidated_key", new TestCacheValue { Id = 7, Name = "Gone" }, ["item_gone"]); + await manager.InvalidateAsync(["item_gone"]); + + var answer = await manager.GetOrSetAsync("invalidated_key", _ => + Task.FromResult?>(null)); + + Assert.Null(answer); + await Assert.ThrowsAsync(() => + manager.GetOrSetAsync("invalidated_key", _ => throw new InvalidOperationException("Simulated API failure"))); + } + + [Fact] + public async Task FailSafe_Enabled_FactoryThrowsAfterInvalidation_ServesTheStaleEntry() + { + using var cache = new MemoryCache(new MemoryCacheOptions()); + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions + { + IsFailSafeEnabled = true, + FailSafeMaxDuration = TimeSpan.FromMinutes(5), + FailSafeThrottleDuration = TimeSpan.Zero + }); + + await PopulateCache(manager, "outage_key", new TestCacheValue { Id = 8, Name = "Stale" }, ["item_stale"]); + await manager.InvalidateAsync(["item_stale"]); + + var served = await manager.GetOrSetAsync("outage_key", _ => + throw new InvalidOperationException("Simulated API failure")); + + Assert.NotNull(served); + Assert.Equal("Stale", served.Value.Name); } [Fact] public async Task FailSafe_Disabled_ReturnsNullWhenFactoryReturnsNull() { using var cache = new MemoryCache(new MemoryCacheOptions()); - using var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMilliseconds(100), IsFailSafeEnabled = false @@ -928,7 +1013,7 @@ public async Task FailSafe_Disabled_ReturnsNullWhenFactoryReturnsNull() public async Task FailSafe_Enabled_NoStaleEntry_ReturnsNullWhenFactoryReturnsNull() { using var cache = new MemoryCache(new MemoryCacheOptions()); - using var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5), IsFailSafeEnabled = true, @@ -947,7 +1032,7 @@ public async Task FailSafe_Enabled_NoStaleEntry_ReturnsNullWhenFactoryReturnsNul public async Task FailSafe_Disabled_ReturnsNullAfterExpiration() { using var cache = new MemoryCache(new MemoryCacheOptions()); - using var manager = new MemoryCacheManager(cache, new DeliveryCacheOptions + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMilliseconds(100), IsFailSafeEnabled = false @@ -966,13 +1051,76 @@ public async Task FailSafe_Disabled_ReturnsNullAfterExpiration() #endregion + [Fact] + public async Task ConfigureFusionCache_DefaultEntryOptions_ReachTheWrites() + { + // What the consumer sets on DefaultEntryOptions is the starting point of every write. A Size of + // 100 under a limit of 150 means the second entry does not fit and is refused - which it would not + // be if the SDK's own Size of 1 had won. + using var cache = new MemoryCache(new MemoryCacheOptions { SizeLimit = 150 }); + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions() + .ConfigureFusionCache(fusion => fusion.DefaultEntryOptions.Size = 100)); + + await PopulateCache(manager, "first", new TestCacheValue { Id = 1, Name = "First" }, ["dep1"]); + await PopulateCache(manager, "second", new TestCacheValue { Id = 2, Name = "Second" }, ["dep1"]); + + Assert.False(await IsFactoryCalledAsync(manager, "first")); + Assert.True(await IsFactoryCalledAsync(manager, "second")); + } + + [Fact] + public async Task InvalidateAsync_IsRememberedForTagsDefaultEntryOptionsDuration() + { + // An invalidation is a tag-expiration entry the next read of each tagged entry checks, so it has to + // outlive the entries it applies to. It is stored with TagsDefaultEntryOptions - ten days unless the + // consumer shortens it, as this test does to make the lifetime observable - not with the write + // options, whose duration would forget a webhook's invalidation long before a quiet entry was read. + // The wait is the observation: a read while the tag data is live applies the invalidation and + // removes the entry, so there is nothing to poll. + TimeSpan? handedToTheHook = null; + using var cache = new MemoryCache(new MemoryCacheOptions()); + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions() + .ConfigureFusionCache(fusion => + { + handedToTheHook = fusion.TagsDefaultEntryOptions.Duration; + fusion.TagsDefaultEntryOptions.Duration = TimeSpan.FromMilliseconds(100); + })); + + Assert.Equal(TimeSpan.FromDays(10), handedToTheHook); + + await PopulateCache(manager, "quiet_key", new TestCacheValue { Id = 1, Name = "Quiet" }, ["dep1"]); + await manager.InvalidateAsync(["dep1"]); + await Task.Delay(400); + + // The shortened tag data has lapsed, so the entry it applied to is served again. + Assert.False(await IsFactoryCalledAsync(manager, "quiet_key")); + } + + [Fact] + public async Task SizeLimitedMemoryCache_WritesInvalidatesAndPurges() + { + // The application's memory cache may carry a size limit, and a cache with one refuses entries that + // declare no size - so every entry the manager writes declares one, tag entries included. + using var cache = new MemoryCache(new MemoryCacheOptions { SizeLimit = 1000 }); + using var manager = FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions()); + + await PopulateCache(manager, "sized_key", new TestCacheValue { Id = 1, Name = "Sized" }, ["dep1"]); + Assert.False(await IsFactoryCalledAsync(manager, "sized_key")); + + await manager.InvalidateAsync(["dep1"]); + Assert.True(await IsFactoryCalledAsync(manager, "sized_key")); + + await ((IDeliveryCachePurger)manager).PurgeAsync(); + Assert.True(await IsFactoryCalledAsync(manager, "sized_key")); + } + #region Jitter Tests [Fact] public void Jitter_DoesNotThrow() { using var cache = new MemoryCache(new MemoryCacheOptions()); - var exception = Record.Exception(() => new MemoryCacheManager(cache, new DeliveryCacheOptions + var exception = Record.Exception(() => FusionCacheManager.CreateMemory(cache, new DeliveryCacheOptions { DefaultExpiration = TimeSpan.FromMinutes(5), JitterMaxDuration = TimeSpan.FromSeconds(30) @@ -1048,7 +1196,7 @@ public async Task ConfigureFusionCacheOptions_Callback_IsInvoked() }; using var memoryCache = new MemoryCache(new MemoryCacheOptions()); - using var manager = new MemoryCacheManager(memoryCache, options); + using var manager = FusionCacheManager.CreateMemory(memoryCache, options); // The callback should have been invoked during construction Assert.True(callbackInvoked); @@ -1073,7 +1221,7 @@ public void ConfigureFusionCacheOptions_Callback_ReceivesFusionCacheOptions() }; using var memoryCache = new MemoryCache(new MemoryCacheOptions()); - using var manager = new MemoryCacheManager(memoryCache, options); + using var manager = FusionCacheManager.CreateMemory(memoryCache, options); Assert.NotNull(receivedOptions); Assert.Equal("ZiggyCreatures.Caching.Fusion.FusionCacheOptions", receivedOptions.GetType().FullName); diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Caching/ThrowingDistributedCache.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/ThrowingDistributedCache.cs new file mode 100644 index 000000000..4737b213b --- /dev/null +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Caching/ThrowingDistributedCache.cs @@ -0,0 +1,20 @@ +using Microsoft.Extensions.Caching.Distributed; + +namespace Kontent.Ai.Delivery.Tests.Caching; + +/// +/// A distributed cache that is down: every operation fails the way a lost Redis connection does. +/// +internal sealed class ThrowingDistributedCache : IDistributedCache +{ + private static IOException Down() => new("The distributed cache is unavailable."); + + public byte[]? Get(string key) => throw Down(); + public Task GetAsync(string key, CancellationToken token = default) => throw Down(); + public void Refresh(string key) => throw Down(); + public Task RefreshAsync(string key, CancellationToken token = default) => throw Down(); + public void Remove(string key) => throw Down(); + public Task RemoveAsync(string key, CancellationToken token = default) => throw Down(); + public void Set(string key, byte[] value, DistributedCacheEntryOptions options) => throw Down(); + public Task SetAsync(string key, byte[] value, DistributedCacheEntryOptions options, CancellationToken token = default) => throw Down(); +} diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/Extensions/ServiceCollectionsExtensionsTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/Extensions/ServiceCollectionsExtensionsTests.cs index 8f08125d2..d5cfca6bd 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/Extensions/ServiceCollectionsExtensionsTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/Extensions/ServiceCollectionsExtensionsTests.cs @@ -468,7 +468,46 @@ public void UseMemoryCache_RegistersKeyedCacheManager() var cacheManager = provider.GetKeyedService("production"); Assert.NotNull(cacheManager); - Assert.IsType(cacheManager); + Assert.Equal(CacheStorageMode.HydratedObject, cacheManager.StorageMode); + } + + [Fact] + public void UseMemoryCache_DefaultClient_RegistersTheManagerUnkeyedToo() + { + // The route a webhook handler takes for the default client: IDeliveryCacheManager, no key. + _serviceCollection.AddDeliveryClient(d => + { + d.Options.Configure(o => + { + o.EnvironmentId = EnvironmentId; + o.EnableResilience = false; + }); + d.UseMemoryCache(); + }); + + var provider = _serviceCollection.BuildServiceProvider(); + + Assert.Same( + provider.GetRequiredKeyedService("Default"), + provider.GetRequiredService()); + } + + [Fact] + public void UseMemoryCache_NamedClient_RegistersNoUnkeyedManager() + { + _serviceCollection.AddDeliveryClient("production", d => + { + d.Options.Configure(o => + { + o.EnvironmentId = EnvironmentId; + o.EnableResilience = false; + }); + d.UseMemoryCache(); + }); + + var provider = _serviceCollection.BuildServiceProvider(); + + Assert.Null(provider.GetService()); } [Fact] @@ -489,7 +528,7 @@ public void UseHybridCache_RegistersKeyedCacheManager() var cacheManager = provider.GetKeyedService("production"); Assert.NotNull(cacheManager); - Assert.IsType(cacheManager); + Assert.Equal(CacheStorageMode.RawJson, cacheManager.StorageMode); } [Fact] @@ -643,7 +682,7 @@ public void UseMemoryCache_AfterHybridCache_ReplacesPreviousCacheManagerRegistra var provider = _serviceCollection.BuildServiceProvider(); var cacheManager = provider.GetRequiredKeyedService("production"); - Assert.IsType(cacheManager); + Assert.Equal(CacheStorageMode.HydratedObject, cacheManager.StorageMode); } [Fact] @@ -666,7 +705,7 @@ public void UseHybridCache_AfterMemoryCache_ReplacesPreviousCacheManagerRegistra var provider = _serviceCollection.BuildServiceProvider(); var cacheManager = provider.GetRequiredKeyedService("production"); - Assert.IsType(cacheManager); + Assert.Equal(CacheStorageMode.RawJson, cacheManager.StorageMode); } [Fact] @@ -710,7 +749,7 @@ public void UseMemoryCache_AfterCustomCacheManager_ReplacesPreviousCacheManagerR var provider = _serviceCollection.BuildServiceProvider(); var cacheManager = provider.GetRequiredKeyedService("production"); - Assert.IsType(cacheManager); + Assert.Equal(CacheStorageMode.HydratedObject, cacheManager.StorageMode); } [Fact] @@ -734,7 +773,7 @@ public async Task UseMemoryCache_DefaultClient_AddsNoClientNameToTheKeys() var sharedMemoryCache = provider.GetRequiredService(); // Same environment, no client name: the default client adds nothing of its own, so this manager // sees its entries. A named client would prefix them and this would miss. - using var unprefixedManager = new MemoryCacheManager( + using var unprefixedManager = FusionCacheManager.CreateMemory( sharedMemoryCache, new DeliveryCacheOptions { @@ -836,7 +875,7 @@ public void UseMemoryCache_WithCustomExpiration_PassesExpirationToManager() var cacheManager = provider.GetKeyedService("production"); Assert.NotNull(cacheManager); - Assert.IsType(cacheManager); + Assert.Equal(CacheStorageMode.HydratedObject, cacheManager.StorageMode); } [Fact] @@ -1055,7 +1094,7 @@ public void UseMemoryCache_WithServiceProviderCallback_InvokesCallbackOnResoluti var cacheManager = provider.GetKeyedService("Default"); Assert.NotNull(cacheManager); - Assert.IsType(cacheManager); + Assert.Equal(CacheStorageMode.HydratedObject, cacheManager.StorageMode); Assert.Equal(TimeSpan.FromHours(3), invokedWithExpiration); } @@ -1151,7 +1190,7 @@ public void UseHybridCache_WithServiceProviderCallback_InvokesCallbackOnResoluti var cacheManager = provider.GetKeyedService("Default"); Assert.NotNull(cacheManager); - Assert.IsType(cacheManager); + Assert.Equal(CacheStorageMode.RawJson, cacheManager.StorageMode); Assert.Equal(TimeSpan.FromHours(4), invokedWithExpiration); } diff --git a/src/delivery/Kontent.Ai.Delivery.Tests/QueryBuilders/CachedQueryExecutorTests.cs b/src/delivery/Kontent.Ai.Delivery.Tests/QueryBuilders/CachedQueryExecutorTests.cs index d8e62b0b5..3ba77d65a 100644 --- a/src/delivery/Kontent.Ai.Delivery.Tests/QueryBuilders/CachedQueryExecutorTests.cs +++ b/src/delivery/Kontent.Ai.Delivery.Tests/QueryBuilders/CachedQueryExecutorTests.cs @@ -1,6 +1,5 @@ using Kontent.Ai.Delivery.Abstractions; using Kontent.Ai.Delivery.Api.QueryBuilders.Helpers; -using Kontent.Ai.Delivery.Caching; using Kontent.Ai.Delivery.SharedModels; namespace Kontent.Ai.Delivery.Tests.QueryBuilders; @@ -8,23 +7,17 @@ namespace Kontent.Ai.Delivery.Tests.QueryBuilders; /// /// Pins how a cached query's result is classified. Under eager refresh the factory also runs on a /// background thread while the stale-but-valid value is returned immediately, so anything it records may -/// belong to a different call than the one reading it. +/// belong to a different call than the one reading it; only what the cache says about the value counts. /// public class CachedQueryExecutorTests { - private const string Key = "items|list"; - [Fact] public async Task ExecuteAsync_BackgroundRefreshWroteAnApiResult_StillReportsACacheHit() { // The shape of an eager-refresh hit: the cache returns a stored value (FromFactory false) while a // background factory writes into the same captured local this call reads. That local says nothing // about the value being returned. - var manager = new StubCacheManager(failSafeActive: false); - var outcome = await CachedQueryExecutor.ExecuteAsync( - manager, - Key, (captureApiResult, _) => { captureApiResult(SuccessfulApiResult()); @@ -38,15 +31,11 @@ public async Task ExecuteAsync_BackgroundRefreshWroteAnApiResult_StillReportsACa } [Fact] - public async Task ExecuteAsync_ServedStaleWhileFailSafeActive_ReportsFailSafe() + public async Task ExecuteAsync_ServedStale_ReportsFailSafe() { - var manager = new StubCacheManager(failSafeActive: true); - var outcome = await CachedQueryExecutor.ExecuteAsync( - manager, - Key, (_, _) => Task.FromResult?>( - new CacheResult("stale", []) { FromFactory = false }), + new CacheResult("stale", []) { FromFactory = false, IsStale = true }), CancellationToken.None); Assert.Equal(CachedQuerySource.FailSafeHit, outcome.Source); @@ -55,12 +44,9 @@ public async Task ExecuteAsync_ServedStaleWhileFailSafeActive_ReportsFailSafe() [Fact] public async Task ExecuteAsync_FactoryProducedTheValue_ReportsFetchedAndKeepsTheApiResult() { - var manager = new StubCacheManager(failSafeActive: true); var apiResult = SuccessfulApiResult(); var outcome = await CachedQueryExecutor.ExecuteAsync( - manager, - Key, (captureApiResult, _) => { captureApiResult(apiResult); @@ -69,7 +55,6 @@ public async Task ExecuteAsync_FactoryProducedTheValue_ReportsFetchedAndKeepsThe }, CancellationToken.None); - // Fail-safe state is irrelevant once this call's own factory produced the value. Assert.Equal(CachedQuerySource.Fetched, outcome.Source); Assert.Same(apiResult, outcome.ApiResult); } @@ -77,12 +62,9 @@ public async Task ExecuteAsync_FactoryProducedTheValue_ReportsFetchedAndKeepsThe [Fact] public async Task ExecuteAsync_NothingCached_ReportsFetchedSoTheFailureSurfaces() { - var manager = new StubCacheManager(failSafeActive: false); var failure = DeliveryResult.Failure("url", System.Net.HttpStatusCode.NotFound, new Error()); var outcome = await CachedQueryExecutor.ExecuteAsync( - manager, - Key, (captureApiResult, _) => { captureApiResult(failure); @@ -94,21 +76,49 @@ public async Task ExecuteAsync_NothingCached_ReportsFetchedSoTheFailureSurfaces( Assert.Same(failure, outcome.ApiResult); } - private static IDeliveryResult SuccessfulApiResult() => - DeliveryResult.Success("value", "url", System.Net.HttpStatusCode.OK, false, null, ResponseSource.Origin); - - private sealed class StubCacheManager(bool failSafeActive) : IDeliveryCacheManager, IFailSafeStateProvider + [Fact] + public async Task ExecuteAsync_OriginUnavailableAndNothingStale_ReportsFetchedWithTheCarriedFailure() { - public bool IsFailSafeActive(string cacheKey) => failSafeActive; + // The factory threw for an outage and the manager had no stale copy, so the exception surfaces + // here carrying the failed result - which is the answer even for a caller whose own factory never + // ran because it was waiting on the same key. + var failure = DeliveryResult.Failure("url", System.Net.HttpStatusCode.ServiceUnavailable, new Error()); - public Task?> GetOrSetAsync( - string cacheKey, - Func?>> factory, - TimeSpan? expiration = null, - CancellationToken cancellationToken = default) where T : class => - throw new NotSupportedException("The executor is handed the cache call, it does not make one."); + var outcome = await CachedQueryExecutor.ExecuteAsync( + (_, _) => throw new OriginUnavailableException(failure), + CancellationToken.None); - public Task InvalidateAsync(string[] dependencyKeys, CancellationToken cancellationToken = default) => - Task.FromResult(true); + Assert.Equal(CachedQuerySource.Fetched, outcome.Source); + Assert.Null(outcome.Cached); + Assert.Same(failure, outcome.ApiResult); + } + + [Theory] + [InlineData(System.Net.HttpStatusCode.NotFound, false)] + [InlineData(System.Net.HttpStatusCode.Forbidden, false)] + [InlineData(System.Net.HttpStatusCode.BadRequest, false)] + [InlineData(default(System.Net.HttpStatusCode), true)] + [InlineData(System.Net.HttpStatusCode.RequestTimeout, true)] + [InlineData(System.Net.HttpStatusCode.TooManyRequests, true)] + [InlineData(System.Net.HttpStatusCode.InternalServerError, true)] + [InlineData(System.Net.HttpStatusCode.ServiceUnavailable, true)] + public void ThrowIfOriginUnavailable_ThrowsForOutagesOnly(System.Net.HttpStatusCode status, bool isOutage) + { + var failure = DeliveryResult.Failure("url", status, new Error()); + + var act = () => CachedQueryExecutor.ThrowIfOriginUnavailable(failure); + + if (isOutage) + { + var thrown = Assert.Throws(act); + Assert.Same(failure, thrown.Result); + } + else + { + act(); + } } + + private static IDeliveryResult SuccessfulApiResult() => + DeliveryResult.Success("value", "url", System.Net.HttpStatusCode.OK, false, null, ResponseSource.Origin); } diff --git a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/CachedItemsFetch.cs b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/CachedItemsFetch.cs new file mode 100644 index 000000000..f16d80546 --- /dev/null +++ b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/CachedItemsFetch.cs @@ -0,0 +1,114 @@ +using Kontent.Ai.Delivery.Caching; + +namespace Kontent.Ai.Delivery.Api.QueryBuilders.Helpers; + +/// +/// The cached fetch the item and listing queries share. The two storage modes differ in what is stored - +/// the hydrated value, or a raw payload rehydrated on every hit - and in nothing else, so the fetch, the +/// failure handling and the provenance the manager reports are in one place. +/// +internal static class CachedItemsFetch +{ + /// The manager to read through. + /// The query's key. + /// The query's expiration override, if any. + /// Calls the API. + /// Records what the factory fetched, for . + /// Hydrates a response and collects the dependencies it carries. + /// The raw payload to store in mode. + /// Rebuilds the value from a stored payload in mode. + /// A token to cancel the operation. + internal static Task?> ExecuteAsync( + IDeliveryCacheManager cacheManager, + string cacheKey, + TimeSpan? expiration, + Func>> fetch, + Action> captureApiResult, + Func> process, + Func toPayload, + Func> rehydrate, + CancellationToken cancellationToken) + where THydrated : class + where TApi : class + => cacheManager.StorageMode == CacheStorageMode.RawJson + ? ExecuteRawAsync(cacheManager, cacheKey, expiration, fetch, captureApiResult, process, toPayload, rehydrate, cancellationToken) + : cacheManager.GetOrSetAsync( + cacheKey, + async ct => + { + var fetched = await FetchAsync(fetch, captureApiResult, process, ct).ConfigureAwait(false); + return fetched is null ? null : new CacheEntry(fetched.Hydrated, fetched.Dependencies); + }, + expiration, + cancellationToken); + + private static async Task?> ExecuteRawAsync( + IDeliveryCacheManager cacheManager, + string cacheKey, + TimeSpan? expiration, + Func>> fetch, + Action> captureApiResult, + Func> process, + Func toPayload, + Func> rehydrate, + CancellationToken cancellationToken) + where THydrated : class + where TApi : class + { + THydrated? hydratedHere = null; + + var cached = await cacheManager.GetOrSetAsync( + cacheKey, + async ct => + { + var fetched = await FetchAsync(fetch, captureApiResult, process, ct).ConfigureAwait(false); + if (fetched is null) + return null; + + hydratedHere = fetched.Hydrated; + return new CacheEntry(toPayload(fetched.Hydrated, fetched.Response), fetched.Dependencies); + }, + expiration, + cancellationToken).ConfigureAwait(false); + + if (cached is null) + return null; + + // Hydrating a miss twice is what this avoids: the factory already built the value in order to + // collect the dependency keys, and rehydrating parses and maps the very same payload again. Only + // this call's own factory result can be reused - FromFactory is false for a cache hit and for a + // background refresh, both of which still rehydrate. + var value = cached.FromFactory && hydratedHere is not null + ? hydratedHere + : await rehydrate(cached.Value, cancellationToken).ConfigureAwait(false); + + // The rehydrated value replaces the stored payload; what the manager said about it carries across. + return new CacheResult(value, cached.DependencyKeys) + { + FromFactory = cached.FromFactory, + IsStale = cached.IsStale, + }; + } + + private static async Task?> FetchAsync( + Func>> fetch, + Action> captureApiResult, + Func> process, + CancellationToken cancellationToken) + where THydrated : class + where TApi : class + { + var result = await fetch(cancellationToken).ConfigureAwait(false); + captureApiResult(result); + if (!result.IsSuccess) + { + CachedQueryExecutor.ThrowIfOriginUnavailable(result); + return null; + } + + var (hydrated, dependencies) = await process(result.Value, cancellationToken).ConfigureAwait(false); + return new Fetched(hydrated, dependencies, result.Value); + } + + private sealed record Fetched(THydrated Hydrated, string[] Dependencies, TApi Response); +} diff --git a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/CachedQueryExecutor.cs b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/CachedQueryExecutor.cs index c8580d614..847acaa8e 100644 --- a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/CachedQueryExecutor.cs +++ b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/CachedQueryExecutor.cs @@ -1,4 +1,4 @@ -using Kontent.Ai.Delivery.Caching; +using Kontent.Ai.Common.Http; namespace Kontent.Ai.Delivery.Api.QueryBuilders.Helpers; @@ -41,41 +41,61 @@ internal readonly record struct CachedQueryOutcome( /// Nothing the factory records can be trusted here. With eager refresh enabled, FusionCache returns the /// stale-but-valid value immediately and runs the factory on a background thread, so a captured local may /// be written by a different call than the one reading it. The cache is the only component that knows -/// which value it handed back, so the decision comes from and, -/// for staleness, from the manager's fail-safe state. +/// which value it handed back, so the decision comes from and +/// . /// internal static class CachedQueryExecutor { - /// The manager to read through. - /// The key being read, used to probe fail-safe state. /// /// Performs the GetOrSetAsync call. Receives the callback the factory must use to record what /// it fetched; the storage-mode split and any post-processing stay with the caller. /// /// A token to cancel the operation. internal static async Task> ExecuteAsync( - IDeliveryCacheManager cacheManager, - string cacheKey, Func>, CancellationToken, Task?>> runCachedFetch, CancellationToken cancellationToken) where TCached : class where TApi : class { IDeliveryResult? apiResult = null; + CacheResult? cached; - var cached = await runCachedFetch(result => apiResult = result, cancellationToken).ConfigureAwait(false); + try + { + cached = await runCachedFetch(result => apiResult = result, cancellationToken).ConfigureAwait(false); + } + catch (OriginUnavailableException unavailable) + { + // The manager had no stale copy to fall back on, so the outage is this call's answer. + return new CachedQueryOutcome( + CachedQuerySource.Fetched, + Cached: null, + (IDeliveryResult)unavailable.Result); + } if (cached is not null && !cached.FromFactory) { - var isFailSafe = cacheManager is IFailSafeStateProvider failSafeProvider - && failSafeProvider.IsFailSafeActive(cacheKey); - return new CachedQueryOutcome( - isFailSafe ? CachedQuerySource.FailSafeHit : CachedQuerySource.CacheHit, + cached.IsStale ? CachedQuerySource.FailSafeHit : CachedQuerySource.CacheHit, cached, ApiResult: null); } return new CachedQueryOutcome(CachedQuerySource.Fetched, cached, apiResult); } + + /// + /// What a factory does with a failed fetch. An outage - no response at all, or a status the SDK's own + /// pipeline would retry - is thrown, so a cache manager with fail-safe can serve a stale copy. Anything + /// else is the origin's answer, and the factory returns null for it: nothing to cache, and no + /// stale copy either, or an unpublished item would keep being served for as long as fail-safe allows. + /// + internal static void ThrowIfOriginUnavailable(IDeliveryResult result) + where TApi : class + { + if (result.StatusCode == default || HttpRetryPredicates.IsRetryableStatusCode(result.StatusCode)) + { + throw new OriginUnavailableException(result); + } + } } diff --git a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/OriginUnavailableException.cs b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/OriginUnavailableException.cs new file mode 100644 index 000000000..bca13d336 --- /dev/null +++ b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/Helpers/OriginUnavailableException.cs @@ -0,0 +1,14 @@ +namespace Kontent.Ai.Delivery.Api.QueryBuilders.Helpers; + +/// +/// Thrown from a cached query's factory when the origin could not be reached, so a cache manager with +/// fail-safe may serve a stale copy. Carries the failed result: the factory runs once for every caller +/// waiting on the same key, and only the caller whose factory ran has captured one. +/// +#pragma warning disable S3871 // Internal on purpose: thrown and caught by the SDK's own cached-query path, never by a consumer +internal sealed class OriginUnavailableException(IDeliveryResult result) + : Exception("The Delivery API could not be reached.") +{ + public IDeliveryResult Result { get; } = result; +} +#pragma warning restore S3871 diff --git a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/ItemQuery.cs b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/ItemQuery.cs index 5a290dcda..d4b9af50d 100644 --- a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/ItemQuery.cs +++ b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/ItemQuery.cs @@ -83,11 +83,24 @@ private async Task>> ExecuteWithCacheAsync( var cacheKey = BuildCacheKey(cacheManager.StorageMode); var outcome = await CachedQueryExecutor.ExecuteAsync, DeliveryItemResponse>( - cacheManager, - cacheKey, - (captureApiResult, ct) => cacheManager.StorageMode == CacheStorageMode.RawJson - ? ExecuteWithRawJsonCacheAsync(cacheManager, cacheKey, waitForLoadingNewContent, captureApiResult, ct) - : ExecuteWithHydratedCacheAsync(cacheManager, cacheKey, waitForLoadingNewContent, captureApiResult, ct), + (captureApiResult, ct) => CachedItemsFetch.ExecuteAsync, DeliveryItemResponse>( + cacheManager, + cacheKey, + CacheExpiration, + fetch: token => FetchFromApiAsync(waitForLoadingNewContent, token), + captureApiResult, + process: ProcessItemAsync, + toPayload: (item, response) => CachedRawItemsPayload.FromItem(item, response.ModularContent), + rehydrate: async (payload, token) => await CachePayloadHelper.RehydrateItemAsync( + payload, + contentDeserializer, + contentItemMapper, + IsDynamicModel, + defaultRenditionPreset, + customAssetDomain, + logger, + token).ConfigureAwait(false), + ct), cancellationToken).ConfigureAwait(false); var cached = outcome.Cached; @@ -113,81 +126,6 @@ private async Task>> ExecuteWithCacheAsync( return WrapSuccess(cached?.Value ?? apiResult.Value.Item, apiResult, cached?.DependencyKeys); } - private async Task>?> ExecuteWithRawJsonCacheAsync( - IDeliveryCacheManager cacheManager, - string cacheKey, - bool? waitForLoadingNewContent, - Action>> captureApiResult, - CancellationToken cancellationToken) - { - IContentItem? hydratedHere = null; - - var cached = await cacheManager.GetOrSetAsync( - cacheKey, - async ct => - { - var result = await FetchFromApiAsync(waitForLoadingNewContent, ct).ConfigureAwait(false); - captureApiResult(result); - if (!result.IsSuccess) - return null; - - var (item, deps) = await ProcessItemAsync(result.Value, ct).ConfigureAwait(false); - hydratedHere = item; - var rawPayload = CachedRawItemsPayload.FromItem(item, result.Value.ModularContent); - return new CacheEntry(rawPayload, deps); - }, - CacheExpiration, - cancellationToken).ConfigureAwait(false); - - if (cached is null) - return null; - - // Hydrating a miss twice is what this avoids: the factory already built the value in order to - // collect the dependency keys, and rehydrating parses and maps the very same payload again. Only - // this call's own factory result can be reused - FromFactory is false for a cache hit and for a - // background refresh, both of which still rehydrate. - if (cached.FromFactory && hydratedHere is not null) - { - return new CacheResult>(hydratedHere, cached.DependencyKeys) { FromFactory = true }; - } - - var item = await CachePayloadHelper.RehydrateItemAsync( - cached.Value, - contentDeserializer, - contentItemMapper, - IsDynamicModel, - defaultRenditionPreset, - customAssetDomain, - logger, - cancellationToken).ConfigureAwait(false); - - // The rehydrated value replaces the stored payload, so provenance has to be carried across. - return new CacheResult>(item, cached.DependencyKeys) { FromFactory = cached.FromFactory }; - } - - private async Task>?> ExecuteWithHydratedCacheAsync( - IDeliveryCacheManager cacheManager, - string cacheKey, - bool? waitForLoadingNewContent, - Action>> captureApiResult, - CancellationToken cancellationToken) - { - return await cacheManager.GetOrSetAsync( - cacheKey, - async ct => - { - var result = await FetchFromApiAsync(waitForLoadingNewContent, ct).ConfigureAwait(false); - captureApiResult(result); - if (!result.IsSuccess) - return null; - - var (item, deps) = await ProcessItemAsync(result.Value, ct).ConfigureAwait(false); - return new CacheEntry>(item, deps); - }, - CacheExpiration, - cancellationToken).ConfigureAwait(false); - } - private async Task>> ExecuteWithoutCacheAsync( Stopwatch? stopwatch, bool? waitForLoadingNewContent, diff --git a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/ItemsQuery.cs b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/ItemsQuery.cs index 3cb649436..4d15a4e5e 100644 --- a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/ItemsQuery.cs +++ b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/ItemsQuery.cs @@ -125,11 +125,24 @@ private async Task>> Execut var cacheKey = BuildCacheKey(cacheManager.StorageMode); var outcome = await CachedQueryExecutor.ExecuteAsync, DeliveryItemListingResponse>( - cacheManager, - cacheKey, - (captureApiResult, ct) => cacheManager.StorageMode == CacheStorageMode.RawJson - ? ExecuteWithRawJsonCacheAsync(cacheManager, cacheKey, waitForLoadingNewContent, captureApiResult, ct) - : ExecuteWithHydratedCacheAsync(cacheManager, cacheKey, waitForLoadingNewContent, captureApiResult, ct), + (captureApiResult, ct) => CachedItemsFetch.ExecuteAsync, DeliveryItemListingResponse>( + cacheManager, + cacheKey, + CacheExpiration, + fetch: token => FetchFromApiAsync(waitForLoadingNewContent, token), + captureApiResult, + process: ProcessItemsAsync, + toPayload: (response, _) => CachedRawItemsPayload.FromListing(response), + rehydrate: (payload, token) => CachePayloadHelper.RehydrateListingAsync( + payload, + contentDeserializer, + contentItemMapper, + IsDynamicModel, + defaultRenditionPreset, + customAssetDomain, + logger, + token), + ct), cancellationToken).ConfigureAwait(false); var cached = outcome.Cached; @@ -158,81 +171,6 @@ private async Task>> Execut return WrapSuccess(WithNextPageFetcher(response), apiResult, cached?.DependencyKeys); } - private async Task>?> ExecuteWithRawJsonCacheAsync( - IDeliveryCacheManager cacheManager, - string cacheKey, - bool? waitForLoadingNewContent, - Action>> captureApiResult, - CancellationToken cancellationToken) - { - DeliveryItemListingResponse? hydratedHere = null; - - var cached = await cacheManager.GetOrSetAsync( - cacheKey, - async ct => - { - var result = await FetchFromApiAsync(waitForLoadingNewContent, ct).ConfigureAwait(false); - captureApiResult(result); - if (!result.IsSuccess) - return null; - - var (response, deps) = await ProcessItemsAsync(result.Value, ct).ConfigureAwait(false); - hydratedHere = response; - var rawPayload = CachedRawItemsPayload.FromListing(response); - return new CacheEntry(rawPayload, deps); - }, - CacheExpiration, - cancellationToken).ConfigureAwait(false); - - if (cached is null) - return null; - - // Hydrating a miss twice is what this avoids: the factory already built the value in order to - // collect the dependency keys, and rehydrating parses and maps the very same payload again. Only - // this call's own factory result can be reused - FromFactory is false for a cache hit and for a - // background refresh, both of which still rehydrate. - if (cached.FromFactory && hydratedHere is not null) - { - return new CacheResult>(hydratedHere, cached.DependencyKeys) { FromFactory = true }; - } - - var response = await CachePayloadHelper.RehydrateListingAsync( - cached.Value, - contentDeserializer, - contentItemMapper, - IsDynamicModel, - defaultRenditionPreset, - customAssetDomain, - logger, - cancellationToken).ConfigureAwait(false); - - // The rehydrated value replaces the stored payload, so provenance has to be carried across. - return new CacheResult>(response, cached.DependencyKeys) { FromFactory = cached.FromFactory }; - } - - private async Task>?> ExecuteWithHydratedCacheAsync( - IDeliveryCacheManager cacheManager, - string cacheKey, - bool? waitForLoadingNewContent, - Action>> captureApiResult, - CancellationToken cancellationToken) - { - return await cacheManager.GetOrSetAsync( - cacheKey, - async ct => - { - var result = await FetchFromApiAsync(waitForLoadingNewContent, ct).ConfigureAwait(false); - captureApiResult(result); - if (!result.IsSuccess) - return null; - - var (response, deps) = await ProcessItemsAsync(result.Value, ct).ConfigureAwait(false); - return new CacheEntry>(response, deps); - }, - CacheExpiration, - cancellationToken).ConfigureAwait(false); - } - private async Task>> ExecuteWithoutCacheAsync( Stopwatch? stopwatch, bool? waitForLoadingNewContent, diff --git a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TaxonomiesQuery.cs b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TaxonomiesQuery.cs index 183ccbc6a..de0bea833 100644 --- a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TaxonomiesQuery.cs +++ b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TaxonomiesQuery.cs @@ -70,8 +70,6 @@ private async Task> ExecuteWit var cacheKey = CacheKeyBuilder.BuildTaxonomiesKey(_params, _serializedFilters); var outcome = await CachedQueryExecutor.ExecuteAsync( - cacheManager, - cacheKey, (captureApiResult, ct) => cacheManager.GetOrSetAsync( cacheKey, async factoryToken => @@ -79,7 +77,10 @@ private async Task> ExecuteWit var result = await FetchFromApiAsync(waitForLoadingNewContent, factoryToken).ConfigureAwait(false); captureApiResult(result); if (!result.IsSuccess) + { + CachedQueryExecutor.ThrowIfOriginUnavailable(result); return null; + } return new CacheEntry(result.Value, BuildDependencies(result.Value.Taxonomies)); }, @@ -149,24 +150,13 @@ private async Task> FetchFromAp return await response.ToDeliveryResultAsync(logger).ConfigureAwait(false); } - private static string[] BuildDependencies(IReadOnlyList taxonomies) - { - var dependencies = new HashSet(StringComparer.OrdinalIgnoreCase) - { - DeliveryCacheDependencies.TaxonomiesListScope - }; - - foreach (var taxonomy in taxonomies) - { - var dependency = CacheDependencyKeyBuilder.BuildTaxonomyDependencyKey(taxonomy.System.Codename); - if (dependency is null) - continue; - - dependencies.Add(dependency); - } - - return [.. dependencies]; - } + private static string[] BuildDependencies(IReadOnlyList taxonomies) => + [ + DeliveryCacheDependencies.TaxonomiesListScope, + .. taxonomies + .Where(taxonomy => !string.IsNullOrWhiteSpace(taxonomy.System.Codename)) + .Select(taxonomy => DeliveryCacheDependencies.ForTaxonomy(taxonomy.System.Codename)), + ]; private static IDeliveryResult WrapSuccess( DeliveryTaxonomyListingResponse response, diff --git a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TaxonomyQuery.cs b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TaxonomyQuery.cs index 833e594a8..0c623f26e 100644 --- a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TaxonomyQuery.cs +++ b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TaxonomyQuery.cs @@ -49,8 +49,6 @@ private async Task> ExecuteWithCacheAsync( var cacheKey = CacheKeyBuilder.BuildTaxonomyKey(codename); var outcome = await CachedQueryExecutor.ExecuteAsync( - cacheManager, - cacheKey, (captureApiResult, ct) => cacheManager.GetOrSetAsync( cacheKey, async factoryToken => @@ -58,7 +56,10 @@ private async Task> ExecuteWithCacheAsync( var result = await FetchFromApiAsync(waitForLoadingNewContent, factoryToken).ConfigureAwait(false); captureApiResult(result); if (!result.IsSuccess) + { + CachedQueryExecutor.ThrowIfOriginUnavailable(result); return null; + } return new CacheEntry((TaxonomyGroup)result.Value, BuildDependencies(result.Value)); }, @@ -117,10 +118,9 @@ private async Task> FetchFromApiAsync( } private static string[] BuildDependencies(ITaxonomyGroup taxonomyGroup) - { - var dependency = CacheDependencyKeyBuilder.BuildTaxonomyDependencyKey(taxonomyGroup.System.Codename); - return dependency is null ? [] : [dependency]; - } + => string.IsNullOrWhiteSpace(taxonomyGroup.System.Codename) + ? [] + : [DeliveryCacheDependencies.ForTaxonomy(taxonomyGroup.System.Codename)]; private static IDeliveryResult WrapSuccess( ITaxonomyGroup taxonomyGroup, diff --git a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TypeQuery.cs b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TypeQuery.cs index 1df667007..4ed3a4a70 100644 --- a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TypeQuery.cs +++ b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TypeQuery.cs @@ -56,8 +56,6 @@ private async Task> ExecuteWithCacheAsync( var cacheKey = CacheKeyBuilder.BuildTypeKey(codename, _params); var outcome = await CachedQueryExecutor.ExecuteAsync( - cacheManager, - cacheKey, (captureApiResult, ct) => cacheManager.GetOrSetAsync( cacheKey, async factoryToken => @@ -65,7 +63,10 @@ private async Task> ExecuteWithCacheAsync( var result = await FetchFromApiAsync(waitForLoadingNewContent, factoryToken).ConfigureAwait(false); captureApiResult(result); if (!result.IsSuccess) + { + CachedQueryExecutor.ThrowIfOriginUnavailable(result); return null; + } return new CacheEntry((ContentType)result.Value, BuildDependencies(result.Value)); }, @@ -124,10 +125,9 @@ private async Task> FetchFromApiAsync( } private static string[] BuildDependencies(IContentType contentType) - { - var dependency = CacheDependencyKeyBuilder.BuildTypeDependencyKey(contentType.System.Codename); - return dependency is null ? [] : [dependency]; - } + => string.IsNullOrWhiteSpace(contentType.System.Codename) + ? [] + : [DeliveryCacheDependencies.ForType(contentType.System.Codename)]; private static IDeliveryResult WrapSuccess( IContentType contentType, diff --git a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TypesQuery.cs b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TypesQuery.cs index af96bb745..3d3ea3a15 100644 --- a/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TypesQuery.cs +++ b/src/delivery/Kontent.Ai.Delivery/Api/QueryBuilders/TypesQuery.cs @@ -76,8 +76,6 @@ private async Task> ExecuteWithCac var cacheKey = CacheKeyBuilder.BuildTypesKey(_params, _serializedFilters); var outcome = await CachedQueryExecutor.ExecuteAsync( - cacheManager, - cacheKey, (captureApiResult, ct) => cacheManager.GetOrSetAsync( cacheKey, async factoryToken => @@ -85,7 +83,10 @@ private async Task> ExecuteWithCac var result = await FetchFromApiAsync(waitForLoadingNewContent, factoryToken).ConfigureAwait(false); captureApiResult(result); if (!result.IsSuccess) + { + CachedQueryExecutor.ThrowIfOriginUnavailable(result); return null; + } return new CacheEntry(result.Value, BuildDependencies(result.Value.Types)); }, @@ -155,24 +156,13 @@ private async Task> FetchFromApiAsy return await response.ToDeliveryResultAsync(logger).ConfigureAwait(false); } - private static string[] BuildDependencies(IReadOnlyList types) - { - var dependencies = new HashSet(StringComparer.OrdinalIgnoreCase) - { - DeliveryCacheDependencies.TypesListScope - }; - - foreach (var type in types) - { - var dependency = CacheDependencyKeyBuilder.BuildTypeDependencyKey(type.System.Codename); - if (dependency is null) - continue; - - dependencies.Add(dependency); - } - - return [.. dependencies]; - } + private static string[] BuildDependencies(IReadOnlyList types) => + [ + DeliveryCacheDependencies.TypesListScope, + .. types + .Where(type => !string.IsNullOrWhiteSpace(type.System.Codename)) + .Select(type => DeliveryCacheDependencies.ForType(type.System.Codename)), + ]; private static IDeliveryResult WrapSuccess( DeliveryTypeListingResponse response, diff --git a/src/delivery/Kontent.Ai.Delivery/Caching/IFailSafeStateProvider.cs b/src/delivery/Kontent.Ai.Delivery/Caching/IFailSafeStateProvider.cs deleted file mode 100644 index c4e0eeafd..000000000 --- a/src/delivery/Kontent.Ai.Delivery/Caching/IFailSafeStateProvider.cs +++ /dev/null @@ -1,14 +0,0 @@ -namespace Kontent.Ai.Delivery.Caching; - -/// -/// Internal capability interface for cache managers that can report whether -/// a given SDK cache key is currently served via fail-safe stale data. -/// -internal interface IFailSafeStateProvider -{ - /// - /// Returns true when fail-safe is active for the specified SDK cache key. - /// - /// SDK cache key (unformatted, before prefixes). - bool IsFailSafeActive(string cacheKey); -} diff --git a/src/delivery/Kontent.Ai.Delivery/DeliveryClient.cs b/src/delivery/Kontent.Ai.Delivery/DeliveryClient.cs index ca9eb4062..471d6a95c 100644 --- a/src/delivery/Kontent.Ai.Delivery/DeliveryClient.cs +++ b/src/delivery/Kontent.Ai.Delivery/DeliveryClient.cs @@ -85,6 +85,14 @@ public static DeliveryClient Create(DeliveryOptions options, Action services.AddDeliveryClient(options, configure)); } + /// + /// The cache this client was registered with, or null when it caches nothing. A client resolved + /// from a container has its manager registered there too, keyed by the client's name and unkeyed for + /// the default client; a client from owns its + /// container, so this is how its cache is invalidated or purged. + /// + public IDeliveryCacheManager? CacheManager => _cacheManager; + private static DeliveryClient CreateOwned(Action register) { var services = new ServiceCollection(); diff --git a/src/delivery/README.md b/src/delivery/README.md index ccacf1e06..5c9062dc4 100644 --- a/src/delivery/README.md +++ b/src/delivery/README.md @@ -1444,14 +1444,16 @@ Typed listing queries include synthetic scope dependencies: - `GetTypes()` → `DeliveryCacheDependencies.TypesListScope` - `GetTaxonomies()` → `DeliveryCacheDependencies.TaxonomiesListScope` -When processing webhooks, invalidate both entity-specific keys and the relevant list scope key: +When processing webhooks, invalidate both entity-specific keys and the relevant list scope key. `DeliveryCacheDependencies` composes the entity keys exactly as the SDK tags them, and the manager resolves unkeyed for the default client, keyed by name for a named one, and as `CacheManager` on a client from `DeliveryClient.Create`: ```csharp using Kontent.Ai.Delivery.Abstractions; +var cacheManager = serviceProvider.GetRequiredService(); + // Item events var itemDependencyKeys = webhookPayload.Data.Items - .Select(i => $"item_{i.Codename}") + .Select(i => DeliveryCacheDependencies.ForItem(i.Codename)) .Append(DeliveryCacheDependencies.ItemsListScope) .ToArray(); @@ -1459,13 +1461,18 @@ await cacheManager.InvalidateAsync(itemDependencyKeys); // Type events await cacheManager.InvalidateAsync( - [$"type_{typeCodename}", DeliveryCacheDependencies.TypesListScope]); + [DeliveryCacheDependencies.ForType(typeCodename), DeliveryCacheDependencies.TypesListScope]); // Taxonomy events await cacheManager.InvalidateAsync( - [$"taxonomy_{taxonomyCodename}", DeliveryCacheDependencies.TaxonomiesListScope]); + [DeliveryCacheDependencies.ForTaxonomy(taxonomyCodename), DeliveryCacheDependencies.TaxonomiesListScope]); + +// Asset events +await cacheManager.InvalidateAsync([DeliveryCacheDependencies.ForAsset(assetId)]); ``` +With fail-safe on, an invalidated entry may still be served stale while the origin is unreachable; an answer from the origin - a `404` for an unpublished item, say - drops it. + #### Purging the SDK Cache Built-in cache managers support invalidating **all** cached entries at once via the optional `IDeliveryCachePurger` capability: @@ -1474,7 +1481,7 @@ Built-in cache managers support invalidating **all** cached entries at once via using Kontent.Ai.Delivery.Abstractions; using Microsoft.Extensions.DependencyInjection; -// Resolve cache manager for the client name used during registration. +// A named client's manager is keyed by its name; the default client's resolves unkeyed as well. var cacheManager = serviceProvider.GetRequiredKeyedService("production"); if (cacheManager is IDeliveryCachePurger purger) { diff --git a/src/delivery/docs/caching-guide.md b/src/delivery/docs/caching-guide.md index 3f719fa83..9290fd32a 100644 --- a/src/delivery/docs/caching-guide.md +++ b/src/delivery/docs/caching-guide.md @@ -77,6 +77,8 @@ Kontent.ai enforces rate limits on API requests: - Development and testing - Low to moderate traffic applications +**What a hit hands back:** the memory cache stores the hydrated objects themselves, so every hit returns the same instances the previous caller received, with the client's `DefaultRenditionPreset` and `CustomAssetDomain` already applied. Treat cached models as read-only - a mutation is visible to every later caller - and purge after changing either option at runtime, since cached entries keep the values they were built with. + ### Hybrid Cache **Pros:** @@ -102,7 +104,11 @@ services.AddDeliveryClient(delivery => Without one, part of the invalidation state stays local to each instance, so whether a node observes another's `InvalidateAsync` depends on the order the two read and invalidated in — a node can go on serving content that was already evicted, until the entry expires by itself. A single-instance application needs no backplane. > [!NOTE] -> **FusionCache hybrid mode limitation:** When using hybrid caching, FusionCache operates in hybrid (L1+L2) mode but [currently stores the same serialized format in both layers](https://github.com/ZiggyCreatures/FusionCache/issues/321). This means the L1 memory layer also holds raw JSON rather than hydrated objects, so every cache hit goes through rehydration. For most workloads the rehydration cost is negligible. If your scenario demands maximum read throughput, use `UseMemoryCache` (pure L1, hydrated objects, no rehydration overhead). +> **Every hybrid hit rehydrates.** The SDK stores the raw payload in both tiers, so that a value can move between them unchanged; a hit therefore parses the JSON and maps the elements again, rich text HTML parse included. That is measurable on a hot path serving large rich-text items, and it is also what makes every hit a fresh instance. If maximum read throughput matters more than sharing the cache across instances, use `UseMemoryCache`. + +Two costs worth knowing about the distributed tier. Every dependency key is a tag, and FusionCache verifies an entry's tags on each read against tag data it keeps in memory - a node that has never seen a tag reads it from the distributed cache once, so the first hit on a fresh node costs one round trip per tag (a listing of a hundred items with their types, assets and taxonomies carries several hundred). And an invalidation is remembered for as long as that tag data lives: ten days by default, adjustable through `ConfigureFusionCache(f => f.TagsDefaultEntryOptions.Duration = …)`, which must stay above your longest expiration, per-query overrides included. + +If the distributed cache is unreachable, the SDK works around it: the memory tier or the origin answers, a two-second circuit breaker keeps a dead Redis from being retried on every request, and FusionCache re-syncs the tier when it is back. Nothing is thrown out of a query for it; enable the `ZiggyCreatures.Caching.Fusion.FusionCache` log category to see it happen. **Cons:** - Network latency (still faster than API calls) @@ -165,10 +171,12 @@ services.AddDeliveryClient("production", delivery => #### Advanced Memory Cache Configuration +The SDK uses the application's `IMemoryCache`, so its options apply. Under a `SizeLimit` every entry the SDK writes counts as one unit, so the limit bounds the number of cached responses rather than their bytes: + ```csharp services.AddMemoryCache(options => { - options.SizeLimit = 1024; // Limit cache size + options.SizeLimit = 1024; // At most 1024 cached responses options.CompactionPercentage = 0.25; // Remove 25% when limit hit }); @@ -307,13 +315,25 @@ services.AddDeliveryClient("production", delivery => Timing matters: the plain `Action` overloads run immediately during service registration and validate cache options immediately. The `(IServiceProvider, DeliveryCacheOptions)` overloads run later, when the keyed singleton `IDeliveryCacheManager` is first resolved from the root provider, so validation is deferred to that first resolution. +#### Reaching into FusionCache + +`ConfigureFusionCache` hands you the `FusionCacheOptions` the SDK built, after its defaults are applied. Every SDK operation starts from the `DefaultEntryOptions` you leave there, so a `Size`, a distributed-cache timeout or the background-operation flags take effect; `TagsDefaultEntryOptions` is what an invalidation is stored with. What the SDK decides stays decided: the duration, fail-safe, jitter and eager-refresh policy come from `DeliveryCacheOptions`, serialization failures are thrown, distributed-cache and backplane failures are not. + +```csharp +delivery.UseHybridCache(cache => cache.ConfigureFusionCache(fusion => +{ + fusion.DefaultEntryOptions.AllowBackgroundBackplaneOperations = true; + fusion.TagsDefaultEntryOptions.Duration = TimeSpan.FromDays(30); +})); +``` + Because the cache manager is a singleton, resolve only singleton-safe dependencies from cache callbacks, such as `IOptions`, `IOptionsMonitor`, configuration, or loggers. Do not depend on scoped/request services such as `IOptionsSnapshot`, `DbContext`, tenant request context, or per-request `HttpContext` state. ### Custom Cache Manager -For advanced scenarios, implement a custom cache manager. The `IDeliveryCacheManager` interface uses a factory-based `GetOrSetAsync` pattern — the factory is invoked on cache miss and returns a `CacheEntry?` (null signals "don't cache"). The method returns `CacheResult?` (a record containing the `Value` and the collected `DependencyKeys`) so that downstream consumers can access dependency metadata. +For advanced scenarios, implement a custom cache manager. The `IDeliveryCacheManager` interface uses a factory-based `GetOrSetAsync` pattern — the factory is invoked on cache miss and returns a `CacheEntry?`. A `null` means the origin has no value for the key: don't cache, and drop any stale copy you keep for fail-safe. A thrown exception means the origin could not be reached: serve a stale copy if you keep one, otherwise let it propagate. The method returns `CacheResult?` (a record containing the `Value` and the collected `DependencyKeys`) so that downstream consumers can access dependency metadata. -Use the default `StorageMode` (`CacheStorageMode.HydratedObject`) for hydrated-object caching (memory), or override `StorageMode` to `CacheStorageMode.RawJson` for raw JSON payload caching (distributed). +Use the default `StorageMode` (`CacheStorageMode.HydratedObject`) for hydrated-object caching (memory), or override `StorageMode` to `CacheStorageMode.RawJson` for raw JSON payload caching (distributed). A manager that serves stale copies while the origin is unreachable sets `IsStale` on the results it serves that way; the SDK reports them as `ResponseSource.FailSafe`. #### Hydrated-object cache manager (memory style) @@ -417,6 +437,8 @@ When `WaitForLoadingNewContent(true)` is enabled for a query, the SDK bypasses l When a client is configured with `UsePreviewApi = true`, the SDK always bypasses local cache reads/writes for that client, even if a cache manager is registered. +A typed query whose model is `IDynamicElements` or `DynamicElements` is cached, but its elements are not mapped, so only item, type and list-scope dependencies are tracked for it - not the assets, taxonomy groups and rich-text links a mapped model would add. + ### Cache Keys Cache keys are automatically generated from query parameters using a deterministic, human-readable format. @@ -490,6 +512,8 @@ When queries include filters, they are hashed using SHA256 (first 12 characters #### Key Prefixing +Every key a client stores lives under `{KeyPrefix}:{EnvironmentId}:`, where `KeyPrefix` defaults to the client's name for a named client and to nothing for the default one. The environment id is always there, because a store can outlive the process and be shared: two applications on different environments sharing one Redis would otherwise compute the same key for "the item `homepage`". Hybrid keys carry a format version after that, `v1:`, so an SDK release that changes what it stores misses on old entries instead of failing to read them. + **Default (single-client) scenario:** ```csharp services.AddDeliveryClient(delivery => @@ -497,7 +521,7 @@ services.AddDeliveryClient(delivery => delivery.Options.Configure(o => o.EnvironmentId = "..."); delivery.UseMemoryCache(); }); -// Keys have NO prefix: item:homepage, items:skip=0:limit=10, etc. +// Keys: {environmentId}:item:homepage, {environmentId}:items:skip=0:limit=10, etc. ``` **Named clients (multi-client scenario):** @@ -507,7 +531,7 @@ services.AddDeliveryClient("production", delivery => delivery.Options.Configure(o => o.EnvironmentId = "..."); delivery.UseMemoryCache(); }); -// Keys are prefixed with client name: production:item:homepage, etc. +// Keys: production:{environmentId}:item:homepage, etc. ``` **Custom prefix:** @@ -517,22 +541,15 @@ services.AddDeliveryClient("production", delivery => delivery.Options.Configure(o => o.EnvironmentId = "..."); delivery.UseMemoryCache(o => o.KeyPrefix = "prod"); }); -// Keys become: prod:item:homepage, prod:items:skip=0:limit=10, etc. +// Keys: prod:{environmentId}:item:homepage, etc. +// In a hybrid cache: prod:{environmentId}:v1:item:homepage ``` -**No prefix (explicit):** -```csharp -services.AddDeliveryClient("production", delivery => -{ - delivery.Options.Configure(o => o.EnvironmentId = "..."); - delivery.UseMemoryCache(o => o.KeyPrefix = ""); -}); -// Keys have no prefix even for named clients -``` +An explicit `KeyPrefix = ""` on a named client puts it in the same namespace as the default client on that environment - the two then share entries, invalidations and purges, which is what you want only if they are the same client registered twice. -This prevents cache collisions when multiple clients share the same underlying cache. +The prefix is handed to FusionCache, so it covers FusionCache's own bookkeeping too: the tag data an invalidation writes and the marker a purge writes. Two clients sharing one memory cache or one Redis cannot reach each other's entries, and purging one leaves the other's alone. -`EnvironmentId` and `DefaultRenditionPreset` are not part of query cache keys. Use separate named clients (or distinct key prefixes) per environment/configuration. If you change either option at runtime on an existing cached client, purge cache (or recreate the client) to avoid serving older entries. +`DefaultRenditionPreset` and `CustomAssetDomain` are not part of the keys; use separate named clients per configuration. The environment id is read when the cache is created, so changing it at runtime on an existing cached client keeps caching under the old namespace - purge, or recreate the client. ### Dependency Tracking @@ -633,47 +650,63 @@ Supported cacheable query builders: - `GetTaxonomy()` - `GetTaxonomies()` +#### Fail-safe + +`IsFailSafeEnabled` lets the cache serve a stale copy when the origin cannot be reached: a request that got no response, or a status the SDK's own pipeline retries (`408`, `429`, `5xx`). Such a result carries `ResponseSource.FailSafe`. An answer from the API is never covered - an item that comes back `404` after being unpublished is dropped from the cache and the failure is returned - so unpublishing takes effect with fail-safe on, and the only content served stale is content the origin could not be asked about. + +An invalidation and fail-safe compose the same way. `InvalidateAsync` expires the entries rather than deleting them when fail-safe is on, so a webhook followed by an outage serves the pre-webhook copy until the origin is back; a webhook followed by an answer drops it. `PurgeAsync(allowFailSafe: true)` keeps the same distinction. + ## Cache Invalidation ### Invalidation Matrix (RC-ready) -Use this matrix when mapping webhook events to SDK dependency invalidation keys: +Use this matrix when mapping webhook events to SDK dependency invalidation keys. Compose the detail keys with `DeliveryCacheDependencies` rather than by hand: they are the exact strings the SDK tags with, trimmed and lower-cased, and `InvalidateAsync` matches case-insensitively. | Endpoint family | Detail dependency key | Listing scope dependency key | |---|---|---| -| Items | `item_{codename}` | `DeliveryCacheDependencies.ItemsListScope` (`scope_items_list`) | -| Types | `type_{codename}` (also tags item/item-list caches containing items of that type) | `DeliveryCacheDependencies.TypesListScope` (`scope_types_list`) | -| Taxonomies | `taxonomy_{codename}` | `DeliveryCacheDependencies.TaxonomiesListScope` (`scope_taxonomies_list`) | +| Items | `DeliveryCacheDependencies.ForItem(codename)` (`item_{codename}`) | `DeliveryCacheDependencies.ItemsListScope` (`scope_items_list`) | +| Types | `DeliveryCacheDependencies.ForType(codename)` (`type_{codename}`; also tags item/item-list caches containing items of that type) | `DeliveryCacheDependencies.TypesListScope` (`scope_types_list`) | +| Taxonomies | `DeliveryCacheDependencies.ForTaxonomy(codename)` (`taxonomy_{codename}`) | `DeliveryCacheDependencies.TaxonomiesListScope` (`scope_taxonomies_list`) | +| Assets | `DeliveryCacheDependencies.ForAsset(id)` (`asset_{id}`; tags every item cache whose asset elements or rich-text images reference it) | none - assets have no listing | Recommended webhook pattern: -- item event: invalidate `item_{codename}` + `scope_items_list` -- type event: invalidate `type_{codename}` + `scope_types_list` — the `type_{codename}` key now covers both the cached type definition and every item/item-list cache whose payload references items of that type, so content-type changes or deletions no longer require falling back to `scope_items_list` -- taxonomy event: invalidate `taxonomy_{codename}` + `scope_taxonomies_list` +- item event: invalidate `ForItem(codename)` + `ItemsListScope` +- type event: invalidate `ForType(codename)` + `TypesListScope` — the type key covers both the cached type definition and every item/item-list cache whose payload references items of that type, so content-type changes or deletions do not require falling back to `ItemsListScope` +- taxonomy event: invalidate `ForTaxonomy(codename)` + `TaxonomiesListScope` +- asset event: invalidate `ForAsset(id)` ### Manual Invalidation -Invalidate specific content: +Resolve the manager first. The default client's resolves unkeyed, a named client's under its name, and a client from `DeliveryClient.Create` owns its container, so its manager is on the client: ```csharp using Kontent.Ai.Delivery.Abstractions; using Microsoft.Extensions.DependencyInjection; -var cacheManager = serviceProvider.GetRequiredKeyedService("production"); +// The default client +var cacheManager = serviceProvider.GetRequiredService(); + +// A named client +var productionCacheManager = serviceProvider.GetRequiredKeyedService("production"); + +// A client from DeliveryClient.Create owns its container, so the manager is on the client +var standaloneCacheManager = standaloneClient.CacheManager; +``` + +Then invalidate specific content (shown for the default client's manager): +```csharp // Invalidate a specific item -await cacheManager.InvalidateAsync(["item_homepage"]); +await cacheManager.InvalidateAsync([DeliveryCacheDependencies.ForItem("homepage")]); -// Invalidate multiple items +// Invalidate multiple entities at once await cacheManager.InvalidateAsync([ - "item_article1", - "item_article2", - "taxonomy_categories"]); - -// Invalidate by dependency -await cacheManager.InvalidateAsync([$"item_{articleCodename}"]); + DeliveryCacheDependencies.ForItem("article1"), + DeliveryCacheDependencies.ForItem("article2"), + DeliveryCacheDependencies.ForTaxonomy("categories")]); // Invalidate a specific type query dependency -await cacheManager.InvalidateAsync(["type_article"]); +await cacheManager.InvalidateAsync([DeliveryCacheDependencies.ForType("article")]); // Invalidate all cached typed item-list queries await cacheManager.InvalidateAsync([DeliveryCacheDependencies.ItemsListScope]); @@ -756,6 +789,7 @@ public class WebhookController : ControllerBase private async Task ProcessWebhookAsync(WebhookNotification notification) { + // The default client's manager resolves unkeyed; a named client's under its name. var cacheManager = _serviceProvider.GetRequiredKeyedService("production"); var dependencies = new List(); @@ -764,23 +798,29 @@ public class WebhookController : ControllerBase // Content item changes affect item queries and item listings. if (item.Type == "content_item") { - dependencies.Add($"item_{item.Codename}"); + dependencies.Add(DeliveryCacheDependencies.ForItem(item.Codename)); dependencies.Add(DeliveryCacheDependencies.ItemsListScope); } // Taxonomy changes affect taxonomy queries and taxonomy listings. if (item.Type == "taxonomy") { - dependencies.Add($"taxonomy_{item.Codename}"); + dependencies.Add(DeliveryCacheDependencies.ForTaxonomy(item.Codename)); dependencies.Add(DeliveryCacheDependencies.TaxonomiesListScope); } // Content type changes affect type queries and type listings. if (item.Type == "content_type") { - dependencies.Add($"type_{item.Codename}"); + dependencies.Add(DeliveryCacheDependencies.ForType(item.Codename)); dependencies.Add(DeliveryCacheDependencies.TypesListScope); } + + // Asset changes affect every item that references the asset. + if (item.Type == "asset") + { + dependencies.Add(DeliveryCacheDependencies.ForAsset(Guid.Parse(item.Id))); + } } // Invalidate all affected cache entries @@ -1122,9 +1162,17 @@ public class MonitoredCacheManager : IDeliveryCacheManager var result = await _inner.GetOrSetAsync(cacheKey, factory, expiration, cancellationToken); stopwatch.Stop(); - _metrics.RecordCacheAccess(result != null, stopwatch.ElapsedMilliseconds); - _logger.LogDebug("Cache {Result} for key: {Key} in {Ms}ms", - result != null ? "HIT/SET" : "MISS", cacheKey, stopwatch.ElapsedMilliseconds); + // FromFactory and IsStale are the only reliable classification: under eager refresh the factory + // also runs for a background refresh, so a flag set inside it belongs to a different call. + var outcome = result switch + { + null => "MISS", + { FromFactory: true } => "FETCHED", + { IsStale: true } => "STALE", + _ => "HIT", + }; + _metrics.RecordCacheAccess(result is { FromFactory: false }, stopwatch.ElapsedMilliseconds); + _logger.LogDebug("Cache {Outcome} for key: {Key} in {Ms}ms", outcome, cacheKey, stopwatch.ElapsedMilliseconds); return result; } @@ -1133,28 +1181,11 @@ public class MonitoredCacheManager : IDeliveryCacheManager } ``` -### 4. Handle Cache Failures Gracefully +### 4. Know What Happens When the Cache Fails -```csharp -public async Task?> GetOrSetAsync( - string cacheKey, - Func?>> factory, - TimeSpan? expiration = null, - CancellationToken cancellationToken = default) where T : class -{ - try - { - return await _inner.GetOrSetAsync(cacheKey, factory, expiration, cancellationToken); - } - catch (RedisConnectionException ex) - { - _logger.LogWarning(ex, "Redis connection failed, bypassing cache"); - // Fall back to calling the factory directly (no caching) - var entry = await factory(cancellationToken); - return entry is null ? null : new CacheResult(entry.Value, entry.Dependencies.ToArray()); - } -} -``` +The built-in managers degrade rather than fail. A distributed cache that cannot be reached is worked around - the memory tier or the origin answers, and a two-second circuit breaker keeps a dead Redis from being retried on every request - and a serialization failure, which is a defect in the SDK's own payloads, is the one cache error that is thrown. Enable the `ZiggyCreatures.Caching.Fusion.FusionCache` log category to see outages, backplane failures and background-refresh errors as they are worked around; the SDK's own invalidation messages log under `Kontent.Ai.Delivery.Caching.FusionCacheManager`. + +A custom manager owns that decision itself. If it wraps an `IDistributedCache`, catch the provider's exception, log it, and fall back to calling the factory so the query still answers. ### 5. Pre-Warm Cache @@ -1290,16 +1321,13 @@ public class LoggingCacheManager : IDeliveryCacheManager TimeSpan? expiration = null, CancellationToken cancellationToken = default) where T : class { - // Wrap the factory to detect cache misses - var wasMiss = false; - var result = await _inner.GetOrSetAsync(cacheKey, async ct => - { - wasMiss = true; - return await factory(ct); - }, expiration, cancellationToken); + var result = await _inner.GetOrSetAsync(cacheKey, factory, expiration, cancellationToken); + // Read the classification off the result. A flag set inside the factory would be wrong under + // eager refresh, where the factory runs in the background for a call that already returned. _logger.LogInformation("Cache {Result} for key: {Key}", - wasMiss ? "MISS+SET" : "HIT", cacheKey); + result switch { null => "MISS", { FromFactory: true } => "MISS+SET", { IsStale: true } => "STALE", _ => "HIT" }, + cacheKey); return result; } @@ -1341,7 +1369,7 @@ if (cacheManager == null) ### Runtime Option Changes with Existing Cache -**Problem**: You changed `EnvironmentId` or `DefaultRenditionPreset` at runtime, but cached responses still reflect the previous setting. +**Problem**: You changed `EnvironmentId`, `DefaultRenditionPreset` or `CustomAssetDomain` at runtime, but cached responses still reflect the previous setting. The environment id is read when the cache is created and is part of every key; the other two are baked into the hydrated objects a memory cache holds. **Solutions**: @@ -1356,11 +1384,11 @@ if (cacheManager == null) **Solutions**: 1. **Use hybrid cache** instead of memory cache -2. **Configure cache size limits**: +2. **Configure a size limit** - every entry the SDK writes counts as one unit, so this bounds the number of cached responses: ```csharp services.AddMemoryCache(options => { - options.SizeLimit = 1024; // Limit number of entries + options.SizeLimit = 1024; }); ``` 3. **Reduce expiration times** @@ -1368,23 +1396,14 @@ services.AddMemoryCache(options => ### Redis Connection Failures -**Problem**: Application crashes when Redis is unavailable. +**Problem**: Redis is unavailable. -**Solutions**: +**What happens**: nothing is thrown out of a query. The distributed tier is worked around - the memory tier or the origin answers - and a two-second circuit breaker keeps the dead connection from being retried on every request; FusionCache re-syncs the tier when it is back. Reads cost an origin call more often while it lasts, and invalidations reach other nodes again only once it is over. -1. **Graceful degradation**: -```csharp -try -{ - return await _cache.GetAsync(key); -} -catch (RedisConnectionException) -{ - return default; // Fall back to API -} -``` +**Solutions**: -2. **Configure connection resilience**: +1. **See it**: enable the `ZiggyCreatures.Caching.Fusion.FusionCache` log category, which reports every worked-around failure at warning level. +2. **Configure connection resilience** so the client reconnects on its own: ```csharp var config = ConfigurationOptions.Parse("localhost:6379"); config.AbortOnConnectFail = false; diff --git a/src/delivery/docs/for-developers.md b/src/delivery/docs/for-developers.md index 306b5345d..ddbde29cb 100644 --- a/src/delivery/docs/for-developers.md +++ b/src/delivery/docs/for-developers.md @@ -385,10 +385,12 @@ var provider = services.BuildServiceProvider(); **Package**: `Kontent.Ai.Delivery.Caching` -Both built-in cache managers are thin wrappers over a shared `FusionCacheManager` engine: +One class, `FusionCacheManager`, in two shapes: -- **`MemoryCacheManager`** — creates a FusionCache instance with L1 only (hydrated objects, `CacheStorageMode.HydratedObject`) -- **`HybridCacheManager`** — creates a FusionCache instance with L1+L2 (raw JSON via `FusionCacheSystemTextJsonSerializer`, `CacheStorageMode.RawJson`) +- **`FusionCacheManager.CreateMemory`** — a FusionCache instance over the application's `IMemoryCache`, L1 only (hydrated objects, `CacheStorageMode.HydratedObject`) +- **`FusionCacheManager.CreateHybrid`** — a FusionCache instance with a private L1 in front of the `IDistributedCache` (raw JSON via `FusionCacheSystemTextJsonSerializer`, `CacheStorageMode.RawJson`) + +Both hand the client's prefix (`{KeyPrefix}:{EnvironmentId}:`, plus `v1:` for hybrid) to FusionCache as its `CacheKeyPrefix`, so FusionCache's tag data and purge markers are namespaced with the entries. Invalidations and purges run with `TagsDefaultEntryOptions`, whose ten-day duration is how long an invalidation is remembered for an entry that has not been read since. A factory that returns `null` means the origin has no value - the stale copy is removed and fail-safe is bypassed for that call - and a factory that throws means the origin could not be reached, which is what fail-safe is for. Staleness is observed per call, on the async context of the read, and returned as `CacheResult.IsStale`. **Invalidation Model:** @@ -824,7 +826,7 @@ The other two overloads are thin: `AddDeliveryClient(Action ...)` registers the keyed manager for that client. Preview cache bypass is enforced by `DeliveryClient` itself (`UsePreviewApi = true` => no cache read/write for that client), not by a cache-manager decorator. diff --git a/src/delivery/docs/multi-client-scenarios.md b/src/delivery/docs/multi-client-scenarios.md index 8cd273574..1e99d1648 100644 --- a/src/delivery/docs/multi-client-scenarios.md +++ b/src/delivery/docs/multi-client-scenarios.md @@ -891,7 +891,7 @@ services.AddDeliveryClient("brand-a", delivery => delivery.Options.Configure(opt **Problem**: Cached content from one client appears for another. -**Solution**: Ensure each client uses its own cache namespace. The SDK does this with per-client key prefixes for named clients (or custom `keyPrefix` values when configured). If you change `EnvironmentId` on an already-cached client at runtime, purge cache or recreate the client. +**Solution**: Ensure each client uses its own cache namespace. The SDK does this for you: every key lives under the client's `DeliveryCacheOptions.KeyPrefix` (the client's name unless you set one) and its environment id, and the prefix covers FusionCache's own bookkeeping too, so two clients sharing a store cannot reach each other's entries, invalidations or purges. If you change `EnvironmentId` on an already-cached client at runtime, purge cache or recreate the client. ---