Add docs for fly-replay-cache (#2011)

* Add docs for fly-replay-cache

* fixup

* Add replay caching to the per-user dev environment blueprint

* Update

* Update

Author: PeterCxy

blueprints/per-user-dev-environments.html.md

@@ -16,7 +16,7 @@ This blueprint explains how to use Fly Machines to securely host ephemeral devel
 Your architecture should include:
 
 - **Router app(s)**
-    - A Fly.io app to handle requests to wildcard subdomains (`*.example.com`). Uses `fly-replay` headers to transparently redirect each request to the correct app and machine. If you have clusters of users (or robots) in different geographic regions, you can spin up a router app in multiple regions (you might also want to consider a globally distributed datastore like [Upstash for Redis](https://fly.io/docs/upstash/redis/#what-you-should-know)). 
+    - A Fly.io app to handle requests to wildcard subdomains (`*.example.com`). Uses `fly-replay` headers to transparently redirect each request to the correct app and machine. If you have clusters of users (or robots) in different geographic regions, you can spin up a router app in multiple region to increase reliability and reduce latency (you might also want to consider a globally distributed datastore like [Upstash for Redis](https://fly.io/docs/upstash/redis/#what-you-should-know)). 
 - **User apps (pre-created)**
     - Dedicated per-user (or per-robot) Fly apps, each containing isolated Fly Machines. App and Machine creation is not instantaneous, so we recommend provisioning a pool of these before you need them so you can quickly assign upon request.
 - **Fly Machines (with optional volumes)**
@@ -32,7 +32,8 @@ Your router app handles all incoming wildcard traffic. Its responsibility is sim
 
 - Extract subdomains (like `alice.example.com` → `alice-123`).
 - Look up the correct app (and optionally machine ID) for that user.
-- Issue a `fly-replay` header directing the Fly Proxy to [internally redirect the request](https://fly.io/docs/networking/dynamic-request-routing) (this should add no more than ~10 milliseconds of latency).
+- Issue a `fly-replay` header directing the Fly Proxy to [internally redirect the request](https://fly.io/docs/networking/dynamic-request-routing) (this should add no more than ~10 milliseconds of latency if the router app is deployed close to the user).
+- When appropriate, use [replay caching](https://fly.io/docs/networking/dynamic-request-routing/#replay-caching) to further reduce latency and load on the router app.
 - Make sure you've added [a wildcard domain](https://fly.io/docs/networking/custom-domain/#get-certified) (*.example.com) to your router app (read more about the [certificate management endpoint here](https://fly.io/docs/networking/custom-domain-api/)).
 
 ### User apps

networking/dynamic-request-routing.html.markerb

@@ -55,6 +55,26 @@ You can combine multiple fields:
 fly-replay: region=sjc;app=app-in-same-org
 ```
 
+## Replay Caching
+
+In cases where the replay target does not change often, you can use the `fly-replay-cache` mechanism to relieve the original app of some load. Here's how it works:
+- Issue a `fly-replay` header as usual
+- Set `fly-replay-cache: example.com/some/path/*`
+  - This needs to be a path-matching pattern ending with a wildcard `/*`. This is where the cached replay is applied.
+  - The domain part should not include ports; use `example.com`, not `example.com:80`.
+  - The pattern must also match the current path of the request.
+- Set `fly-replay-cache-ttl-secs: number_of_seconds`
+
+If the replay target does eventually change, the replay target may proactively invalidate the cache by
+- Issuing a `fly-replay` back to the origin
+- Setting `fly-replay-cache: invalidate`
+
+<div class="note icon">
+**Note**: Replay caching is an optimization, not a guarantee. Your app should _not_ depend on this mechanism to function.
+The app issuing `fly-replay` still serves as the ultimate source of truth, and we may decide to consult that app at any moment even if a replay cache has previously been set.
+To ensure reliable operation, the app issuing `fly-replay` should still have multiple instances deployed in multiple regions.
+</div>
+
 ## Implementation Details
 
 ### Requirements and Limitations
@@ -67,6 +87,12 @@ For large uploads that exceed the 1MB limit, consider:
 - Using direct-to-storage uploads where possible
 - Using the [fly-prefer-region](#the-fly-prefer-region-request-header) header instead
 
+For `fly-replay-cache`, the following limitations apply:
+- The `state` field cannot be set in the `fly-replay` intended to be cached
+- The TTL needs to be a minimum of 10 seconds
+- Only one step of lookup is performed in the cache; as such, if the target app issues another `fly-replay-cache`, the caching behavior in this case is undefined
+- The `fly-replay-src` header (described below) will _not_ be set for requests replayed through the cache
+
 ### The fly-replay-src Header
 
 When a request is replayed, Fly Proxy adds a `fly-replay-src` header containing metadata about the original request:
@@ -80,6 +106,8 @@ When a request is replayed, Fly Proxy adds a `fly-replay-src` header containing
 
 This header is useful for tracking request paths and implementing consistency patterns. See the [official Ruby client](https://github.com/superfly/fly-ruby/blob/main/lib/fly-ruby/regional_database.rb#L74-L76+external) for an example of using these fields to prevent read-your-write inconsistency.
 
+This header is not set when the request is replayed through a cached `fly-replay` entry (`fly-replay-cache`).
+
 ## Alternative Routing Headers
 
 For cases where `fly-replay` isn't suitable, Fly.io provides two alternative request headers: