Document advisory storage lease limits

Author: eldryoth

CHANGELOG.md

@@ -66,6 +66,8 @@ behavior when the change improves security or project direction.
 - Acquire storage-bin leases before layout initialization, partition blocking
   work by subsystem with critical capacity reserved, and fail closed when a
   disk lookup cannot obtain blocking admission.
+- Qualify storage-bin leases as advisory and require per-replica storage or
+  verified cross-node locking plus orchestration-level single-writer controls.
 - Pin publishing actions, security-tool installs, and container base images to
   immutable reviewed versions and digests.
 

README.md

@@ -73,7 +73,7 @@ Fluxheim is licensed under the European Union Public Licence 1.2.
 | --- | --- | --- |
 | Proxy cache | ✅ | Vhost and route-scoped cache policies. |
 | Memory cache | ✅ | Bounded in-memory cache tier. |
-| Disk cache | ✅ | Filesystem and storage-bin disk backends with ordered eviction, one process-wide persistence worker, pre-initialization cross-process lease enforcement, live inspection, fail-closed lookup admission, and coalesced durable index writes. |
+| Disk cache | ✅ | Filesystem and storage-bin disk backends with ordered eviction, one process-wide persistence worker, pre-initialization advisory lease enforcement, live inspection, fail-closed lookup admission, and coalesced durable index writes. |
 | Tiered cache | ✅ | Memory plus disk storage plans. |
 | Encrypted disk cache | ✅ | Optional local-key and OpenBao transit encryption paths. |
 | Static-file cache | ✅ | Optional local static-file caching. |

docs/cache-backends.md

@@ -235,11 +235,23 @@ internal cache implementation.
 - Each allocator acquires an exclusive `.fluxheim-storage-bin.lock` filesystem
   lease immediately after creating and canonicalizing the empty root, before
   creating or validating the manifest, data directory, index, or bins. The
-  lease remains held for the allocator's complete lifetime. Another Fluxheim
-  process, including a second replica pointed at the same persistent volume,
-  fails to open or initialize the root while that lease is held. Admin and CLI
-  inspection resolve the registered live cache by vhost/route and never
-  construct a temporary allocator or write an independent index.
+  lease remains held for the allocator's complete lifetime. On local filesystems
+  and shared filesystems with reliable cross-node `flock` semantics, another
+  cooperating Fluxheim process fails to open or initialize the root while that
+  lease is held. The lease is advisory: unrelated software can ignore it, and
+  some NFS/CSI implementations do not provide reliable cross-node advisory
+  locking. Admin and CLI inspection resolve the registered live cache by
+  vhost/route and never construct a temporary allocator or write an independent
+  index.
+- Production HA deployments should use one storage-bin root per replica on
+  local, `ReadWriteOnce`, or `ReadWriteOncePod` storage. Share cache warmth with
+  peer fill rather than mounting one `ReadWriteMany` root into multiple
+  replicas. Use shared RWX storage only after a deployment acceptance test has
+  started two Fluxheim replicas simultaneously, confirmed that the loser fails
+  before changing the manifest, stopped the winner, and confirmed that the
+  remaining replica can acquire the lease without changing persisted metadata.
+  High-assurance deployments must also enforce single-writer ownership at the
+  orchestration layer.
 - Native disk-cache lookup is admitted through a dedicated blocking-work class.
   Saturation is distinct from a cache miss: Fluxheim serves an eligible stale
   memory object when available and otherwise returns `503` without contacting

docs/config-reference.md

@@ -2193,6 +2193,15 @@ table defines the allocator shape:
 reserve full bin files ahead of object writes, and `max_open_bins` bounds the
 number of concurrently opened bin files.
 
+The storage-bin backend holds a root-local advisory lock before creating or
+validating persistent layout metadata. Cross-node exclusion therefore depends
+on correct `flock` behavior from the selected filesystem and CSI driver. Use a
+separate local, `ReadWriteOnce`, or `ReadWriteOncePod` cache volume per replica.
+Do not share one RWX storage-bin root between replicas unless cross-node lock
+behavior has been explicitly tested; high-assurance deployments should enforce
+single-writer ownership through orchestration as well. Peer fill is the
+supported mechanism for sharing cache warmth between independent roots.
+
 `[cache.disk.encryption]` is disabled by default. When `enabled = true` with
 `provider = "local"`, Fluxheim encrypts disk cache objects with AES-256-GCM
 before they are written to the filesystem or storage-bin backend. The local key

release-notes/RELEASE_NOTES_1.7.7.md

@@ -68,6 +68,10 @@ deterministic unsupported-call rejection.
   outside request admission for operational work.
 - Acquire storage-bin ownership before any manifest or data-layout mutation,
   preventing a losing process from modifying first-start metadata.
+- Document that storage-bin ownership uses advisory filesystem locking: use a
+  per-replica local/RWO volume by default, and require verified cross-node
+  `flock` behavior plus orchestration-level single-writer enforcement before
+  using shared RWX storage in high-assurance deployments.
 - Partition blocking work by class under `224` non-critical and `256` total
   ceilings, reserve `32` critical slots, and return `503` rather than contacting
   origin when disk-cache lookup admission is saturated and no stale memory