Skip to content

Commit b0933ee

Browse files
committed
docs(adr): correct two claims the record makes that the code does not
The record said every gate is rechecked inside the destructive step itself. What is rechecked there is the proof's health generation, the answerability veto, the announced writes, and that every key held only by the old store is in the approved set. The network gates, rank and commitment delivery and possession, are rechecked immediately before that call and outside the guard. The window on those is the seconds it takes to take the guard rather than the hours a verification pass can run for, so the argument holds, but the two are not the same claim. It also said the per-volume lock is held until the space comes back. It is released when the old store is unlinked and its directory renamed aside; the deletion itself runs detached so the node can serve while it happens, and on a large store that takes minutes. The next node in the queue can begin copying while the previous one's directory is still on the disk. That is deliberate and worth stating rather than claiming a tighter guarantee than there is. Adds what the lock's location depends on, which is a deployment fact no node can check for itself, and how to set it.
1 parent 706b74e commit b0933ee

1 file changed

Lines changed: 21 additions & 6 deletions

File tree

docs/adr/ADR-0014-file-based-chunk-store-and-lmdb-retirement.md

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -231,10 +231,14 @@ Per node, in order:
231231
because `remove_dir_all` is not: a failure partway through leaves a directory that can
232232
no longer be opened as an environment, and recording completion on top of that would
233233
have the node claim it had finished over a half-deleted store. **This is where the disk
234-
comes back.** Every gate is rechecked inside the destructive step itself, in the same
235-
critical section that proves no other task holds the store, because the verification
236-
pass alone can run for hours and a write whose file half failed adds a key in the
237-
meantime.
234+
comes back.** The gates that can change while nobody is looking are rechecked inside
235+
the destructive step itself, in the same critical section that proves no other task
236+
holds the store: the proof's health generation, the answerability veto, the announced
237+
writes, and that every legacy-only key is in the approved set. The network gates, rank
238+
and commitment delivery and possession, are rechecked immediately before that call and
239+
outside the guard, so the window on those is the seconds it takes to take the guard
240+
rather than the hours the verification pass can run for. Both matter, and they are not
241+
the same claim.
238242
6. **Refetch** the shortfall through ordinary replication, with the freed space to do it in.
239243

240244
The delete gate is the pruner's existing retention contract
@@ -297,8 +301,19 @@ immediately, because it is never unable to serve.
297301

298302
Separately, a host-wide advisory lock serialises migrations sharing a volume, held from the
299303
first copy through retirement, so a node cannot release it and let eleven others start
300-
before it has returned a byte. The two limits answer different questions: the lock is about
301-
one machine's disk, the wave is about one chunk's replicas.
304+
before it has finished copying. It is released when the environment is unlinked and its
305+
directory renamed aside, not when the last byte comes back: the deletion itself runs
306+
detached so the node can serve while it happens, and it can take minutes on a large store.
307+
So the next node in the queue can begin its copy while the previous one's tombstone is
308+
still on the disk. That is deliberate, and it is worth stating rather than claiming a
309+
tighter guarantee than there is. The two limits answer different questions: the lock is
310+
about one machine's disk, the wave is about one chunk's replicas.
311+
312+
Where the lock file lives is a deployment fact, and the wrong answer is silent: nodes that
313+
cannot see each other's lock each take one and report success. A host whose nodes do not
314+
share a `/tmp`, which is any host using `PrivateTmp=true`, has to be told where the lock
315+
lives through `ANT_MIGRATION_LOCK_DIR`. The node logs the path it locked at so this can be
316+
answered from a log rather than inferred from a unit file.
302317

303318
## What the review added
304319

0 commit comments

Comments
 (0)