bedrock: lock lease renewal and expiry

[christopher-buss]

Parent

#114

What to build

A Lease on the hold, so a Deploy killed by a cancelled CI job does not block every later deploy behind manual intervention. CI runners kill process trees on cancellation, and a hold that outlives its holder forever is the dominant operational complaint against locking in comparable tools.

The hold carries a deadline, renewed while the deploy runs. A hold whose lease stops being renewed expires and may be taken over.

Blocked by the conditional State write because that is what makes takeover safe: a holder that kept running past its expired lease cannot overwrite the State written by whoever took over.

Acceptance criteria

• The lock record carries a deadline, renewed periodically while the Deploy runs
• A hold whose lease is still being renewed is never taken over, however long the deploy runs
• A hold whose lease has expired can be taken over by a waiting deploy
• Takeover is itself conditional, so two waiters cannot both take over the same expired hold
• A holder that lost its lease and kept running fails its State write rather than overwriting the newer record
• Renewal failure is surfaced rather than silently continuing as though the hold were still held
• 100% coverage on touched source, no surviving mutants

Blocked by

• bedrock: conditional state write #555
• bedrock: s3 lock acquisition and release #556