Skip to content

prerender-v0.10.1 — @harperfast/prerender

Choose a tag to compare

@harper-joseph harper-joseph released this 27 Jul 22:09
1e27cfb

Fixes the repair sweep shipped in v0.10.0, which never ran.

The bug

Clicking "Run repair sweep" reported:

Last sweep failed: cacheKey is not indexed and not combined with any other conditions

The primary key is not flagged indexed in Harper's attribute metadata, and Table.js's sort-alignment branch rejects a sort on a non-indexed attribute unless a condition accompanies it. Harper does inject its own scan condition — but only after that check, so having the first page rely on that injection threw before the scan started. The sweep failed on its first call, so nothing was written: no partial repair, no corruption.

The fix

The walk no longer pages, sorts, or uses a cursor at all — one streamed pass:

RenderTarget.search({ select: ['cacheKey', 'renderInterval', 'sitemapUrl'] })

Paging by primary key had made correctness depend on the storage engine returning rows in key order. That is not the engine's contract to keep (v5 defaults to RocksDB), and the failure mode is a repair tool silently skipping rows. Removing the cursor removes the assumption rather than documenting it.

Two consequences:

  • The transaction rule is now structural. Restores are collected during the scan and written after it closes, so no write can be issued while the cursor is open — previously true only by careful paging.
  • The cap bounds writes only. The scan always runs to completion, so the reported missing is the true size of the gap even when only maxRestores of it was repaired this pass. The loop never breaks early, so the iterator is always fully consumed and its read transaction always released.

render.reconcile.batchSize is removed; it described paging that no longer exists.

Also

Tests were mutation-checked rather than trusted: reintroducing the v0.10.0 sort fails the suite, as does interleaving writes into the scan. A new test asserts the sweep returns identical stats for shuffled and sorted input, pinning order-independence as a property.

189 tests.

Field note

Measured on a live four-node cluster while diagnosing this, via the operations socket: RenderTarget 14,742 on every node, and the RenderSchedule residency shards summing to exactly 14,742. So on that deployment the schedule gap is currently zero and a working sweep restores nothing — the reconciler is a safety net there, not an outstanding repair.