Skip to content

prerender-v0.9.0 — @harperfast/prerender

Choose a tag to compare

@harper-joseph harper-joseph released this 27 Jul 20:41
a48f01c

First release containing the URL-explainer hang fix. Anyone on 0.8.0–0.8.2 should upgrade: on a multi-node cluster the explainer hung indefinitely for most URLs.

Covers PRs #32, #33, #34 and #36.

The hang (found in production)

RenderSchedule is residency-pinned via setResidencyById, and harper-pro's replicator registers a sourceLoad on every replicated table. A point get for a row owned by another node therefore took Harper's cross-node path:

getRecord(request) {
  return new Promise((resolve, reject) => {
    ws.send(encode(message));
    awaitingResponse.set(requestId, { resolve, reject });   // no timeout, ever
  });
}

That promise settles only when a GET_RECORD_RESPONSE arrives. No deadline — an unanswered peer hangs the request forever.

Rendezvous hashing spreads ownership evenly, so (N−1)/N of URLs are owned elsewhere — ~75% on a 4-node cluster. This was the common path, not an edge case.

Fixed by reading node-locally with { replicateFrom: false }, as claim, refreshQueueStatus, and the admin overview scan always did, plus a 3s deadline on every explainer read.

Cross-node answers, without the hang

Node-local reads alone would leave the explainer unable to answer for ~75% of URLs. So when this node isn't the owner and has no local row, it asks the owner over HTTPS via a new leaf route POST /prerender_admin/schedule — a bounded request in place of an unbounded one.

  • Destination is always a hostname from the cluster's own node list, never derived from the request.
  • Only the caller's authorization/cookie are forwarded; the peer re-runs its own super_user check, so no authority is granted that the caller lacked. Both work cluster-wide (users are replicated; the session cookie is issued for the shared parent domain).
  • Bounded by management.peerTimeoutMs (2.5s).
  • The route never proxies onward, so residency disagreement can't cause a loop.

New config: management.proxyToOwner (default true), management.peerTimeoutMs (2500). Set proxyToOwner: false to keep every read strictly node-local.

Queue pause/resume, now actually working

Two defects fixed since 0.8.0, both of which made the control silently misreport:

  • 0.8.1 — the intent read used a string select, which projects to a bare scalar rather than a record. Reading .paused off a boolean gave undefined, treated as "no opinion", so every pause resolved to "not paused" while the row sat in the table looking applied.
  • 0.8.2setPause re-resolved the intent in the same request, but a row deleted earlier in that request is still visible to the read. A resume therefore re-read the row it had just deleted and reported "still paused". Fixed by resolving from the just-written value.

Verified end to end against a live instance: pause, claim gating, resume, cluster pause, a per-node paused: false override running through a cluster pause, clearing that override falling back to it, and all three validation rejections.

Degraded reads never masquerade as absences

A timed-out read yields a null row, which was rendered identically to a row read and found empty (Not scheduled — nothing will render this URL., No cached page under this key., and the miss/no render target badges). That turned a degraded response into a confident false negative — the worst answer for someone debugging a page that isn't rendering.

Every empty state and both badges now distinguish unknown from absent, and verdict.reliable says plainly whether the derived verdict can be trusted.

Notes

  • 170 tests. The read-side suite uses fakes reproducing Harper's projection and write-visibility semantics, so both queue-control defects are pinned.
  • The cross-node and timeout paths are unreachable on a single node — worth confirming on a cluster after deploy that residency.scheduleSource reads owner <node> for a remotely-owned URL.
  • Upstream: getRecord()'s missing timeout is a harper-pro defect. Any component doing a point read on a residency-pinned replicated table can hang a request indefinitely.