Make a runaway pljs function killable, and bound its stack - #37
Open
sfc-gh-okalaci wants to merge 4 commits into
Open
Make a runaway pljs function killable, and bound its stack#37sfc-gh-okalaci wants to merge 4 commits into
sfc-gh-okalaci wants to merge 4 commits into
Conversation
sfc-gh-okalaci
force-pushed
the
up/05-leaks
branch
from
August 18, 2026 09:15
9c0080d to
10aa254
Compare
sfc-gh-okalaci
force-pushed
the
up/06-operability
branch
from
August 18, 2026 09:15
7f8dfe2 to
583c53f
Compare
sfc-gh-okalaci
force-pushed
the
up/05-leaks
branch
from
August 18, 2026 09:24
10aa254 to
c8028d0
Compare
sfc-gh-okalaci
force-pushed
the
up/06-operability
branch
from
August 18, 2026 09:24
583c53f to
2f4e9a4
Compare
_PG_init() installed its own signal(SIGINT/SIGTERM/SIGABRT) handlers, replacing
PostgreSQL's for the life of the backend, and the QuickJS interrupt handler consulted
only pljs's private os_pending_signals bitmask. statement_timeout arrives as SIGALRM ->
QueryCancelPending, which that bitmask never observes, so a runaway JavaScript loop could
not be cancelled and the backend hung until it was killed.
The damage was not limited to pljs. Any backend that had ever run a pljs function lost
statement_timeout, pg_cancel_backend(), pg_terminate_backend() and fast shutdown for
every subsequent query in that session, whatever language it was written in. Observed
in practice: a while(true) function that ran for 2h37m of CPU, ignored cancel, terminate
and SIGTERM, and blocked around 40 pg_ctl restart attempts because the backend would not
exit and held its database open. SIGKILL was the only remedy.
pljs no longer installs handlers. The interrupt handler reads QueryCancelPending and
ProcDiePending directly, and each JS_Call/JS_Eval caller runs CHECK_FOR_INTERRUPTS()
once QuickJS has unwound to an exception, so the real PostgreSQL error is raised rather
than a JavaScript one.
Adds sql/pg_cancellation.sql. It also covers the case that would otherwise defeat the
whole thing: try { while(true){} } catch(e) {} still cancels, because the vendored
QuickJS marks the interrupt uncatchable.
QuickJS only performs stack checks when built with CONFIG_STACK_CHECK, and pljs relied on the vendored JS_DEFAULT_STACK_SIZE without ever setting a limit. Unbounded JavaScript recursion therefore ran the C stack into the ground and killed the process with SIGSEGV instead of raising a catchable "stack overflow". The budget is now set explicitly, derived from max_stack_depth so the DBA's setting is what governs it, with a floor so a small max_stack_depth still leaves JavaScript usable. Adds sql/pg_stack_depth.sql, covering pure-JavaScript recursion and mutual JavaScript/SQL recursion, and asserting the backend survives with a catchable error.
JS_NewRuntime() records the C-stack top once, and pljs creates the runtime in _PG_init -- the shallowest point in the backend. The budget set from that anchor describes stack that a real call does not have: JavaScript runs far deeper, since SQL -> JS -> pljs.execute -> SQL -> JS nests arbitrarily, and every level consumes stack the anchor knows nothing about. The consequence is the opposite of a weakened guard, and worse: at any nested depth QuickJS believes it is nearer the end of the stack than it is, so it refuses to continue while plenty of stack remains. Measured, a function recursing through pljs.execute() failed at depth 250 with QuickJS's "stack overflow" while depth 150 succeeded -- legitimate work rejected. JS_UpdateStackTop() re-anchors the measurement at each of the four entry points into JavaScript. Depth 300 then succeeds, and unbounded nesting is stopped by check_stack_depth() instead, which is the limit the DBA configured. Adds sql/pg_nested_stack_anchor.sql. It asserts which limit stops the nesting rather than a particular depth, because the depth at which either limit is reached depends on per-frame stack usage and so varies by platform and build flags.
The QuickJS interrupt handler consulted QueryCancelPending and ProcDiePending only, so conditions that also require a query to unwind were missed: a lost client connection, a recovery conflict, an idle-in-transaction timeout. It now consults InterruptPending as well, keeping the two specific flags as the fast path. The same paths leaked the exception value. The JS_FreeValue() sat after the error report, which does not return, so it was dead code -- one leaked QuickJS reference per cancelled or failed call rather than only on a cancel. Extract, free, then report. No regression test accompanies this. The widened condition covers interrupts a SQL script cannot raise on demand -- ClientConnectionLost, recovery conflict, idle-in-transaction timeout -- and the reference it stops leaking is one JSValue per interrupted call.
sfc-gh-okalaci
force-pushed
the
up/05-leaks
branch
from
August 18, 2026 09:29
c8028d0 to
8dcf76a
Compare
sfc-gh-okalaci
force-pushed
the
up/06-operability
branch
from
August 18, 2026 09:29
2f4e9a4 to
19364ba
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three operability fixes. The first is the most severe thing in this stack for anyone
running pljs in production.
Query cancellation was broken process-wide
_PG_init()installed its ownsignal(SIGINT/SIGTERM/SIGABRT)handlers, clobberingPostgreSQL's for the life of the backend, and the QuickJS interrupt handler consulted
only pljs's private
os_pending_signalsbitmask.statement_timeoutarrives asSIGALRM→QueryCancelPending, which that bitmask never observes.The consequences went well beyond pljs: any backend that had ever run a pljs function
lost
statement_timeout,pg_cancel_backend(),pg_terminate_backend()and fastshutdown for every subsequent query in that session, whatever language it was written
in. A
while(true)JavaScript loop was unkillable short ofSIGKILL, and because thebackend would not exit it blocked
pg_ctl restartand held its database sopg_regresscould not drop it. Observed: 2h37m of CPU and roughly 40 blocked restartattempts.
pljs no longer installs handlers. The interrupt handler reads
QueryCancelPendingandProcDiePendingdirectly, and eachJS_Call/JS_Evalcaller runsCHECK_FOR_INTERRUPTS()once QuickJS has unwound, so the real error is raised ratherthan a JavaScript one. The vendored QuickJS marks the interrupt uncatchable, so
try { while(true){} } catch(e) {}cannot defeat it.The interrupt check is widened at the same time, from
QueryCancelPending || ProcDiePendingto includeInterruptPending, so a lost client connection, a recoveryconflict or an idle-in-transaction timeout also unwinds JavaScript. Those paths were
additionally leaking the exception value: the
JS_FreeValuesat after a report that doesnot return, so it was dead code.
The JavaScript stack budget was measured from the wrong place
JS_NewRuntime()records the C-stack top once, in_PG_init, and the budget is sizedrelative to that anchor. Real JavaScript runs far deeper — SQL → JS →
pljs.execute→SQL → JS — and each level consumes stack the anchor knows nothing about. Because
_PG_initis the shallowest point in the backend, QuickJS believes it is nearer the endof the stack than it is and refuses to continue: measured, a function recursing through
pljs.execute()failed at depth 250 while depth 150 succeeded, with plenty of stackleft.
JS_UpdateStackTop()is now called at each entry into JavaScript, and deepnesting is bounded by
check_stack_depth()— the limit the DBA configured.The budget is derived from
max_stack_depth, and the test proves that: the achievablerecursion depth tracks it linearly (512kB → 268, 1024kB → 537, 2048kB → 1074,
4096kB → 2148), and the assertion fails if the explicit limit is not set.
Not included: re-throwing a failed commit
A
pljs.commit()that genuinely fails is still converted into a catchable JavaScriptexception, so a function can carry on running with no valid transaction state under it.
That is a real problem, and the obvious fix —
PG_RE_THROW()from thePG_CATCH—does not work:
pljs_commit()is a C function that QuickJS called, so re-throwingsiglongjmps past the interpreter's own frame list and the next JavaScript call in thesession faults. Reproduced with a deferred unique constraint, which fails at
COMMIT:Fixing it properly means letting QuickJS unwind first and re-raising the saved error
once control is back in C — the shape the interrupt handler already uses — which is a
larger change than belongs in this series. Left for a follow-up rather than shipped
half-done.
pljs.memory_limitbeing applied to the live runtime onSETmoved to the leaks PR,where the heap-leak tests depend on it.
Commits
Honor query cancellation instead of hijacking backend signalsBound the JavaScript stack size explicitlyRe-anchor the JavaScript stack budget at each entry into JavaScriptWiden the interrupt check, and stop leaking the exception valueEvery commit in this series builds from clean and passes the full ordered suite on its
own, verified per commit on PostgreSQL 17. The tip is additionally green on PostgreSQL
16, 17, 18 and 19beta3 — the versions this repository's CI matrix builds — with
pljs.memory_limit=64, and under AddressSanitizer with no reports.