Make the validator validate, and keep the function cache coherent - #38
Open
sfc-gh-okalaci wants to merge 3 commits into
Open
Make the validator validate, and keep the function cache coherent#38sfc-gh-okalaci wants to merge 3 commits into
sfc-gh-okalaci wants to merge 3 commits into
Conversation
sfc-gh-okalaci
force-pushed
the
up/07-validator-and-cache
branch
from
August 18, 2026 07:13
4295ab5 to
f2d5dcd
Compare
sfc-gh-okalaci
force-pushed
the
up/06-operability
branch
from
August 18, 2026 09:15
7f8dfe2 to
583c53f
Compare
sfc-gh-okalaci
force-pushed
the
up/07-validator-and-cache
branch
from
August 18, 2026 09:15
f2d5dcd to
e7fa72b
Compare
sfc-gh-okalaci
force-pushed
the
up/06-operability
branch
from
August 18, 2026 09:24
583c53f to
2f4e9a4
Compare
sfc-gh-okalaci
force-pushed
the
up/07-validator-and-cache
branch
from
August 18, 2026 09:24
e7fa72b to
401b753
Compare
pljs_call_validator() read fcinfo->flinfo->fn_oid -- its own OID. The OID of the function being created arrives as PG_GETARG_OID(0). So it fetched its own pg_proc row and compiled that row's prosrc: the C symbol name "pljs_call_validator", which parses as a bare JavaScript identifier. The compile always succeeded, every invalid body was accepted at CREATE FUNCTION time, and the syntax error appeared only on the first call. Correcting the OID alone is not enough, and this is the part worth explaining. A pljs body is a function *body*, not a program: `return 42;` is a syntax error at top level, so validating the raw prosrc rejects almost every valid function. The body has to be wrapped exactly the way compilation wraps it, including the argument names and a trigger's implicit variables. The wrapper construction is therefore extracted from pljs_compile_function() and shared, so the validator and the compiler agree by construction rather than by two copies staying in step. The validator compiles with JS_EVAL_FLAG_COMPILE_ONLY -- it must not run the body -- and releases the value and the context on both paths. check_function_bodies = off still skips validation, so restoring a dump whose functions reference not-yet-created objects keeps working. Adds sql/pg_validator.sql: an invalid body rejected at CREATE time, the valid shapes a raw-prosrc check would wrongly reject (a bare return, named arguments, a trigger), and deferral under check_function_bodies = off with the error surfacing on first call.
Creating or replacing a pljs function called pljs_cache_reset(), which walks every per-user cache entry and destroys the per-user JSContext with it. JS_FreeContext() does not free a context that still has live references into it, so the old context was not necessarily reclaimed -- and the next call built a fresh one. A backend doing repeated DDL therefore accumulated contexts on the QuickJS heap, where PostgreSQL's memory accounting cannot see them. Measured on a tight CREATE OR REPLACE FUNCTION loop: around 536MB of resident memory and then a SIGSEGV, within seconds. Afterwards, roughly 20MB and no crash, with throughput up because the common case no longer rebuilds an interpreter per statement. pljs_cache_function_remove() drops just the entry for the OID being replaced, from every user's cache, releasing the compiled function and the cached source. The context survives, which is correct: it is per-user state, not per-function state.
The compiled-function cache is per session and pljs registers no syscache invalidation callback, so the only invalidation was the one the validator performs in the backend issuing the DDL. CREATE OR REPLACE FUNCTION was therefore invisible to every other session: one that had already called the function kept running the body it first compiled, for the rest of its life, however many times the function was replaced. Deploying a new definition meant recycling every existing connection, and nothing said so -- which makes it look like a failed deploy rather than a caching rule. pljs_func already declared fn_xmin and fn_tid for this purpose; neither was ever written or read. They now record which pg_proc tuple an entry was compiled from, and every cache hit compares them against the tuple the caller already holds. CREATE OR REPLACE writes a new tuple version while keeping the OID, so a mismatch is exactly the signal that the compiled copy is stale. This is what plpgsql does; see plpgsql_compile(). The check lives inside pljs_cache_function_find() rather than at its call sites, so neither lookup can be changed later without it. It also settles DROP FUNCTION followed by OID reuse, where a new function inherits a dropped one's OID: the tuple differs, so the mismatch is caught rather than the old body running under the new name. Adds sql/pg_cross_backend_invalidation.sql, which replaces a function from a second backend and asserts this one sees the new body, for both the direct call path and pljs.find_function().
sfc-gh-okalaci
force-pushed
the
up/06-operability
branch
from
August 18, 2026 09:29
2f4e9a4 to
19364ba
Compare
sfc-gh-okalaci
force-pushed
the
up/07-validator-and-cache
branch
from
August 18, 2026 09:29
401b753 to
0711556
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three fixes to
CREATE FUNCTIONand the per-session compiled-function cache. These dochange when an error surfaces, which is why they are separate from the crash fixes.
The validator never validated anything
pljs_call_validator()readfcinfo->flinfo->fn_oid— its own OID, not the OID of thefunction being created, which arrives as
PG_GETARG_OID(0). So it fetched its ownpg_procrow and compiled that row'sprosrc: the C symbol namepljs_call_validator, which parses as a bare JavaScript identifier. The compile alwayssucceeded, every invalid body was accepted at
CREATEtime, and the syntax errorappeared only on the first call.
Correcting the OID alone is not enough. A pljs body is a function body, not a
program, so
return 42;is a syntax error at top level — validating the rawprosrcrejects almost every valid function. The wrapper construction is therefore extracted
from
pljs_compile_function()and shared, so the validator and the compiler agree byconstruction rather than by two copies staying in sync.
check_function_bodies = offstill skips validation, so restoring a dump whose functions reference not-yet-created
objects keeps working.
Repeated DDL grew the backend without bound
Creating or replacing a function called
pljs_cache_reset(), which destroys everyper-user
JSContext. QuickJS will not free a context that still has live referencesinto it, so the old one was not necessarily reclaimed and the next call built a fresh
one. A tight
CREATE OR REPLACE FUNCTIONloop reached roughly 536MB and then crashedthe backend within seconds. Dropping only the affected function's entry keeps the
context — which is per-user state, not per-function state. Measured on the same loop:
~20MB peak, no crash, and throughput rises because the common case no longer rebuilds
an interpreter per statement.
Whether the context survived is observable from JavaScript, which is what
sql/pg_targeted_invalidation.sqlchecks: anything held onglobalThislives in thatcontext, so it is still there after unrelated DDL exactly when the context was not
destroyed. Without the fix it reads
(gone).A replaced function kept running the old body elsewhere
The cache is per session and there is no syscache invalidation callback, so
CREATE OR REPLACE FUNCTIONwas invisible to every other backend: a session that hadalready called the function ran the body it first compiled for the rest of its life.
Deploying a new definition required recycling every existing connection, and nothing
said so.
pljs_funcalready declaredfn_xminandfn_tidfor this; they were never writtenor read. They now record which
pg_proctuple an entry was compiled from and arechecked on every cache hit, which is what plpgsql does. This also settles
DROP FUNCTIONfollowed by OID reuse, where the old body could otherwise run under the newfunction's name.
Commits
Validate the function being created, not the validator itselfInvalidate only the replaced function, not the whole context cacheDetect a stale cached function instead of running the old bodyEvery commit in this series builds from clean and passes the full ordered suite on its
own, verified per commit on PostgreSQL 17. The tip is additionally green on PostgreSQL
16, 17, 18 and 19beta3 — the versions this repository's CI matrix builds — with
pljs.memory_limit=64, and under AddressSanitizer with no reports.