fix(security): authorize schema-unqualified SQL against the table the engine resolves - #1961
Conversation
…table GHSA-5c29-q62v-jrwf. The SQL authorization layer derived the affected schema/table set from the AST's `databaseid`. A statement that omitted the schema qualifier left that field empty, so `addSchemaTableToMap` recorded nothing and `verifyPermsAST` ran `hasPermissions` against an empty map — which authorizes by vacuous truth. The v2 engine's binder then resolved the same bare name to a concrete database via `pickDefaultDatabase` and executed against it. Two independent name resolutions, one of which silently resolved to "nothing", let any authenticated user reach any uniquely-named table. Verified against a live instance with a role holding no grant at all on the target: SELECT id, ssn FROM UnqualSecret WHERE id='...' -> 200, row returned DELETE FROM UnqualSecret WHERE id='...' -> 200, row destroyed UPDATE UnqualSecret SET ssn='tampered' -> 200, row tampered INSERT INTO UnqualSecret ... -> 200, row written So the exposure is not confidentiality-only as originally reported: bare INSERT/UPDATE/DELETE bypass the check the same way, because `chooseOperation` pre-populates `parsed_sql_object` and thereby skips evaluateSQL's "No schema specified" guard for the ops `sql` path. Bare JOIN targets and `SELECT *` are affected too. The fix removes the divergence rather than patching either side: - `sqlEngine/binder/defaultDatabase.ts` becomes the single owner of the default-database rule; the binder's `pickDefaultDatabase` now delegates to it. - The statement bucket resolves bare FROM/JOIN/INSERT/UPDATE/DELETE targets through that same rule and writes the result back onto the AST, so authorization and execution share one resolution instead of performing two that can disagree. A name that resolves to zero or several databases is left bare rather than guessed. - `verifyPermsAST` fails closed: an empty affected-table set is legal only for a statement that names no table (`SELECT ABS(-12)`). A statement that does name one but produced no entry is denied, so any future recording gap degrades to a denial instead of a bypass. Tests: integration suite covering read (indexed and full-scan), system table, DELETE/UPDATE/INSERT with mutation checks, JOIN, and `SELECT *`, plus positive cases proving legitimate bare references still resolve. 9 of its 11 cases fail on the unfixed tree. Unit tests cover resolution, ambiguity, and the registry -unavailable path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tatement Cross-model review (Gemini) found the first cut's fail-closed guard was too coarse. It tested whether the affected-schema map was empty overall, so a statement mixing a resolvable table with an unresolvable one passed on the strength of the table that did resolve — the unresolvable reference was neither recorded nor denied. It also missed constructs the collectors never model. Replace the global check with the actual invariant: every table the statement references must appear in the affected-attribute map that permission checking iterates. `getUnauthorizedTableRefs()` returns a description of each reference that does not, and verifyPermsAST denies on any. That covers, in one rule, unresolvable bare names, a mixed resolvable/unresolvable statement, a derived table (whose reach we cannot determine), and a SELECT's INTO target (which the collectors do not record). An empty result still means a calc-only select. Also from the review: - Own-property lookup when matching a table name against a database. `FROM toString` previously matched every database via Object.prototype, and the binder would have bound the inherited method as a table resource. - Load the database registry once per statement instead of once per bare reference. - Consolidate the duplicate `require` of the statement bucket in verifyPermsAST. - Guard the three `affectedAttributes.get(schema)` dereferences in getSelectAttributes. A statement whose FROM resolved while a JOIN did not now reaches this code and crashed on the undefined schema entry — a 500 where the statement should get a clean 403. Empirically checked before fixing, so the scope claims are grounded: derived tables, WHERE subqueries, CTEs and UNION arms all fail before executing (500 / 400), and `SELECT ... INTO` returns rows without writing. None were live bypasses; they are latent gaps that now fail closed rather than depending on a downstream engine error. Residual, noted in the PR: references reachable only from a WHERE subquery are still not enumerated — unreachable today because those statements error out. Tests: 3 new integration cases (mixed resolvable/unresolvable FROM, derived table, SELECT INTO) and 10 unit cases over getUnauthorizedTableRefs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hasPermissions iterates the affected-attribute map, so a table missing from it is never checked and an empty map authorizes by vacuous truth. Write down the two rules that follow — resolve names once through defaultDatabase.ts, and make any new SQL construct either record its references or report them — so the next change to the SQL layer doesn't reintroduce GHSA-5c29-q62v-jrwf. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request addresses a security vulnerability (GHSA-5c29-q62v-jrwf) where schema-unqualified SQL statements could bypass authorization checks. It centralizes default-database resolution in a new module, resolves unqualified table references in place during AST interpretation, and fails closed on any unresolvable or unauthorized table references. Additionally, it includes comprehensive integration and unit tests to verify the fix. The reviewer's feedback suggests guarding the "databases" object in "bindTableRef" before calling "Object.hasOwn" to prevent a potential "TypeError" if the object is null or undefined.
|
Reviewed the commit pushed since the last review (9e9ca92 — ORDER BY schema guard matching sibling collectors); no blockers found. |
…authorize behavior fef6947/b8a665ea0 made schema-unqualified SQL resolve to the one database that defines the table, authorize against it, and execute — by design (see integrationTests/security/sql-unqualified-table-authz.test.ts's POSITIVE case and unitTests/sqlTranslator/unqualified-table-resolution.test.js, both of which assert this resolve-and-succeed behavior). `orders` exists only in `northnwd`, so `select * FROM orders` now runs like the qualified form instead of throwing "schema not defined for table orders" — the old assertion predates the fix and was failing CI. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…r record Review round 2. The attribute collectors only walk a statement's top-level FROM, JOINs and write target, so a table reachable only from a nested or compound query is invisible to permission checking. getTableTargets now reports those constructs as opaque, and getUnauthorizedTableRefs refuses them: - WHERE subqueries (`queries`), EXISTS subqueries (`exists`), UNION branches (`union`/`unionall`) and an INSERT's source SELECT (`select`). - A SELECT's INTO target is now unconditionally opaque rather than a named reference. As a named reference it satisfied the membership test against the entry the FROM collector created, so `SELECT * INTO data.t FROM data.t` authorized a write on the strength of a read. Verified: that statement returns 200 today. Opaque rather than recursing: a nested reference would still never be recorded in the affected-attribute map, so descending to name it would not make it checkable — only refusable, which opaque already achieves. `INSERT INTO t SELECT ... FROM src` turns out never to respond at all — the request wedges (confirmed with a bounded probe: no response in 8s; an unbounded suite run hung past 900s). Authorization previously let it through, since the source table is invisible and only the target's insert grant is checked, so any user holding insert on one table could hang requests. Refusing the construct closes that for non-super-users; the underlying hang still needs its own fix and is filed separately. Also resolve a column's database through the table-to-schema map, which is keyed by alias AND base table name, instead of falling back to the FROM table's database. Once bare names resolve, a statement can span two databases with no aliases, and the old fallback attributed a joined table's columns to the wrong database. Reported as an attribute-permission bypass; I could NOT reproduce that — `SELECT UnqualVault.ssn FROM UnqualPublic JOIN UnqualVault` is denied with "Attribute 'ssn' does not exist" both with and without this change, so it lands as correctness hardening, not a proven bypass fix, and its test is a characterization guard. Plus: guard `databases` before Object.hasOwn in bindTableRef (bot review). Tests: 8 new integration cases and 8 new unit cases. Verified non-vacuous by reverting each guard — the WHERE-subquery, EXISTS and UNION cases return 500 instead of 403 without it, and with the INTO and nested-source guards removed the suite wedges entirely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AlaSQL parses `JOIN (SELECT ...) AS s` with the source on `join.select` and no `join.table`. Passing that absent table to addRef dropped it from both `named` and `opaque`, so it left the inventory entirely — and attribute collection then died on `join.table.as`, surfacing as a 500 before getUnauthorizedTableRefs could refuse it with a 403. Mark a table-less JOIN opaque explicitly, and skip it during attribute collection. Not a present bypass — no engine executes this form — but it closes the last hole in the fail-closed inventory and stops derived-JOIN support from being a trap later. Unit and integration coverage added; both fail (500, not 403) without the fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review round 2 — what changed, and one finding I could not reproduceAn earlier review pass raised three issues on threads that no longer exist (the pending review was replaced before I could reply). Recording the outcomes here so they are not lost, since two of them changed the code materially. Nested and compound queries were invisible to permission checking — fixed in I marked them opaque rather than recursing into them, because a nested reference would still never be recorded in the affected-attribute map — descending to name it would not make it checkable, only refusable, which opaque already achieves. This turned up a hang.
An attribute-permission bypass on unaliased cross-database joins — could not reproduce. I built the reported scenario exactly: two databases, no aliases anywhere, The attribute check is reached by some other path and On verificationEvery guard added in this round was checked by reverting it and re-running, so none of the new tests are vacuous:
Local runs: 429 unit passing, 632 integration passing (security, SQL engine, northwind), lint clean. Coverage caveatThe Codex leg of the local cross-model review timed out and that reviewer fell back to its own analysis — same model family as the author, so it did not count as outside coverage. The findings above came from the repo's own CI review bots and from Gemini. Worth a human pass on the authorization logic specifically. Generated with Claude Opus 5. |
…ttribute map getSelectAttributes() walked columns, WHERE, JOIN ON, and ORDER BY to build the affected-attribute map that checkAttributePerms() enforces against, but never GROUP BY or HAVING. A column referenced only there (`SELECT COUNT(*) FROM t GROUP BY ssn`, `... HAVING ssn = 'x'`) never entered the map, so a role denied READ on that attribute could still observe its distribution — unauthorized consumption the pending draft review flagged as a blocker. Mirrors the existing WHERE-clause collection so it shares the same resolution and lookup fallback behavior. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
df39ac2 marked UNION/UNION ALL branches opaque because the attribute collectors never descend into them, but missed EXCEPT and INTERSECT — the same alasql compound-query family, carried on `ast.except`/`ast.intersect` exactly like `ast.union`/`ast.unionall`. A table reachable only from one of these branches was invisible to `getUnauthorizedTableRefs()` and never refused. Confirmed live on the unfixed tree: an EXCEPT/INTERSECT branch targeting the forbidden table is not authorization-denied — it reaches execution and only happens to fail with a 500 from the search layer, the same "protected by an engine accident, not by authorization" pattern this PR already calls out and closes for full-scan SELECTs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…UNPIVOT Addresses the pending draft review's inline comments (kriszyp, PENDING review 4790743002), which named three concrete gaps beyond the GROUP BY/HAVING one already fixed: - ORDER BY resolved a column's schema via `schemaLookup` alone (aliases only), instead of `resolveColumnSchema` (alias + base table name via `tableToSchemaLookup`, same as WHERE/columns since df39ac2). An unaliased joined table's column landed on the FROM table's schema entry instead of its own, so the column never reached the joined table's attribute list and its attribute-level permission was never checked — e.g. `SELECT Public.id FROM data.Public JOIN vault.Vault ON Public.id = Vault.id ORDER BY Vault.ssn` with `vault.Vault.ssn` denied. - PIVOT/UNPIVOT carry their pivoted column on `pivot.expr`/`pivot.columnid`, which the collectors never walk. Marked opaque, same treatment as derived tables and SELECT INTO — not used anywhere in Harper today, so refusing it changes no working behavior. - The existing "attribute-level denial ... cross-database join" integration test joined on non-matching row ids (`public-row-1` / `vault-row-1`), so the join always returned zero rows and the test passed regardless of whether the attribute check ran. Added a matching-key row and an `isDenied` assertion; confirmed the real response is a 403 with "Attribute 'ssn' does not exist". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The SELECT-list, WHERE, GROUP BY, and HAVING collectors all bail out with a clean skip when affectedAttributes has no entry for the resolved schema. The ORDER BY collector dereferenced the same lookup unguarded. Add the matching guard so a future change to the resolution invariant degrades to a skip instead of a crash. Addresses review feedback on PR #1961. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
| /** | ||
| * Override hook for tests. Restored to default by passing null. | ||
| */ | ||
| export function _setDatabasesLoader(loader: (() => DatabaseRegistry) | null): void { |
There was a problem hiding this comment.
Not sure I understand why an exported function begins with an underscore. It's fairly common to prefix local variables or unused variables with an underscore, but an exported function is public and should not have an underscore. This looks like legacy code, but while you're touching it, might as well fix it.
Fixes the authorization bypass tracked in GHSA-5c29-q62v-jrwf.
The bug
The SQL authorization layer derived the affected schema/table set from the AST's
databaseid. When a statement omitted the schema qualifier that field was empty, soaddSchemaTableToMaprecorded nothing andverifyPermsASTranhasPermissionsagainst an empty map.hasPermissionsiterates that map — an empty map authorizes by vacuous truth. The v2 engine's binder then resolved the same bare name to a concrete database viapickDefaultDatabaseand executed against it.Two independent name resolutions, one of which silently resolved to "nothing".
Scope is wider than the advisory
The advisory describes this as confidentiality-only. Verified against a live instance, with a role holding no grant at all on the target table:
SELECT id, ssn FROM UnqualSecret WHERE id='…'200—[{"id":"secret-row-1","ssn":"111-22-3333"}]DELETE FROM UnqualSecret WHERE id='…'200— row destroyedUPDATE UnqualSecret SET ssn='tampered'200— row tamperedINSERT INTO UnqualSecret …200— row writtenSo it is integrity and availability too, not just confidentiality. Writes bypass because
chooseOperationpre-populatesparsed_sql_object, which skipsevaluateSQL's "No schema specified" guard on the opssqlpath. Bare JOIN targets andSELECT *are affected the same way.Two advisory claims did not reproduce, and the advisory should be corrected:
SELECT * FROM hdb_userreturns200 []— thesystemdatabase isn't ingetDatabases(), sopickDefaultDatabasecan't resolve it. No credential material is exposed.EngineUnsupportedErrorand falls back to legacy, which rejects unqualified names. The indexed-predicate form is the live read vector. Both were unauthorized regardless — the denial came from an engine capability limit, not from authorization.The fix
Rather than patching either side, remove the divergence.
sqlEngine/binder/defaultDatabase.ts(new) is the single owner of the default-database rule. The binder'spickDefaultDatabasedelegates to it.verifyPermsASTfails closed per reference. Every table the statement references must appear in the affected-attribute map; any that doesn't is denied. Checked per reference rather than on the map being empty overall — a statement mixing a resolvable table with an unresolvable one has a non-empty map and would otherwise slip through on the strength of the table that did resolve. An empty result still means a calc-only select (SELECT ABS(-12)).I did not take the advisory's suggested "reject unqualified SELECT targets": bare names are a deliberate v2 feature (
pickDefaultDatabasehas an explicit ambiguity error), so banning them would paper over the resolution mismatch instead of fixing it.Behavior changes beyond the bypass
FROM (SELECT …)) and aSELECT … INTOtarget are now refused for non-super-users. Neither executes today — Harper rejects derived tables with a 500 and ignoresINTOwithout writing — so nothing that worked stops working, but they are no longer authorized-by-omission if either is ever implemented.500 schema not defined, because the resolved database reaches the legacy path too.getSelectAttributes; it now returns a clean 403. Guarded at all threeaffectedAttributes.get(schema)dereferences.Known limitation
References reachable only from a WHERE subquery are still not enumerated. Not exploitable today — those statements fail with
Circular referencebefore executing — but the enumeration is FROM/JOIN/write-target only, so a future subquery implementation must extendgetTableTargets. Recorded as an invariant inAGENTS.md.Testing
integrationTests/security/sql-unqualified-table-authz.test.ts— 14 cases: read (indexed and full-scan), system table, DELETE/UPDATE/INSERT each with a follow-up check that the table was not mutated, JOIN,SELECT *, mixed resolvable/unresolvable FROM, derived table,SELECT INTO, plus positive cases proving legitimate bare references still resolve and return rows.unitTests/sqlTranslator/unqualified-table-resolution.test.js— 16 cases over resolution, ambiguity,Object.prototypenames, and every branch ofgetUnauthorizedTableRefs.Review
Cross-model review found the first cut's fail-closed guard was too coarse (global instead of per-reference), plus the
Object.prototypetable-name match, the per-reference registry load, and the duplicaterequire— all fixed inb8a665ea0. Every flagged construct was checked empirically before fixing; the "subquery bypass" reported as a blocker is a latent gap, not a live bypass, since those statements error out before executing.Coverage caveat: the Codex leg timed out, and that reviewer fell back to its own analysis — same model family as the author, so it does not count as cross-model coverage. Gemini (
agy) is the only genuine outside-model pass here, and it is the one that caught the per-reference gap. A second outside-model pass before merge would be worthwhile.Generated with Claude Opus 5.
Docs
No companion documentation PR needed.
reference/operations-api/sql.mdalready states that tables are referenced asdatabase.table, and already listsUNIONas unsupported (✗). The constructs this PR now refuses for non-super-users — UNION branches, WHERE/EXISTS subqueries, derived tables and derived JOINs,INSERT ... SELECT, andSELECT ... INTO— are either documented as unsupported or undocumented, and none of them execute successfully today, so no documented behavior changes. Bare table names remain undocumented; the docs continue to steer users to the qualified form.Follow-up filed
#1965 — INSERT INTO ... SELECT never returns (request hangs). Found while validating this fix. Refusing the construct here removes the vector for non-super-users, but super-users return before the authorization check and still reach the hang, so it needs its own fix.