|
| 1 | +# ParserSQL |
| 2 | + |
| 3 | +This directory vendors the [ParserSQL] SQL parser library that ProxySQL |
| 4 | +links into `libsqlparser.a` and uses for `mysql-set_parser_algorithm=3` |
| 5 | +/ `pgsql-set_parser_algorithm=3` (the ParserSQL-backed SET parser) and |
| 6 | +related dialect-aware parsing paths. |
| 7 | + |
| 8 | +[ParserSQL]: https://github.com/ProxySQL/ParserSQL |
| 9 | + |
| 10 | +## Layout |
| 11 | + |
| 12 | +- `parsersql-<VERSION>.tar.gz` — vendored upstream release tarball, |
| 13 | + expected to be byte-identical to `git archive --prefix=ParserSQL-<VERSION>/ |
| 14 | + v<VERSION>` against the [upstream repo][ParserSQL]. |
| 15 | +- `parsersql` — symlink to the extracted source directory |
| 16 | + (`parsersql-<VERSION>/`), created by the top-level build at extract |
| 17 | + time. The symlink is what every other Makefile rule and `-I` / |
| 18 | + `-L` flag references, so the rest of the build is version-agnostic. |
| 19 | +- `README.md` — this file. |
| 20 | + |
| 21 | +`deps/Makefile`'s `parsersql/parsersql/libsqlparser.a` target unpacks |
| 22 | +the tarball (renaming `ParserSQL-*` → `parsersql-*` for case-insensitive |
| 23 | +filesystem friendliness) and runs `make lib` inside it. |
| 24 | + |
| 25 | +## Upstream |
| 26 | + |
| 27 | +- Repository: **https://github.com/ProxySQL/ParserSQL** |
| 28 | +- License: see `parsersql/LICENSE` after extraction. |
| 29 | +- Maintainers: same team as ProxySQL (sysown). It is a sibling project, |
| 30 | + not a third-party dependency — bug reports and PRs are welcome there. |
| 31 | + |
| 32 | +## Bumping the version |
| 33 | + |
| 34 | +```bash |
| 35 | +# In the ParserSQL repo, after tagging vX.Y.Z: |
| 36 | +git archive --format=tar.gz --prefix=ParserSQL-X.Y.Z/ vX.Y.Z \ |
| 37 | + -o /tmp/parsersql-X.Y.Z.tar.gz |
| 38 | + |
| 39 | +# In ProxySQL: |
| 40 | +cd deps/parsersql |
| 41 | +rm parsersql-*.tar.gz |
| 42 | +cp /tmp/parsersql-X.Y.Z.tar.gz . |
| 43 | +rm parsersql && ln -s parsersql-X.Y.Z parsersql |
| 44 | +# Then git add the new tarball + symlink and commit. |
| 45 | +``` |
| 46 | + |
| 47 | +The Makefile auto-detects the version via `parsersql-*.tar.gz` and |
| 48 | +`ParserSQL-*/` glob patterns, so no Makefile change is required for |
| 49 | +a routine bump. |
| 50 | + |
| 51 | +## Working on the parser: fix it upstream, don't work around it |
| 52 | + |
| 53 | +When `set_parser_algorithm=3` (or any other ParserSQL-backed code path) |
| 54 | +misbehaves, the **preferred fix is in [ParserSQL][ParserSQL] itself**, |
| 55 | +not a workaround in `lib/Query_Processor_ParserSQL.cpp` or in the |
| 56 | +PgSQL_Session / MySQL_Session handlers. |
| 57 | + |
| 58 | +Reasons: |
| 59 | + |
| 60 | +1. **One source of truth.** ParserSQL exists to be the SQL parsing |
| 61 | + layer. If ProxySQL silently compensates for a parser bug, the next |
| 62 | + consumer (or a future ProxySQL refactor) will hit the same bug and |
| 63 | + re-discover the workaround. |
| 64 | +2. **Tests live where the code lives.** Every parser fix should ship |
| 65 | + with regression coverage in `tests/test_*.cpp` upstream so the bug |
| 66 | + can't come back. Workarounds in ProxySQL only get tested by the |
| 67 | + downstream TAP suite, which is heavier and slower to run. |
| 68 | +3. **Single version bump.** The expected workflow is: identify all |
| 69 | + related parser issues at once, fix them upstream in one PR, tag a |
| 70 | + new release, and bump here exactly once — rather than dribbling a |
| 71 | + ProxySQL workaround per bug followed by a parser bump that then |
| 72 | + needs the workarounds reverted. |
| 73 | +4. **AST richness.** ParserSQL already carries node-type information |
| 74 | + (e.g. `NODE_LITERAL_STRING` vs `NODE_COLUMN_REF` for `'val'` vs |
| 75 | + `name`, `FLAG_IDENT_DELIMITED` for `"name"` vs `name`). When |
| 76 | + ProxySQL collapses everything to text in the walker, that |
| 77 | + information is lost and downstream code can't make correct |
| 78 | + decisions. Push richer typing into the AST upstream rather than |
| 79 | + re-deriving it in the consumer. |
| 80 | + |
| 81 | +### When a ProxySQL-side change is the right call |
| 82 | + |
| 83 | +- The fix is about *what ProxySQL does with* a correctly-parsed AST |
| 84 | + (e.g. which validator to apply, which session variable to track, |
| 85 | + whether to lock a hostgroup). The AST is fine; the policy is in |
| 86 | + ProxySQL. |
| 87 | +- The fix is in the walker (`lib/Query_Processor_ParserSQL.cpp`) — for |
| 88 | + example, handling a new `NODE_*` type that ParserSQL emits but the |
| 89 | + walker hasn't been taught about yet. |
| 90 | +- The bug is in ProxySQL's pre/post processing around the parse |
| 91 | + (digest, normalization, scope-prefix stripping, etc.). |
| 92 | + |
| 93 | +### When a ParserSQL fix is the right call |
| 94 | + |
| 95 | +- A SQL form that PostgreSQL or MySQL documents as valid returns |
| 96 | + `ParseResult::ERROR` or `PARTIAL`, or produces an AST shape that |
| 97 | + loses information the standard preserves. |
| 98 | +- Two semantically-distinct inputs produce identical AST nodes (the |
| 99 | + parser is throwing away information the consumer needs). |
| 100 | +- A keyword or shorthand isn't recognized (`SET SCHEMA`, `SET LOCAL`, |
| 101 | + `SET TRANSACTION ...`). |
| 102 | +- The lexer truncates or mis-classifies tokens (`@@\`var\`` losing |
| 103 | + the closing backtick, `$word` falling through to a generic |
| 104 | + identifier instead of being rejected, etc.). |
| 105 | + |
| 106 | +## Audit history |
| 107 | + |
| 108 | +Significant audits / bumps: |
| 109 | + |
| 110 | +- **v1.0.8 (2026-05-23)** — Tokenizer: unclosed delimited identifier |
| 111 | + (`"name` in PG, `` `name `` in MySQL) now emits `TK_ERROR` instead |
| 112 | + of silently consuming everything to EOF as one giant identifier. |
| 113 | + Closes the case-#172 cascade in ProxySQL's |
| 114 | + `pgsql-set_parameter_validation_test-t` under `set_parser_algorithm=3`, |
| 115 | + where `SET search_path = "unclosed_quote, public` was being parsed |
| 116 | + as identifier `unclosed_quote, public` and then accepted by the |
| 117 | + search_path validator — corrupting the stored value with what PG |
| 118 | + itself would have rejected outright. |
| 119 | + ([PR #44](https://github.com/ProxySQL/ParserSQL/pull/44)) |
| 120 | +- **v1.0.7 (2026-05-23)** — Tokenizer: allow `$` as PG identifier |
| 121 | + continuation char (per PG lexical-syntax docs). Before, `SET |
| 122 | + search_path = schema$1` truncated to `schema` and `$1` fell through |
| 123 | + as a placeholder, breaking ProxySQL's `set_parser_algorithm=3` path |
| 124 | + in `pgsql-set_parameter_validation_test-t` (case 169). First-char |
| 125 | + constraint preserved (`$<word>` at token start still emits |
| 126 | + `TK_ERROR`); MySQL behaviour unchanged. |
| 127 | + ([PR #43](https://github.com/ProxySQL/ParserSQL/pull/43)) |
| 128 | +- **v1.0.6 (2026-05-23)** — Hot-fix on top of v1.0.5: `parse_set()`'s |
| 129 | + PARTIAL → ERROR downgrade was too aggressive for multi-assignment |
| 130 | + SETs where one element is malformed alongside well-formed ones |
| 131 | + (`SET sql_mode='X', whatever=, autocommit=1`). v1.0.6 only |
| 132 | + downgrades when the parse produced no children at all, so the |
| 133 | + successful elements remain in the AST. |
| 134 | + ([PR #42](https://github.com/ProxySQL/ParserSQL/pull/42)) |
| 135 | +- **v1.0.5 (2026-05-23)** — Post-1.0.4 audit follow-up after a wider |
| 136 | + SET-form sweep. Adds three new node types |
| 137 | + (`NODE_SET_ROLE`, `NODE_SET_SESSION_AUTHORIZATION`, |
| 138 | + `NODE_SET_CONSTRAINTS`) so PG non-GUC SET forms are no longer |
| 139 | + misclassified as unknown GUC assignments; recognizes |
| 140 | + schema-qualified GUC names (`SET pg_catalog.search_path`, |
| 141 | + `SET myapp.setting`); preserves the full `SET TIME ZONE INTERVAL |
| 142 | + '1' HOUR` literal (was dropping value + unit); and surfaces |
| 143 | + clearly-malformed SET inputs (`SET = 1`, `SET x = ;`, `SET x =`, |
| 144 | + `SET x = ,foo`, `SET search_path TO`, bare `SET`, `SET GLOBAL`) |
| 145 | + as `ParseResult::ERROR` instead of `PARTIAL`. |
| 146 | + ([PR #40](https://github.com/ProxySQL/ParserSQL/pull/40)) |
| 147 | +- **v1.0.4 (2026-05-23)** — Six fixes from the initial |
| 148 | + `set_parser_algorithm=3` audit: backtick-delimited `@@\`var\`` |
| 149 | + identifiers, PG `SET SCHEMA` / `SET SEED` shorthands, identifier |
| 150 | + quote-style flag (`FLAG_IDENT_DELIMITED`), bare PG `$word` → |
| 151 | + `ERROR`. |
| 152 | + ([PR #39](https://github.com/ProxySQL/ParserSQL/pull/39)) |
| 153 | +- **v1.0.3** — PG `SET TIME ZONE` alias + PG multi-value list |
| 154 | + (`SET search_path TO 'a', 'b'`). |
| 155 | + ([PR #38](https://github.com/ProxySQL/ParserSQL/pull/38)) |
0 commit comments