fix(es/parser): Keep recoverable errors from speculative TypeScript parses - #12137
fix(es/parser): Keep recoverable errors from speculative TypeScript parses#12137David Sherret (dsherret) wants to merge 2 commits into
Conversation
…arses
`try_parse_ts` set `Context::IgnoreError` for the whole speculative parse
and only restored it afterwards, so every recoverable error emitted while
building the AST was dropped even when the parse succeeded and its AST was
kept. `export namespace Foo {` with no closing brace therefore parsed
without a single diagnostic, while `namespace Foo {` correctly reported
`Expected '}'`.
`LexerCheckpoint` now records the length of both error buffers and
truncates them in `checkpoint_load`, the same way `CapturingCheckpoint`
rolls back captured tokens. With rollback available, `try_parse_ts` and
`try_parse_ts_bool` no longer suppress errors: a discarded speculative
parse discards its errors along with the rest of the checkpoint, while a
kept one keeps them.
Flow still suppresses, because it reports `_` in a type argument list as
an error even where Flow allows it and un-suppressing surfaces those
false positives.
`parse_ident` no longer reports strict mode reserved words inside a type
position. Parameter names of a function type are not bindings, so
`(package: Package) => void` is valid; this was already reported for
non-exported interfaces and was only hidden for exported ones by the
suppression removed above.
🦋 Changeset detectedLatest commit: 9eba6f1 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Disclosure: All of this PR is AI generated and the above. I opened as a draft PR because I have yet to review, but feel free to use it as a starting point and take it over. |
Merging this PR will improve performance by 2.41%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | es/lints/libs/three |
50.3 ms | 49.1 ms | +2.41% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing dsherret:fix/try-parse-ts-error-rollback (9eba6f1) with main (3802924)
Footnotes
-
61 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
|
The overall direction makes sense to me: diagnostics from a successful speculative parse should be preserved, while diagnostics from a discarded parse should be rolled back. However, making every Could we scope diagnostic rollback to the speculative operation that owns those diagnostics, or otherwise make the commit/rollback semantics explicit for nested checkpoints? |
Description:
Opening this as a draft because there are two design decisions in here I am not happy with and would like your call on before it goes any further. Those are up front, below the root cause.
Root cause
try_parse_tssetsContext::IgnoreErrorfor the duration of a speculative parse and only restores it after the closure returns. Suppressing errors is correct on the failure path — the parse is thrown away, so its errors should go too. It is wrong on the success path: the parse is kept, but every recoverable error emitted while producing it has already been dropped on the floor.The concrete symptom:
yields zero diagnostics, while the non-
exportformcorrectly yields
Expected '}', got '<eof>'. Theexportform routes throughtry_parse_ts; the plain form does not.export module Foo {is affected the same way.This fix makes errors roll back with the checkpoint rather than being suppressed:
lexer/state.rs—LexerCheckpointgainserrors_len/module_errors_len, andcheckpoint_loadtruncates both. This mirrors the existingCapturingCheckpointpattern inlexer/capturing.rs, which already truncates its captured-token buffer on rollback. Both error buffers are append-only untiltake_errors, so truncating to a saved length is a sound rollback.parser/typescript.rs—try_parse_tsandtry_parse_ts_boolno longer setContext::IgnoreError.ts_look_aheadkeeps the suppression, since it always reloads the checkpoint and the errors would be discarded regardless; suppressing there just avoids the wasted work.Motivating downstream bug
dprint-plugin-typescript panics in
gen_membered_bodyon this input, because its syntax-error gate has nothing to filter — swc reports no error at all, so the formatter proceeds on a truncated AST: dprint/dprint-plugin-typescript#802Open question 1: the Flow carve-out (this is a smell)
should_ignore_error_while_backtracking()currently returnsself.syntax().flow(). TypeScript gets the corrected behavior on every speculative path; Flow keeps the old blanket suppression. A syntax-conditional error policy is not something I would normally want to add.It is there because un-suppressing for Flow breaks exactly 7 hermes parity fixtures:
All 7 are the same issue:
_is inis_flow_reserved_type_name, andparse_ts_type_refreports TS1003 for it in every type-argument list. In Flow,_is legal as an implicit type argument in call andnewpositions (test<_>()), but illegal in an annotation — and the corpus fixturetypes/annotations/underscore_is_reserved_elsewhere.js(var x: Generic<_> = 3;) explicitly expects the error. So it cannot simply be allowed everywhere; the two positions genuinely need to be distinguished.Distinguishing them wants a new context flag, and
Contextis abitflags u32with all 32 bits already allocated (bit 31 isDisallowFlowAnonFnType). Widening tou64in a hot, frequently-copied parser struct felt like a decision for you rather than for me, so I left the carve-out in and am raising it here instead of hiding it.Happy to split this into two PRs if you prefer, and I think that may well be the better shape:
errors_len/module_errors_lenonLexerCheckpoint. WithIgnoreErrorstill set bytry_parse_ts, this is a pure mechanism change with no observable behavior change at all._handling can distinguish call position from annotation position.That would let (a) land uncontroversially and keep the Flow question isolated in (b). Say the word and I will restructure.
Open question 2: the
Context::InTypegate inparser/ident.rsparse_identno longer emitsInvalidIdentInStrictwhenContext::InTypeis set. On its face this is an unrelated behavior change bundled into an error-plumbing fix, and I want to flag it rather than let it slide through.In fairness to it, two things are true:
It is a real bug fix on its own. A parameter name in a function type is not a binding, so the strict-mode reserved-word restriction does not apply. On
maintoday,type A = (package: string) => void;wrongly reportsInvalidIdentInStrict, and so doesinterface I { m(package: string): void }— while theexported formexport type A = (package: string) => void;reports nothing, because the speculative path swallows it. Somainis currently inconsistent with itself here, depending on whetherexportis present.It is not actually separable from this change. Un-suppressing errors without it regresses Parser error for callback signature with "package" as parameter name #7186: with the
ident.rshunk reverted and the rest of this PR in place,spec_tests__typescript__issue_7186__1__input_tsandspec_tests__typescript__issue_7186__2__input_tsxboth fail. Those are the fixtures added by fix(es/parser): Allow usingpackageas a parameter name in TS interface #7438, and they only pass onmaintoday becauseexport interfaceroutes throughtry_parse_tsand the error is suppressed. Removing the suppression exposes it.So it is load-bearing rather than opportunistic — but the gate is written broadly (all of
InType), and if you would rather see it as its own PR with its own fixtures, landing first, I am glad to split it out; this PR would then rebase on top of it.Verified that it does not over-suppress real bindings — these all still report
InvalidIdentInStrictafter the change:function f(package) {},function f(package: string) {},var package = 1;,class C { constructor(private package: string) {} }.Test results (as actually measured, on Windows):
cargo test -p swc_ecma_parser --no-fail-fast(default features):tests/typescript.rstargetmain(2f3e88d)cargo test -p swc_ecma_parser --features verify,flow --no-fail-fast:tests/typescript.rstargetmain(2f3e88d)The +1 in each case is the new regression fixture. No test fails on this branch that does not also fail on
mainat the same commit — I ran the baseline explicitly rather than assuming it.The failures, all pre-existing and identical on
main:test262::identity(both configs) andtest262::error(verify,flow) — I do not have thetest262-parsersubmodule checked out locally, so the test262 suite could not run at all; it fails on a missing directory. Unverified by me either way.errors_tests__typescript_errors__type_only_import_specifier__invalid_type_only__input_ts(default features only) —import { type import } from 'mod'should fail but parses. Present identically onmain; behavior is unchanged by this PR.New regression fixture
tests/typescript-errors/export-namespace-unclosed/: confirmed it fails onmain("should fail, but parsed as ...") and passes with this change.No existing expected-output files were modified —
git diff upstream/main --statis 6 files, and both fixture files are additions.Not measured: I scoped builds to
-p swc_ecma_parser. Downstream crates (swc,swc_ecma_transforms, …) were not built or tested, so any fixture churn there is unknown. Since this PR makes the parser surface more diagnostics on TS speculative paths, downstream suites that assert on error output are the most plausible place for fallout, and CI will be a better judge than my local run.AI assistance disclosure: this change was developed with AI assistance. I have reviewed the diff and the test results, and independently re-ran both the baseline and the branch rather than taking the reported numbers on trust.
BREAKING CHANGE:
None in API terms. It is a behavior change: TypeScript sources that route through
try_parse_tsand contain recoverable errors will now surface diagnostics that were previously silently dropped. That is the point of the fix, but consumers gating on "did the parser report anything" may see newly-reported errors on input they previously got silence for.Related issue (if exists):
packageas a parameter name in TS interface #7438 (see open question 2)