Skip to content

Commit 6a60e73

Browse files
committed
TextMate fidelity: fix operator longest-match, split relational, scope enum members
Three exact-scope drop-in gaps surfaced by scope-coverage's new family-only dump, two of them real tokenization bugs the family-level bench could not see (comparison and assignment share the `keyword` family): 1. operator-overrides was emitted as `(group1_all)|(group2_all)|…`, one alternation per scope group. A regex alternation is ordered-first-match, not longest-match, so a short operator in an earlier group shadowed a longer one in a later group: `=` (assignment) matched the first character of `===` (comparison), tokenizing `===` as three `=`. Rebuilt as ONE global length-descending alternation with a capture per operator → `===` before `=`, `**=` before `**` before `*`, across all groups. 2. `< > <= >=` are RELATIONAL, not comparison (the convention the official grammar uses; `== != === !==` stay comparison). `<=`/`>=` had no scope override at all and fell to the bare `keyword.operator` catch-all. Added a `keyword.operator.relational` group (js/ts/tsx) and renamed the bare-`<>` fallback + the multiline-generic `>` end-capture to it. Driven by the grammar scope map — no hardcoded operator list in the generator. 3. enum members now scope `variable.other.enummember` (the convention paralleling the `storage.type.enum` keyword → `entity.name.type.enum` name remap). An enum gets its own `#enum-body` region — keyed on the `*.enum` scope subtype, not the word "enum" — whose member matcher captures an identifier in member position (after `{`/`,`, before `=`/`,`/`}`), excluding initializer-value identifiers. Object literals and class bodies are untouched (the region is enum-only). Exact-scope drop-in fidelity 71.7% → 74.5%, family-only 19.3% → 16.6%. All gates held: per-bug strict 100% (37/37, beats official 97.3%), family chart 87.8%, test-issues 318/0 (the two `<` cases now assert relational, matching official), JS 92.6%, sanity 15/15, agnostic 5/5, TSX 22/22. The tree-sitter @operator list only reorders (it does not split relational/comparison) — semantically identical. scope-coverage now dumps the per-shape family-only + divergent fix targets so the remaining gaps are explicit.
1 parent f10ddb6 commit 6a60e73

12 files changed

Lines changed: 486 additions & 78 deletions

File tree

examples/javascript.tmLanguage.json

Lines changed: 113 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1519,36 +1519,141 @@
15191519
"name": "storage.type.function.arrow.js"
15201520
},
15211521
"operator-overrides": {
1522-
"match": "(>>>=|\\*\\*=|<<=|>>=|\\?\\?=|\\|\\|=|&&=|\\+=|-=|\\*=|/=|%=|&=|\\|=|\\^=|=)|(===|!==|==|!=)|(>>>|<<|>>)|(\\?\\?|\\|\\||&&)|(\\+\\+|--)|(\\*\\*|\\*|\\+|-|/|%)|(\\||\\^|&)|(!|~)",
1522+
"match": "(>>>=)|(\\*\\*=)|(<<=)|(>>=)|(\\?\\?=)|(\\|\\|=)|(&&=)|(===)|(!==)|(>>>)|(\\+=)|(-=)|(\\*=)|(/=)|(%=)|(&=)|(\\|=)|(\\^=)|(\\?\\?)|(\\|\\|)|(&&)|(==)|(!=)|(<=)|(>=)|(<<)|(>>)|(\\*\\*)|(\\+\\+)|(--)|(\\*)|(=)|(\\|)|(\\^)|(&)|(<)|(>)|(\\+)|(-)|(/)|(%)|(!)|(~)",
15231523
"captures": {
15241524
"1": {
15251525
"name": "keyword.operator.assignment.js"
15261526
},
15271527
"2": {
1528-
"name": "keyword.operator.comparison.js"
1528+
"name": "keyword.operator.assignment.js"
15291529
},
15301530
"3": {
1531-
"name": "keyword.operator.bitwise.shift.js"
1531+
"name": "keyword.operator.assignment.js"
15321532
},
15331533
"4": {
1534-
"name": "keyword.operator.logical.js"
1534+
"name": "keyword.operator.assignment.js"
15351535
},
15361536
"5": {
1537-
"name": "keyword.operator.increment-decrement.js"
1537+
"name": "keyword.operator.assignment.js"
15381538
},
15391539
"6": {
1540-
"name": "keyword.operator.arithmetic.js"
1540+
"name": "keyword.operator.assignment.js"
15411541
},
15421542
"7": {
1543-
"name": "keyword.operator.bitwise.js"
1543+
"name": "keyword.operator.assignment.js"
15441544
},
15451545
"8": {
1546+
"name": "keyword.operator.comparison.js"
1547+
},
1548+
"9": {
1549+
"name": "keyword.operator.comparison.js"
1550+
},
1551+
"10": {
1552+
"name": "keyword.operator.bitwise.shift.js"
1553+
},
1554+
"11": {
1555+
"name": "keyword.operator.assignment.js"
1556+
},
1557+
"12": {
1558+
"name": "keyword.operator.assignment.js"
1559+
},
1560+
"13": {
1561+
"name": "keyword.operator.assignment.js"
1562+
},
1563+
"14": {
1564+
"name": "keyword.operator.assignment.js"
1565+
},
1566+
"15": {
1567+
"name": "keyword.operator.assignment.js"
1568+
},
1569+
"16": {
1570+
"name": "keyword.operator.assignment.js"
1571+
},
1572+
"17": {
1573+
"name": "keyword.operator.assignment.js"
1574+
},
1575+
"18": {
1576+
"name": "keyword.operator.assignment.js"
1577+
},
1578+
"19": {
1579+
"name": "keyword.operator.logical.js"
1580+
},
1581+
"20": {
1582+
"name": "keyword.operator.logical.js"
1583+
},
1584+
"21": {
1585+
"name": "keyword.operator.logical.js"
1586+
},
1587+
"22": {
1588+
"name": "keyword.operator.comparison.js"
1589+
},
1590+
"23": {
1591+
"name": "keyword.operator.comparison.js"
1592+
},
1593+
"24": {
1594+
"name": "keyword.operator.relational.js"
1595+
},
1596+
"25": {
1597+
"name": "keyword.operator.relational.js"
1598+
},
1599+
"26": {
1600+
"name": "keyword.operator.bitwise.shift.js"
1601+
},
1602+
"27": {
1603+
"name": "keyword.operator.bitwise.shift.js"
1604+
},
1605+
"28": {
1606+
"name": "keyword.operator.arithmetic.js"
1607+
},
1608+
"29": {
1609+
"name": "keyword.operator.increment-decrement.js"
1610+
},
1611+
"30": {
1612+
"name": "keyword.operator.increment-decrement.js"
1613+
},
1614+
"31": {
1615+
"name": "keyword.operator.arithmetic.js"
1616+
},
1617+
"32": {
1618+
"name": "keyword.operator.assignment.js"
1619+
},
1620+
"33": {
1621+
"name": "keyword.operator.bitwise.js"
1622+
},
1623+
"34": {
1624+
"name": "keyword.operator.bitwise.js"
1625+
},
1626+
"35": {
1627+
"name": "keyword.operator.bitwise.js"
1628+
},
1629+
"36": {
1630+
"name": "keyword.operator.relational.js"
1631+
},
1632+
"37": {
1633+
"name": "keyword.operator.relational.js"
1634+
},
1635+
"38": {
1636+
"name": "keyword.operator.arithmetic.js"
1637+
},
1638+
"39": {
1639+
"name": "keyword.operator.arithmetic.js"
1640+
},
1641+
"40": {
1642+
"name": "keyword.operator.arithmetic.js"
1643+
},
1644+
"41": {
1645+
"name": "keyword.operator.arithmetic.js"
1646+
},
1647+
"42": {
1648+
"name": "keyword.operator.logical.prefix.js"
1649+
},
1650+
"43": {
15461651
"name": "keyword.operator.logical.prefix.js"
15471652
}
15481653
}
15491654
},
15501655
"operators": {
1551-
"match": "\\.\\.\\.|<=|>=|<|>",
1656+
"match": "\\.\\.\\.",
15521657
"name": "keyword.operator.js"
15531658
},
15541659
"scope-punctuation-bracket-round": {

examples/javascript.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -446,6 +446,7 @@ export const jsScopes = {
446446
'keyword.operator.expression': ['instanceof', 'new', 'delete', 'void', 'typeof'],
447447
'keyword.operator.assignment': ['=', '+=', '-=', '*=', '/=', '%=', '**=', '<<=', '>>=', '>>>=', '&=', '|=', '^=', '??=', '||=', '&&='],
448448
'keyword.operator.comparison': ['==', '!=', '===', '!=='],
449+
'keyword.operator.relational': ['<', '>', '<=', '>='],
449450
'keyword.operator.logical': ['||', '&&', '??'],
450451
'keyword.operator.arithmetic': ['+', '-', '*', '/', '%', '**'],
451452
'keyword.operator.increment-decrement': ['++', '--'],

examples/tree-sitter/javascript/queries/highlights.scm

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -88,10 +88,10 @@
8888
[
8989
">>>=" "**=" "<<=" ">>=" "??=" "||=" "&&=" "==="
9090
"!==" ">>>" "+=" "-=" "*=" "/=" "%=" "&="
91-
"|=" "^=" "==" "!=" "||" "&&" "??" "**"
92-
"++" "--" "<<" ">>" "<=" ">=" "=" "+"
93-
"-" "*" "/" "%" "!" "~" "|" "&"
94-
"^" "<" ">"
91+
"|=" "^=" "==" "!=" "<=" ">=" "||" "&&"
92+
"??" "**" "++" "--" "<<" ">>" "=" "<"
93+
">" "+" "-" "*" "/" "%" "!" "~"
94+
"|" "&" "^"
9595
] @operator
9696
[
9797
"(" ")" "{" "}" "[" "]"

examples/tree-sitter/tsx/queries/highlights.scm

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -105,10 +105,10 @@
105105
[
106106
">>>=" "**=" "<<=" ">>=" "??=" "||=" "&&=" "==="
107107
"!==" ">>>" "+=" "-=" "*=" "/=" "%=" "&="
108-
"|=" "^=" "==" "!=" "||" "&&" "??" "**"
109-
"++" "--" "<<" ">>" "<=" ">=" "=" "+"
110-
"-" "*" "/" "%" "!" "~" "|" "&"
111-
"^" "<" ">"
108+
"|=" "^=" "==" "!=" "<=" ">=" "||" "&&"
109+
"??" "**" "++" "--" "<<" ">>" "=" "<"
110+
">" "+" "-" "*" "/" "%" "!" "~"
111+
"|" "&" "^"
112112
] @operator
113113
[
114114
"(" ")" "{" "}" "[" "]"

examples/tree-sitter/typescript/queries/highlights.scm

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -104,10 +104,10 @@
104104
[
105105
">>>=" "**=" "<<=" ">>=" "??=" "||=" "&&=" "==="
106106
"!==" ">>>" "+=" "-=" "*=" "/=" "%=" "&="
107-
"|=" "^=" "==" "!=" "||" "&&" "??" "**"
108-
"++" "--" "<<" ">>" "<=" ">=" "=" "+"
109-
"-" "*" "/" "%" "!" "~" "|" "&"
110-
"^" "<" ">"
107+
"|=" "^=" "==" "!=" "<=" ">=" "||" "&&"
108+
"??" "**" "++" "--" "<<" ">>" "=" "<"
109+
">" "+" "-" "*" "/" "%" "!" "~"
110+
"|" "&" "^"
111111
] @operator
112112
[
113113
"(" ")" "{" "}" "[" "]"

0 commit comments

Comments
 (0)