Skip to content

Fix keyword tokens matching identifier prefixes (Channel.fromPath mis-parse)#22

Merged
edmundmiller merged 53 commits into
ci-bump-tree-sitter-clifrom
fix-keyword-prefix-tokenization
Jul 7, 2026
Merged

Fix keyword tokens matching identifier prefixes (Channel.fromPath mis-parse)#22
edmundmiller merged 53 commits into
ci-bump-tree-sitter-clifrom
fix-keyword-prefix-tokenization

Conversation

@edmundmiller

@edmundmiller edmundmiller commented Jul 2, 2026

Copy link
Copy Markdown
Member

Problem

Channel.fromPath(...) mis-parses: the lexer tokenizes the Channel.from prefix of fromPath via the channel_from rule, producing ERROR nodes. This breaks structural matching in ast-grep (e.g. nf-core lint rules matching $CH.ifEmpty(null) against Channel.fromPath(x).ifEmpty(null)).

Fix

  1. word: $ => $.identifier — enables tree-sitter keyword extraction so 'from'/'of'/'value' only match complete identifiers, never prefixes.
  2. Generic channel_factory rule — Channel.<anyFactory>(args) catch-all so fromPath, fromFilePairs, empty, watchPath, etc. parse cleanly without enumerating each one. The dedicated rules still win for their exact names via keyword extraction.
  3. channel_expression as method_call receiver — chains like Channel.fromPath(x).ifEmpty(null) now parse as a clean method_call.

Requires tree-sitter CLI ≥ 0.25 (older lexers prefer extracted keywords over longer identifier matches — verified broken on 0.24.7, working on 0.26.10). Runtime pins bumped to 0.25 for ABI 15.

Also fixed (unmasked by the corpus fix)

tree-sitter test previously aborted at the stale corpus expectation, hiding:

  • queries/tags.scm / queries/highlights.scm referenced fields (name:, method:) and node types (function_declaration, comment, for) the grammar never defined — rewritten against the actual node inventory
  • tree-sitter.json file-types had leading dots so .nf was never associated with the grammar

⚠️ Maintainer action needed

  • CI workflow pin (couldn't push: token lacks workflow scope) — .github/workflows/ast-grep-distribution.yml line 59 must change npm install -g tree-sitter-cli@^0.24.5 → @^0.26.10, otherwise CI's tree-sitter generate step regenerates a parser with the old broken lexer behavior.
  • linux-x64 lib — only lib/macos-arm64/libnextflow.dylib is rebuilt here; grab the linux artifact from CI after the pin bump.

Known remaining (pre-existing, out of scope)

test/highlight/*.nf assertions spec features the grammar doesn't have (def function declarations, for loops, named fields, @property highlighting) — they never ran before this PR and still fail. Suggest tracking grammar fields + richer highlighting as a follow-up issue.

Verification

  • Corpus: 61/61 pass (was 59/61 + suite abort)
  • Channel.fromPath(params.input).ifEmpty(null) parses ERROR-free; ast-grep pattern $CH.ifEmpty(null) matches it (verified via ast-grep-py 0.44 loading the rebuilt dylib)
  • nf-core/tools structural lint tests pass against the rebuilt parser

🤖 Generated with Claude Code

https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD

edmundmiller and others added 4 commits July 2, 2026 17:28
@edmundmiller @claude
build: bump tree-sitter CLI to 0.26, runtimes to 0.25
a0ce85e 
Keyword extraction (word rule) requires the 0.25+ lexer, which prefers
longer identifier matches over extracted keywords. ABI 15 parsers need
runtime >= 0.25 in the node/rust/python bindings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
fix: keyword tokens no longer match identifier prefixes (Channel.from…
d33ebbd 
…Path)

- Add word rule so 'from'/'of'/'value' only match complete identifiers;
  previously Channel.fromPath tokenized as 'from' + ERROR
- Add generic channel_factory rule for factories without dedicated
  rules (fromPath, fromFilePairs, empty, watchPath, ...)
- Allow channel_expression as method_call receiver so chains like
  Channel.fromPath(x).ifEmpty(null) parse cleanly
- Unskip Channel factories corpus test; refresh stale expectations
  (closure_block) that predated the committed grammar
- Rebuild macos-arm64 dylib (linux-x64 needs a CI rebuild)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
fix: repair query files and file-types so tree-sitter test runs
1f21414 
Previously masked by the corpus failure aborting the suite:
- tags.scm/highlights.scm referenced fields and node types the grammar
  never defined; rewrite against the actual node inventory
- tree-sitter.json file-types had leading dots, so .nf files were never
  associated with the grammar

Highlight assertion files in test/highlight/ remain aspirational
(def functions, for loops, fields) and still fail; tracked as follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
build: export tree_sitter_nextflow symbol from the python binding
d8a9afd 
setuptools' TREE_SITTER_HIDE_SYMBOLS kept the parser function out of the
macOS export trie (visible in nm but not dlsym-able). Dropping the macro
lets the installed wheel double as an ast-grep customLanguages library:
register its _binding shared object directly, no separate dylib needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller edmundmiller changed the base branch from main to ci-bump-tree-sitter-cli July 2, 2026 23:23
@edmundmiller edmundmiller mentioned this pull request Jul 2, 2026
Merged
edmundmiller and others added 24 commits July 2, 2026 18:39
@edmundmiller @claude
fix: allow function calls and bare identifiers as pipe operations
419a270 
ch | ifEmpty(null) and ch | view previously produced ERROR nodes since
pipe_operation only accepted map closures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
feat: parse real nf-core module files without errors
826f949 
samtools/sort main.nf went from 75 ERROR nodes (rest-of-file swallowed
after the container directive) to 0. Added:

- process directives: tag/label/conda/container/publishDir/cpus/...
  (generic identifier + args rule)
- ternary expressions (right-assoc, so nested chains work), elvis '?:',
  unary '!', 'in' and 'instanceof' operators, index access x[0],
  property access on non-identifier receivers ((expr).name)
- method_call/function_call/list as binary operands
- Groovy statement prelude in script/shell/exec/stub sections before
  the script string; interpolated strings as script_content
- tuple input/output declarations with named options (emit:, optional:)
- method_call inlines its navigation path (a.b.c(...)) instead of a
  reduced dotted_identifier receiver, fixing 3+-segment dotted chains
  (task.ext.when) that previously forced an early reduce and ERROR'd

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
docs: strict-syntax parity roadmap + parse-rate harness
b4b84cc 
parse_rate.py measures the share of real nf-core/modules files that
parse without ERROR nodes and clusters the failures by construct.
Baseline after the grammar expansion: 50.0% (1035/2070). The ranked
failure list drives Phase 1 of ROADMAP.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
feat: parse floats, string escapes, assert-message, ==~, emit options
300cfc6 
Phase 1 grammar burn-down (ROADMAP.md), 50.0% -> 66.4% parse rate on
nf-core/modules:

- float_literal (0.8, 1.5e3); floats/maps as binary operands
- single- and double-quoted string escape sequences incl. $ and \'
- assert cond : message (also allowed in script-section preludes)
- ==~ exact-match regex operator
- interpolated map keys ("${task.process}": ...)
- braceless if bodies: if (cond) error "msg"
- chained subscripts m[0][2]
- emit_declaration: path "x", emit: y, topic: z (single-qualifier
  outputs with named options); option_entry/option_value give
  declaration options a value type that excludes command_expression

Known remaining: multi-line declaration sequences where a bare-identifier
option value abuts the next line's qualifier need newline-sensitive
termination (external scanner) — tracked for Phase 1 follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
docs: record 66.4% parse rate; scanner is next Phase 1 item
6ee836b 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
ci: PyPI release workflow (dormant until first release)
caf969e 
cibuildwheel abi3 wheels for macOS+Linux (x64/arm64) + sdist, trusted
publishing on release. Includes a macOS export-trie check that the
tree_sitter_nextflow symbol is present so the wheel stays usable as an
ast-grep dynamic language library. Does nothing until a GitHub release
is published and PyPI trusted publishing is configured.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
feat: newline-sensitive statement termination via external scanner
e8ae7ee 
66.4% -> 76.1% parse rate on nf-core/modules.

src/scanner.c emits a _terminator token at newlines/';' only where the
parser state accepts one (valid_symbols), so line continuations inside
brackets and mid-expression need no special handling. A one-char
lookahead suppresses the terminator before continuation tokens (. ? : ,
closing brackets, and the 'else' keyword); comment lines are folded into
the terminator run so a comment between statements yields one terminator.

Process bodies now parse in three ordered phases (directives ->
input/output/when -> script/stub) and each declaration self-terminates,
removing the directive-vs-input and directive-vs-prelude-assignment
ambiguities the terminator exposed. Unblocks multi-line inputs,
multi-statement script preludes, and script+stub processes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
fix: preserve comment nodes at statement boundaries
ccb73c3 
The scanner previously folded comments into the terminator token, which
removed line_comment/block_comment nodes from the tree and broke
comment-based lint rules (TODO detection missed TODOs in script
sections). Now the scanner returns false when a comment follows the
newline run, so the comment is lexed as an ordinary extra and the
terminator is emitted after it — one terminator between statements, and
the comment survives as a node.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
feat: allow " inside triple-quoted GStrings
c2fd751 
76.1% -> 77.2% parse rate. interpolated_triple_quoted_string used
string_content (/[^$"\\]+/), which stopped at any double quote, so a
YAML-style versions block inside a script string broke:
    """
    "${task.process}":
        tool: ...
    """
New triple_string_content token only stops at the closing """, $ and \\,
so lone/doubled quotes are ordinary content.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
@edmundmiller @claude
feat: parse slashy-string regex operands and // inside strings
4d06672 
Two tokenization fixes for real-world process scripts:

- slashy_string is now a single token() (was a 3-part seq whose / and
  body collided with the division operator and escape tokens) and is a
  valid right operand of binary_expression, so `x ==~ /re/` and
  `m = s =~ /re/` parse. First body char excludes * and / so /*...*/
  and // comments still win.
- string_content is token(prec(1, ...)) so a // inside a double-quoted
  string (e.g. sed 's/.$//') is string content, not a line_comment
  extra that ate the closing quote and ran off the end of the file.

Parse rate over nf-core/modules: 77.2% -> 79.4% (1598 -> 1643 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
chore: sync package-lock to tree-sitter-cli 0.26.10
01c13a3 
Lockfile still pinned 0.24.6 after package.json moved to ^0.26.10.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: parse named arguments in function calls
31fbf5a 
path(reads, arity: '1..*', stageAs: 'input/*') and other Groovy
map-style named args (key: value) now parse: function_call arguments
accept option_entry, not just simple_expression.

Parse rate over nf-core/modules: 79.4% -> 84.1% (1643 -> 1740 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: compound assignment, !in/!instanceof, unary ~/-/+, << operators
18628b2 
- assignment accepts +=, -=, *=, /=, %=, **=, <<=, >>=, &=, |=, ^=, ?=
  and dotted/index/property LHS (args += ..., meta.x = ...)
- binary operators add <=>, <<, !in, !instanceof
- unary_expression adds ~ (Groovy bitwiseNegate -> Pattern, e.g. ~/re/),
  and numeric sign -/+

Bitwise & | ^ >> >>> intentionally omitted: | collides with the channel
pipe operator.

Parse rate over nf-core/modules: 84.1% -> 86.5% (1740 -> 1791 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: dynamic directive closures, input named options, property access
1b62a6f 
- directive accepts a closure body: memory { 7.GB * task.attempt }
- input_declaration accepts a qualifier with named options:
  path ref, name: 'ref/*' (emit_declaration, previously output-only)
- property_expression is now a simple_expression and a binary operand,
  and accepts an integer_literal receiver, so memory-unit literals
  7.GB / 280.MB and (expr).prop chains parse everywhere

Parse rate over nf-core/modules: 86.5% -> 87.6% (1791 -> 1813 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: cast (as) and constructor (new) expressions
dbea8b4 
def n = (task.cpus * 0.9) as int
def y = new groovy.yaml.YamlBuilder()

Parse rate over nf-core/modules: 87.6% -> 88.2% (1813 -> 1825 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: template directive, top-level function definitions, return
7d3a54f 
- template 'main.R' as a script section body (external script file)
- def name(params) { ... } function definitions at top level, with
  typed params, param defaults, and optional -> return type
- return / return expr statements in functions, closures, script
  preludes (previously return e parsed as two bogus statements)

Parse rate over nf-core/modules: 88.2% -> 90.2% (1825 -> 1868 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: if statements in workflow main/body
c98a604 
Conditional logic (if/else) in workflow main: and body sections, common
in subworkflows.

Parse rate over nf-core/modules: 90.2% -> 90.5% (1868 -> 1873 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: method-call named args, chained-call/constructor receivers, ?. …
b654535 
…and *.

- method_call arguments accept option_entry: ch.combine(x, by: 0)
- method_call receiver accepts function_call and constructor_call:
  foo().bar(), new JsonSlurper().parseText(t)
- navigation supports safe-navigation ?. and spread *. operators:
  task.ext.args?.contains(x), pairs*.join(' ')

Parse rate over nf-core/modules: 90.5% -> 91.3% (1873 -> 1890 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: assert in blocks/closures, bare tuple components, template()
5126f5f 
- assert_statement allowed in block and closure_block bodies
  (input.collect { f -> assert f.name != x : 'msg' })
- tuple components accept bare identifiers (stdout) and env('V')
- template('file') parenthesised form, alongside template 'file'

Tried and reverted (net regressions): multi-arg command_expression
(exit 1, 'msg') and triple-string method receivers ('''...'''.stripIndent())
both introduce ambiguity that breaks more files than they fix.

Parse rate over nf-core/modules: 91.3% -> 92.3% (1890 -> 1910 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: bitwise & / ^ operators and multiMap/branch label statements
694ae05 
- restore bitwise & and ^ as binary operators (ix & 1); | stays out
  because it collides with the channel pipe operator
- label_statement (name: expr) in closure blocks, used by multiMap and
  branch operators to name their emitted channels

Parse rate over nf-core/modules: 92.3% -> 92.4% (1910 -> 1913 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: method chains on process output channels
6694b8f 
PROCESS.out.channel.join(...).map { } — process_output is now a
method_call receiver (common in subworkflow channel plumbing).

Parse rate over nf-core/modules: 92.4% -> 93.3% (1913 -> 1931 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
fix: terminator-separate statements inside blocks and closures
8cca076 
A bare-RHS assignment on one line no longer absorbs the next line's
identifier as a command_expression: x = a / y = b were parsed as
x = (a y) = b (ERROR at the second =). block and closure_block now allow
an optional terminator between statements so the newline ends the first.

Parse rate over nf-core/modules: 93.3% -> 93.8% (1931 -> 1942 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
fix: terminator-separate statements in workflow sections
57bc153 
Consecutive assignments/statements in workflow main:/emit:/body no
longer merge across newlines (a = X.out / b = Y.out was parsed as one
broken assignment). workflow_body/main/emit and take: now allow a
terminator between items.

Known limitation: with terminators, an LALR reduction can place a
trailing take: identifier or a second main: statement at workflow_body
level rather than nested in the section. This is error-free (all nodes
present) and does not affect the parse-rate metric; corpus expectations
updated to match. A section-greedy restructure is tracked in ROADMAP.

Parse rate over nf-core/modules: 93.8% -> 96.1% (1942 -> 1990 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: quote abutting interpolation in triple-quoted GStrings
e2978b9 
"\${task.process}": inside """...""" now parses as a general expression
(assignment RHS, block statement), not only as a script body. A lone or
doubled " that triple_string_content cannot absorb (because it abuts an
interpolation) is matched by a low-precedence quote token; maximal munch
still prefers the 3-char """ closer.

Parse rate over nf-core/modules: 96.1% -> 96.5% (1990 -> 1998 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
edmundmiller and others added 25 commits July 2, 2026 23:55
@edmundmiller @claude
feat: exit statement (code, message) and env outputs with emit
fa92b72 
- exit 1, 'message' — two-arg exit statement in blocks, closures, and
  script preludes (targeted rule; avoids the ambiguity a general
  multi-arg command_expression introduced)
- env 'VER', emit: x — env qualifier accepted in emit_declaration, and
  env_input broadened to interpolated (double-quoted) strings

Parse rate over nf-core/modules: 96.5% -> 96.8% (1998 -> 2003 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: exit(code, message) parenthesised form
30514ad 
exit(1, 'msg') alongside exit 1, 'msg'. Making exit a keyword had broken
the paren (function-call) form.

Parse rate over nf-core/modules: 96.8% -> 97.0% (2003 -> 2008 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: bare ternary statement in script preludes
30f6a98 
cond ? a.each { } : b.each { } used for side effects now parses as a
script_statement.

Parse rate over nf-core/modules: 97.0% -> 97.1% (2008 -> 2011 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: Groovy trailing closure after args, and cast as binary operand
5140ae4 
- method(args) { closure } — trailing closure after parenthesised args
  (s.replaceAll(/re/) { m -> ... })
- cast_expression is now a binary operand: (x as Float) > 0, a ^ b casts

Parse rate over nf-core/modules: 97.1% -> 97.2% (2011 -> 2013 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: full-expression process-invocation args, property on call result
75e330e 
- process_invocation arguments accept any simple_expression and named
  args, not just identifiers: SAMTOOLS_SORT(ch, [[], []], ''),
  FOO(ch.map { }, x.first()), PROCESS(a, key: v)
- property_expression accepts a function_call receiver: file(x).baseName

Parse rate over nf-core/modules: 97.2% -> 97.8% (2013 -> 2024 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: no-paren call with closure, computed map keys, pipe continuation
e07f6c3 
- command_expression accepts a closure: multiMapCriteria { ... }
- map_entry key can be a parenthesized (computed) expression: [(key): v]
- scanner treats a leading | as a continuation so multi-line channel
  pipes parse: ch \n | combine(x) \n | map { }

Parse rate over nf-core/modules: 97.8% -> 98.0% (2024 -> 2028 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: safe-navigation in property chains, process output as binary op…
fe227b0 
…erand

- dotted_identifier supports ?. : task.ext?.args, logSettings?.x
- process_output is a binary operand: bwa_index ?: BWAMEM1_INDEX.out.index

Parse rate over nf-core/modules: 98.0% -> 98.1% (2028 -> 2030 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: script string followed by a template call
f968487 
Several template-based modules leave a dead """...""" before the
template '...' call. script_content now accepts a script string
optionally trailed by a template, so these parse (previously the
template was mis-read as a misplaced process directive).

Parse rate over nf-core/modules: 98.1% -> 98.6% (2030 -> 2041 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
docs: parse-rate journey to 98.6% and remaining-failure taxonomy
aa77f2c 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: try/catch/finally statements
0b62384 
Present in current nf-core lib code (deprecated by 26.04 strict syntax but
still shipped). try_statement in blocks, closures, and script preludes;
catch supports typed/untyped and multi-type parameters.

Parse rate over nf-core/modules: 98.6% -> 98.6% (2041 -> 2042 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: catch/finally on a new line after the try block
d0ebb60 
Scanner treats a leading else/catch/finally as a statement continuation,
so } \n catch (e) { ... } parses (matches the existing } \n else handling).

Parse rate over nf-core/modules: 98.6% -> 98.7% (2042 -> 2043 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: chained pipe operations (a | map | combine | view)
edde5ad 
pipe_expression is now left-recursive, so multi-stage channel pipes parse
(previously only a single | worked).

Parse rate over nf-core/modules: 98.7% -> 98.7% (2043 -> 2044 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: destructuring (multiple) assignment
2756911 
(meta, seq) = fn() — destructuring_pattern as an assignment target, not
just in def declarations.

Parse rate over nf-core/modules: 98.7% -> 98.8% (2044 -> 2046 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: bare binary-expression statement in script preludes
96dc385 
outputs << 'x' and similar side-effecting binary ops parse as a
script_statement (a bare string is not a binary, so no ambiguity with the
trailing script content).

Parse rate over nf-core/modules: 98.8% -> 98.9% (2046 -> 2047 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: index/subscript on parenthesized and property expressions
c4e8b4d 
(input =~ /re/)[0][1] and (expr).prop[i] now parse (index_expression
base accepts parenthesized_expression and property_expression), incl. as
a multi-line ternary consequence.

Parse rate over nf-core/modules: 98.9% -> 99.0% (2047 -> 2049 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: pipe operator with closure (multiMap/branch), index on interp s…
9e47b91 
…tring

- ch | multiMap { } / | branch { } / | filter { } (operator_closure in
  pipe_operation)
- "${x}"[range] index on an interpolated string

Parse rate over nf-core/modules: 99.0% -> 99.1% (2049 -> 2051 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: command with triple-string argument (error """...""")
f109c94 
Parse rate over nf-core/modules: 99.1% -> 99.1% (2051 -> 2052 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: typed closure parameters ({ Path p -> ... })
2e6e87f 
Closure params may carry a type (Path path, MemoryUnit memory), used by
JsonGenerator .addConverter calls and similar.

Parse rate over nf-core/modules: 99.1% (2052 / 2070; unblocks typed
params, file still gated on triple-string method calls).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
docs: update parse-rate journey to 99.1% and remaining taxonomy
9ee6cff 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: method calls on triple-quoted strings ("""...""".stripIndent())
db2f985 
Root fix for the largest residual cluster: '"""..."""' now always parses
as interpolated_triple_quoted_string (it already handles no-interpolation
content), removing the overlap with the plain triple_quoted_string atomic
token that made a triple-string method receiver ambiguous. plain
triple_quoted_string is now single-quote-only ('''...'''). A dedicated
string_method_call rule requires at least one .method(...) so a bare
triple-string script body is never mis-parsed as a method receiver.

Corpus expectations regenerated ("""...""" -> interpolated_triple_quoted_string);
all :skip tests preserved.

Parse rate over nf-core/modules: 99.1% -> 99.3% (2052 -> 2056 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
fix: lenient string escape sequences (\\ + any char)
5cf2ae4 
Groovy treats an unrecognised \x as literal backslash + x. Accept \ +
any character (bash line continuations \<newline>, \ , etc.) besides
\uXXXX, so triple-quoted script bodies with shell escapes don't ERROR
now that """...""" routes through interpolated_triple_quoted_string.

Parse rate over nf-core/modules: 99.3% -> 99.4% (2056 -> 2058 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
docs: parse-rate 99.4%, stripIndent cluster resolved, remaining taxonomy
204c968 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: tolerate whitespace in section markers (stub :)
59a3703 
Section markers are seq(word, ':') instead of atomic 'word:' tokens, so a
stray space before the colon parses.

Parse rate over nf-core/modules: 99.4% -> 99.5% (2058 -> 2059 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
feat: semicolon terminates before }, typed function definitions
90d9610 
- scanner: an explicit ; always terminates — even before } (return a?b:c;})
  and even when a comment follows (stmt; /* c */); a trailing same-line
  comment after ; is folded into the terminator so it is not a boundary
  extra that would end a script prelude early.
- function_definition accepts a return type instead of def:
  String getExt(String args) { ... } with Groovy-typed params (Type name).

Fixes the jvarkit vcf2table/vcffilterjdk/vcfpolyx custom-function modules.

Parse rate over nf-core/modules: 99.5% -> 99.6% (2059 -> 2062 / 2070).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller @claude
docs: parse-rate 99.6%; 8 remaining failures categorised
dc5bda3 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edmundmiller edmundmiller merged commit fe705b9 into ci-bump-tree-sitter-cli Jul 7, 2026
2 checks passed
@edmundmiller edmundmiller deleted the fix-keyword-prefix-tokenization branch July 7, 2026 18:50
edmundmiller added a commit that referenced this pull request Jul 7, 2026
@edmundmiller @claude
ci: bump tree-sitter-cli to ^0.26.10
6154267 
Keyword extraction (word rule, PR #22) needs the 0.25+ lexer; the CI
generate step would silently revert the Channel.fromPath tokenization
fix if it regenerates with 0.24.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaTroL9k3feq3MTAptE9bD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@edmundmiller