TM Grammar Test Tools

Generate, refresh, and run assertions for VSCode TextMate grammar tests directly in VS Code. Works out of the box in grammar packages, and in any test file once the needed grammar is available in VS Code or configured.

Generate Assertions

GIF

GIF fallback for GitHub, which doesn't render <video>. Link to mp4.

Run Tests

GIF

GIF fallback for GitHub, which doesn't render <video>. Link to mp4.

Quick Start

Open a syntax test file whose first line matches:

<comment token> SYNTAX TEST "<language scope>" "optional description"

Use CodeLens, Code Actions (lightbulb), or the Command Palette to run one of:
- Insert Assertions is the primary command. It automatically switches between line and range behavior based on the current cursor or selection. See Command Behavior for the exact rules.
- Insert Line Assertions safely generates or refreshes whole source line(s).
- Replace Line Assertions fully replaces an existing line assertion block.
- Insert Range Assertions generates assertions for the selected range or token at the cursor.
- ... (Full) / ... (Minimal) variants override tmGrammarTestTools.scopeMode for that invocation.
The extension loads grammars from installed VS Code contributions (when tmGrammarTestTools.autoLoadInstalledGrammars is enabled), the nearest or configured package.json, and optional provider output. It then tokenizes from the top of the syntax test up to each targeted source line and inserts or refreshes the assertion block under that line.

User-facing line and column numbers are 1-based unless explicitly noted otherwise.

Most commands also write context, timing, and grammar-loading details to the TM Grammar Test Tools Output panel.

Testing UI runs are subject to VS Code's own testing.saveBeforeTest behavior; see Testing UI.

You can bind keyboard shortcuts for all the extension commands.

Example keybindings.json snippet

[
  // Universal assertions: line-oriented for plain cursors / whole-line selections,
  // range-oriented for partial selections
  {
    "key": "ctrl+alt+l",
    "command": "tmGrammarTestTools.insertAssertions",
    "when": "editorTextFocus"
  },

  // Universal assertions: force full or minimal for this invocation
  {
    "key": "ctrl+alt+1",
    "command": "tmGrammarTestTools.insertAssertionsFull",
    "when": "editorTextFocus"
  },
  {
    "key": "ctrl+alt+2",
    "command": "tmGrammarTestTools.insertAssertionsMinimal",
    "when": "editorTextFocus"
  },
  {
    "key": "ctrl+alt+3",
    "command": "tmGrammarTestTools.insertAssertionsMinimal",
    "args": {
      "minimalHeaderScopeFactoring": "keepSharedHeader",
      "minimalTailScopeCount": 2
    },
    "when": "editorTextFocus"
  },

  // Explicit line assertions
  {
    "key": "ctrl+alt+shift+l",
    "command": "tmGrammarTestTools.insertLineAssertions",
    "when": "editorTextFocus"
  },
  {
    "key": "ctrl+alt+shift+1",
    "command": "tmGrammarTestTools.insertLineAssertionsFull",
    "when": "editorTextFocus"
  },
  {
    "key": "ctrl+alt+shift+2",
    "command": "tmGrammarTestTools.insertLineAssertionsMinimal",
    "when": "editorTextFocus"
  },

  // Replace line assertions
  {
    "key": "ctrl+alt+shift+r",
    "command": "tmGrammarTestTools.replaceLineAssertions",
    "when": "editorTextFocus"
  },
  {
    "key": "ctrl+alt+shift+3",
    "command": "tmGrammarTestTools.replaceLineAssertionsFull",
    "when": "editorTextFocus"
  },
  {
    "key": "ctrl+alt+shift+4",
    "command": "tmGrammarTestTools.replaceLineAssertionsMinimal",
    "when": "editorTextFocus"
  },

  // Explicit range assertions
  {
    "key": "ctrl+alt+;",
    "command": "tmGrammarTestTools.insertRangeAssertions",
    "when": "editorTextFocus"
  },
  {
    "key": "ctrl+alt+'",
    "command": "tmGrammarTestTools.insertRangeAssertionsFull",
    "when": "editorTextFocus"
  },
  {
    "key": "ctrl+alt+shift+'",
    "command": "tmGrammarTestTools.insertRangeAssertionsMinimal",
    "when": "editorTextFocus"
  }
]

Command Behavior

Existing assertion lines are skipped during tokenization so TextMate rule state is preserved across source lines.
First-column tokens are emitted with the <--/<~-- syntax when needed.
Insert Assertions is selection-intent aware:
- with a single plain cursor on a source line, it behaves like Insert Line Assertions
- with a whole-line selection, it also behaves like Insert Line Assertions
- with a partial selection, or with multiple cursors/selections touching the same source line, it behaves like Insert Range Assertions
- when multiple lines are touched, it resolves each source line independently
- CodeLens uses this command and scopes that auto behavior to the attached source line only, ignoring selections on other lines
Line assertion commands are line-oriented:
- with empty selection they target the line at cursor(s)
- with non-empty selection they target each touched non-blank source line top-to-bottom
- a non-empty selection made entirely of whitespace-only source lines is treated as intentional and command targets those lines too
Insert Line and Insert Range either insert new assertions or, for existing blocks, perform a safe refresh, meaning they don't touch assertion lines that contain negative assertions.
Range commands are selection- or token-range oriented:
- with a non-empty selection(s), they generate assertions for the selected characters
- with an empty selection(s), they resolve the token at the cursor position(s) and use that token as the range. For example, if the cursor is in the middle of quux;, it expands to the full quux token and emits assertions for it
- when only part of a line is selected and that line already has assertions, they insert new assertions into the existing block (instead of replacing it)
- when the whole line is selected, they behave like Line assertion commands
- that behavior is independent for each line touched by the selection(s) or cursor(s)
- Range commands skip blank or whitespace-only lines, unless selection is made entirely of whitespace-only lines.
(Minimal) command variants:
- may omit the header scope when it is shared by every token and there is at least one more specific scope to show.
- factor shared parent scopes so broader scopes are emitted once before narrower child scopes
- can either omit that shared header scope from the factored output or keep it as part of the shared factored prefix via tmGrammarTestTools.minimalHeaderScopeFactoring
- can retain either the last one or last two scopes on terminal token assertions via tmGrammarTestTools.minimalTailScopeCount
Replace Line commands always replace the whole assertion block for each targeted source line, so use with caution: they may wipe out negative assertions and weaken the test.

Code Actions and CodeLens expose the safe Insert commands. The potentially destructive Replace Line commands are available from the command palette.

Settings

tmGrammarTestTools.scopeMode can be full or minimal. The generic Line and Range commands use that setting. The explicit Full and Minimal commands override it for that invocation. Default is full.
- Insert Assertions, Insert Line Assertions, and Insert Range Assertions all follow this rule.
tmGrammarTestTools.minimalHeaderScopeFactoring controls whether minimal mode omits the shared syntax-test header scope from factored output or keeps it in the shared factored prefix. Allowed values are omitSharedHeader (default) and keepSharedHeader.
tmGrammarTestTools.minimalTailScopeCount defaults to 1 and, in minimal mode, keeps the last one or two scopes on terminal token assertions even when broader parent scopes were already factored out. Invalid values are clamped and logged as warnings.
- minimalHeaderScopeFactoring and minimalTailScopeCount command arguments can override these per invocation on any command whose effective scope mode is minimal, including all explicit ...Minimal commands. That is useful for custom keybindings such as binding Insert Assertions (Minimal) with { "minimalHeaderScopeFactoring": "keepSharedHeader", "minimalTailScopeCount": 2 }

Minimal mode examples

// "omitSharedHeader" (default)
  $foo;
# ^^^^ variable.other.php
# ^ punctuation.definition.variable.php

// "keepSharedHeader"
  $foo;
# ^^^^ source.php variable.other.php
# ^ punctuation.definition.variable.php

// "minimalTailScopeCount": 1 (default)
  /[a-zz]/
// ^^^^^^ string.regexp.js constant.other.character-class.set.regexp
// ^    ^ punctuation.definition.character-class.regexp
//  ^^^ constant.other.character-class.range.regexp

// "minimalTailScopeCount": 2
  /[a-zz]/
// ^^^^^^ string.regexp.js constant.other.character-class.set.regexp
// ^    ^ constant.other.character-class.set.regexp punctuation.definition.character-class.regexp
//  ^^^ constant.other.character-class.set.regexp constant.other.character-class.range.regexp

tmGrammarTestTools.compactRanges defaults to true and merges disjoint caret ranges when they share the same rendered scope list and the tmgrammar assertion syntax can represent the merge.
tmGrammarTestTools.autoLoadInstalledGrammars defaults to true and controls whether installed VS Code grammars are loaded before local and provider grammars.
tmGrammarTestTools.enableCodeActions defaults to true and adds Code Actions for inserting assertions at the current cursor or selection, plus explicit line/range alternatives when useful.
tmGrammarTestTools.enableCodeLens defaults to true and adds source-line CodeLens commands that insert assertions for that line, switching between line and range behavior based on the current selection on that line.
tmGrammarTestTools.hideCodeLensOnCommentLines defaults to true and hides CodeLens on source lines that look like language comments according to the active language configuration, with the syntax-test header comment token used as a fallback.
tmGrammarTestTools.configPath points to the grammar package package.json when the nearest one is not the right source for the current syntax test.
tmGrammarTestTools.grammarProvider.* controls optional external grammar loading. See Grammar Provider.
tmGrammarTestTools.testDiscovery.include / exclude optionally add workspace files to the Testing view by glob. Matching files are treated as candidate syntax tests and validated lazily when expanded or run, so use reasonably narrow patterns.
Debugging: tmGrammarTestTools.logGrammarDetails defaults to false and, when enabled, logs detailed grammar selection info in the Output panel. Assertion generation logs the actually used grammar scopes with source labels; test runs log the merged grammar load order.

Testing UI

The extension integrates with VS Code’s native Testing UI.

Open syntax test files are discovered in the Testing view.
tmGrammarTestTools.testDiscovery.include can also add candidate syntax test files across the workspace. Those file items are validated lazily when expanded or run.
The extension creates one file item per discovered syntax test and one child item per source line that has an assertion block.
You can run a whole file or a single asserted source line from the Testing view or gutter.
Test execution currently uses the vscode-tmgrammar-test runner.
In trusted workspaces, the extension prefers a local vscode-tmgrammar-test dependency resolved from the active file's own project. For untitled drafts, it resolves from the effective workspace folder. If none is found, it falls back to the runner bundled with the extension.
In untrusted workspaces, the extension skips local runner loading and always uses the bundled runner.
If a nearby package.json declares vscode-tmgrammar-test but the dependency cannot be resolved, the extension warns and falls back to the bundled runner.
If a local vscode-tmgrammar-test package resolves but is unusable or incompatible, the test run fails explicitly against that local runner instead of silently switching to the bundled one.
The Output panel logs which runner source was selected for each test run.
Test runs use the current editor text, including unsaved edits, but VS Code may still save the file before running tests unless testing.saveBeforeTest is disabled in your settings.json.
Failures are shown in the Test Results UI. The Go to Error action selects the failing assertion line.
Right-clicking a failing test exposes Go to Source Range, which selects the source-line range covered by that failing assertion.
Currently, the Debug action uses the same runner as Run; debugger integration is not implemented yet.

Grammar Loading

The extension can load grammars from:

installed VS Code extensions (including built-in ones)
the nearest local package.json, or the one pointed to by tmGrammarTestTools.configPath
the optional grammar provider command

If your syntax test is not inside the grammar extension repo, the usual ways to point it at the right grammars are:

set tmGrammarTestTools.configPath to the package.json that contributes the relevant grammar
use a grammar provider when the needed grammars are generated, split across files, or not fully described by package.json

The loading rules are then:

When tmGrammarTestTools.autoLoadInstalledGrammars is false, installed VS Code grammars are skipped and only local package.json plus provider grammars are used.
For the same exact scope name, precedence follows that fixed load order: installed VS Code grammars first (when enabled) → then local package.json grammars → then provider grammars.
Injection grammars are additive. A local or provider injection grammar can extend a base grammar that comes from an installed or built-in VS Code extension, either by adding more specific scopes within existing content or by contributing injected regions.
If tmGrammarTestTools.grammarProvider.command is set, the extension runs it on each invocation and uses the returned grammar files for the current dump.

Grammar Provider

You can configure a grammar provider via workspace, workspace-folder, or global settings.json. This is useful when the grammars you want to test are generated, split across files, or not fully described by a nearby package.json (for example, when the source grammar is in .cson).

Example usage:

{
  "tmGrammarTestTools.grammarProvider.command": "node buildAndExportGrammars.js",
  "tmGrammarTestTools.grammarProvider.cwd": "${workspaceFolder}", // optional
  "tmGrammarTestTools.grammarProvider.scopes": ["source.js"], // optional
}

Provider command output can be either:

newline-separated grammar file paths
a JSON array of paths or grammar objects
a JSON object with a grammars array

Example output shape:

[
  "syntaxes/source.base.tmLanguage.json",
  {
    "path": "syntaxes/source.injection.tmLanguage.json",
    "scopeName": "source.injection",
    "injectTo": ["source.base"]
  }
]

See a small example provider that prints a JSON array of relative grammar paths.

Provider grammars participate in the normal load order described in Grammar Loading: exact scope-name matches override earlier sources, while injection grammars remain additive.

Supported variables in tmGrammarTestTools.grammarProvider.command:

${workspaceFolder}
${projectRoot}
${file}
${fileDirname}
${fileBasename}

Supported variables in tmGrammarTestTools.grammarProvider.cwd:

${workspaceFolder}
${projectRoot}
${fileDirname}

If tmGrammarTestTools.grammarProvider.scopes is set, the provider runs only when the syntax-test header scope exactly matches one of the configured values. Leave it empty or unset to allow the provider for any scope.

If ${workspaceFolder} is used in command or cwd, the active file must belong to a workspace folder.

${projectRoot} resolves to the nearest ancestor of the active file that contains package.json or .git. If neither is found, it resolves to the directory containing the file.

If tmGrammarTestTools.grammarProvider.cwd is empty or unset, the extension runs the provider command from the active document's workspace folder and falls back to ${projectRoot} when the file is outside the workspace.

command and cwd are resolved independently, so you can specify one in the workspace's .vscode/settings.json and the other in global settings.json, but in most cases it is reasonable to keep them together.

CLI

The CLI is currently available from a local checkout of this repository; it is not distributed separately yet. Clone the repository to use it.

For scripted use outside VS Code.

The CLI is read-only: it prints generated assertions to stdout and never modifies the file.

cd <this-repo-root>
npm run dump-assertions -- --file <syntax-test-file> --line <lineNumber>
# or
npm run dump-assertions -- --file <syntax-test-file> --range <startLine:startColumn-endLine:endColumn>

# From outside this repo:

# first compile:
cd <this-repo-root> && npm run compile

# then invoke it from anywhere:
cd <anywhere>
node <this-repo-root>/out/cli.js --file <syntax-test-file> <...>

Arguments and Options

Required:

--file <syntax-test-file> points to the syntax test file.
At least one target is required: --line and/or --range.

Targets:

--line <lineNumber> generates assertions for a line containing source text. 1-based. You can repeat it.
--range <startLine:startColumn-endLine:endColumn> generates assertions for a selected range using 1-based inclusive columns. You can repeat it too.
You can specify both at the same time.

Grammar loading:

--config <package.json> loads grammars from a grammar package manifest. If you omit it, the CLI searches upward from --file for a package.json with contributes.grammars.
--provider-command <command> runs the command and loads the returned grammars.
--provider-cwd <cwd> sets the provider working directory. If omitted, the CLI runs the provider from ${projectRoot} for --file.
--provider-scope <scope> is repeatable and limits provider execution to exact syntax-test header scope matches.
--provider-timeout-ms <ms> sets the provider timeout; the CLI fails if the provider does not finish in time.

Render options:

--scope-mode <full|minimal> controls full vs minimal rendering.
--minimal-header-scope-factoring <omitSharedHeader|keepSharedHeader> controls whether minimal mode omits the shared syntax-test header scope from the factored output or keeps it as part of the shared factored prefix. It can only be used with --scope-mode minimal, and --compare applies it to the minimal half of the comparison output.
--minimal-tail-scope-count <1|2> controls how many trailing scopes minimal mode keeps on terminal assertions. It can only be used with --scope-mode minimal, and --compare applies it to the minimal half of the comparison output.
--compact-ranges enables disjoint caret compaction. Enabled by default.
--no-compact-ranges disables disjoint caret compaction.

Output options:

--json prints structured JSON output. This is the default.
--plain prints only the generated assertion lines.
--compare prints the source line plus both minimal and full assertion blocks in plain text.
--log-level <silent|info|debug> prints CLI diagnostics to stderr.

Notes:

The CLI prints to stdout and never modifies the file.
With --log-level info, the CLI logs a short summary similar to the extension Output panel. --log-level debug also logs the effective grammar-usage trace used for assertion generation.
It currently loads grammars only from local package.json and/or --provider-command. It does not auto-load installed VS Code grammars.

Full usage example:

node <this-repo-root>/out/cli.js \
  --file /grammar-package/path/to/test.php \
  --provider-command "node utils/exportCsonGrammar.js" \
  --provider-cwd /grammar-package \
  --line 3 \
  --scope-mode minimal \
  --plain

Development

Testing

Run npm test.

Fixture

This repo also includes a minimal fixture grammar under fixtures/simple-grammar.

To try it inside this repo workspace:

Start the Run Extension debug configuration from VS Code's Run and Debug view.
In the Extension Host window, open fixtures/simple-grammar/tests/example.simple-poc.
Try the extension features: CodeLens above source lines, Code Actions on a selection, command-palette commands, Testing UI gutter, etc.

Name		Name	Last commit message	Last commit date
.vscode		.vscode
examples/grammar-provider		examples/grammar-provider
fixtures/simple-grammar		fixtures/simple-grammar
media		media
src		src
test		test
.gitignore		.gitignore
.vscodeignore		.vscodeignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json
tsconfig.test.json		tsconfig.test.json

Navigation Menu

Repository navigation

About

Repository files navigation

TM Grammar Test Tools

Generate Assertions

Run Tests

Quick Start

Command Behavior

Settings

Testing UI

Grammar Loading

Grammar Provider

CLI

Arguments and Options

Development

Testing

Fixture

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

TM Grammar Test Tools

Generate Assertions

Run Tests

Quick Start

Command Behavior

Settings

Testing UI

Grammar Loading

Grammar Provider

CLI

Arguments and Options

Development

Testing

Fixture

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages