Implement external scanner for raw_text #15
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
src/tree-sitter-javascript/scanner.h
Outdated
| // | ||
| // To Update: | ||
| // - delete everything under the comment block | ||
| // - find and replace "tree_sitter_javascript" with "tree_sitter_javascript" |
There was a problem hiding this comment.
seems like something went wrong here 🙃 haha
There was a problem hiding this comment.
Sure did! Will update.
| JSX_TEXT, | ||
| }; | ||
|
|
||
| void *tree_sitter_javascript_external_scanner_create() { return NULL; } |
There was a problem hiding this comment.
shouldn't these be tree_sitter_glimmer_javascript?
There was a problem hiding this comment.
At the moment these aren’t in use, and duplicated version with glimmer_javascript are in scanner.c.
I see three options; leave them, call them, or remove them.
I thought it would be nice to have the ability to just copy paste scanner.c from GitHub into scanner.h. So maybe best to just call them from scanner.c?
There was a problem hiding this comment.
They are now called from scanner.c
There was a problem hiding this comment.
yea, copy-paste ability is very nice
To simplify having a custom external scanner, I've moved JS scanner into `tree-sitter-javascript/scanner.h`. This will allow us to have custom scanner implementation and still allow us to easily update the upstream scanner.
It allows greater control and allows us to expose more of the template. External scanner seems to be the way if we want to expose more than just the `<template>` token. This obviously makes it more tedious to update the upstream TypeScript grammar. Especially the `GlimmerTokenType` which must continue the sequence of the parent's TokenType.
Some editor features requires the abillity to capture the brackets independent from the tag name. e.g. Zed captures `<` as brackets and highlight them different from the tag name. It uses it to highlight the opposite opening/closing pair when the cursor is on one of the brackets.
mogstad
force-pushed
the
push-nrwrmwwzrtvk
branch
from
07f27d5 to
7688609
Compare
NullVoxPopuli
approved these changes
Mar 20, 2025
|
thank you!!! |
Open
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Same as ember-tooling/tree-sitter-glimmer-typescript#10, but moved the upstream parser into
src/tree-sitter-javascript/scanner.hto make it easier to keep up with changes upstream. This partly matches what’s already in place fortree-sitter-glimmer-typescript.