Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update julia grammar #520

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update julia grammar #520

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 5 additions & 14 deletions lang/semgrep-grammars/src/semgrep-julia/grammar.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,12 @@ const base_grammar = require('tree-sitter-julia/grammar');
module.exports = grammar(base_grammar, {
name: 'julia',

conflicts: ($, previous) => previous.concat([
[$.typed_parameter, $._expression],
]),

/*
Support for semgrep ellipsis ('...') and metavariables ('$FOO'),
if they're not already part of the base grammar.
*/
rules: {
semgrep_ellipsis: $ => '...',
semgrep_ellipsis: _ => '...',

catch_clause: $ => prec(1, seq(
'catch',
Expand All @@ -33,8 +29,8 @@ module.exports = grammar(base_grammar, {
// metavariable usually is. This is fine, because having a slightly more
// permissive grammar is OK, we will just dispatch in the Generic
// translation.
semgrep_extended_metavariable: $ =>
/\$[A-Z_][a-zA-Z_0-9]*/,
semgrep_extended_metavariable: _ =>
/\$[A-Z_][a-zA-Z_0-9]*/,

// Metavariables
// We allow an identifier to be a simple metavariable regex, so that
Expand Down Expand Up @@ -69,23 +65,18 @@ module.exports = grammar(base_grammar, {
// tree stays the same. Otherwise, we will fail `tree-sitter` tests.
alias(prec(999,
$.semgrep_extended_metavariable
), $.identifier)
), $.identifier)
),

_expression: ($, previous) => choice(
previous,
$.semgrep_ellipsis,
$.deep_expression
$.deep_expression
),

_statement: ($, previous) => choice(
previous,
$.semgrep_ellipsis,
),

typed_parameter: ($, previous) => choice(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why removing this? We can still have ellpisis in type parameters even with this removal?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Julia doesn't have a distinction between expressions and patterns, so function signatures are parsed as function calls, and checked in the lowering phase (after macro expansion). Having these parameter rules added a lot of duplication and conflicts, so we removed them.

We can still have ellpisis in type parameters even with this removal?

Yes. Ellipsis would now be parsed as splat_expression.

previous,
$.semgrep_ellipsis,
),
}
});
19 changes: 11 additions & 8 deletions lang/semgrep-grammars/src/semgrep-julia/test/corpus/semgrep.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,11 @@ end

(source_file
(function_definition
(identifier)
(parameter_list
(typed_parameter
(semgrep_ellipsis)))
(signature
(call_expression
(identifier)
(argument_list
(semgrep_ellipsis))))
(return_statement
(integer_literal))))

Expand Down Expand Up @@ -122,9 +123,11 @@ end

(source_file
(function_definition
(interpolation_expression
(identifier))
(parameter_list)
(signature
(call_expression
(interpolation_expression
(identifier))
(argument_list)))
(assignment
(interpolation_expression
(identifier))
Expand Down Expand Up @@ -219,7 +222,7 @@ end
(argument_list
(vector_expression))
(do_clause
(parameter_list
(argument_list
(identifier
(semgrep_extended_metavariable)))
(return_statement
Expand Down
2 changes: 1 addition & 1 deletion lang/semgrep-grammars/src/tree-sitter-julia
Open in desktop
Submodule tree-sitter-julia updated 33 files
+39 −0 .editorconfig
+2 −2 .github/workflows/ci.yml
+4 −4 Cargo.toml
+110 −0 Makefile
+14 −4 README.md
+7 −5 binding.gyp
+16 −0 bindings/c/tree-sitter-julia.h
+11 −0 bindings/c/tree-sitter-julia.pc.in
+13 −0 bindings/go/binding.go
+15 −0 bindings/go/binding_test.go
+5 −0 bindings/go/go.mod
+14 −22 bindings/node/binding.cc
+28 −0 bindings/node/index.d.ts
+3 −15 bindings/node/index.js
+5 −0 bindings/python/tree_sitter_julia/__init__.py
+1 −0 bindings/python/tree_sitter_julia/__init__.pyi
+27 −0 bindings/python/tree_sitter_julia/binding.c
+0 −0 bindings/python/tree_sitter_julia/py.typed
+139 −287 grammar.js
+24 −3 package.json
+29 −0 pyproject.toml
+57 −0 setup.py
+590 −1,418 src/grammar.json
+253 −898 src/node-types.json
+1,263,903 −2,024,028 src/parser.c
+4 −4 src/scanner.c
+54 −0 src/tree_sitter/alloc.h
+290 −0 src/tree_sitter/array.h
+34 −5 src/tree_sitter/parser.h
+211 −175 test/corpus/definitions.txt
+26 −22 test/corpus/expressions.txt
+2 −2 test/corpus/operators.txt
+27 −19 test/corpus/statements.txt
Loading