Syntax highlighting of c++ preprocessor interprets '/*' inside string as a comment

[jonner]

Summary

The syntax highlighting that helix uses seems to have some issues with the character sequence '/*' within a string in C++ preprocessor symbols. It treats the character sequences as the start of a comment even though it is within a quoted string literal. When the language is set to 'c', it works, but it fails for 'cpp'.

Reproduction Steps

I tried this:

1. download foo.cpp
2. hx foo.cpp

I expected this to happen:

it should highlight the int foo(); line as a function

Instead, this happened:

the int foo(); line is highlighted as a comment

Helix log

No response

Platform

Linux

Terminal Emulator

ptyxis

Installation Method

fedora package

Helix Version

helix 25.07.1 (a05c151)

[m4rch3n1ng]

this is an issue with the upstream https://github.com/tree-sitter/tree-sitter-c grammar. doing tree-sitter parse on that file gives me this:

(translation_unit [0, 0] - [7, 0]
  (preproc_def [0, 0] - [5, 0]
    name: (identifier [0, 8] - [0, 11])
    value: (preproc_arg [0, 12] - [0, 32])
    (comment [0, 32] - [4, 23]))
  (declaration [6, 0] - [6, 10]
    type: (primitive_type [6, 0] - [6, 3])
    declarator: (function_declarator [6, 4] - [6, 9]
      declarator: (identifier [6, 4] - [6, 7])
      parameters: (parameter_list [6, 7] - [6, 9]))))

where you can see, that there is a comment that spans 4 lines (0 - 4) and only one declaration.

you should probably report this upstream, however given that that repo is seemingly not very actively maintained it might not get fixed anytime soon.

[jonner]

I guess this should actually be against tree-sitter-cpp, rather than tree-sitter-c since the c syntax works fine?

[m4rch3n1ng]

does it? i tried mine with the c grammar (as i didn't see the c++ in your title and assumed it was c based on the code lol) and it didn't work for me. the cpp grammar also just an extension of the C grammar, and i don't think the preprocessor stuff is handled differently in c++ than in c.

[jonner]

@m4rch3n1ng when I do set-language c in helix, it highlights things properly. Does it not do this for you?

[m4rch3n1ng]

no it doesn't work for me, however i am using a newer version of helix than you. when i use the version from your commit the c grammar actually works and the c++ version doesn't, so i assume this is a regression in the c grammar, that has been updated to my version. i'll do a git bisect and figure out where the regression came from and i'll report back here.

[m4rch3n1ng]

ok found the regression, it's in tree-sitter/tree-sitter-c@d607e5e (technically git bisect points to tree-sitter/tree-sitter-c@43a147a, but that was only for generating the c files and headers for that commit). not sure what the issue here is actually as that seems to fix something else, but it seems to not be happy with that?

[m4rch3n1ng]

also, i think i'll just report the bug upstream (and link here). thanks for the report! (even if it wasn't to the correct repo) i don't know why i said this, but you can just open the issue yourself lol.

[m4rch3n1ng]

oh actually nvmd, this didn't actually really work before: it was just parsing the // in the url as a line comment and then worked due to that:

(translation_unit [0, 0] - [7, 0]
  (preproc_def [0, 0] - [1, 0]
    name: (identifier [0, 8] - [0, 11])
    value: (preproc_arg [0, 12] - [0, 18])
    (comment [0, 18] - [0, 35]))
  (function_definition [2, 0] - [2, 12]
    type: (primitive_type [2, 0] - [2, 3])
    declarator: (function_declarator [2, 4] - [2, 9]
      declarator: (identifier [2, 4] - [2, 7])
      parameters: (parameter_list [2, 7] - [2, 9]))
    body: (compound_statement [2, 10] - [2, 12]))
  (comment [4, 0] - [4, 23])
  (declaration [6, 0] - [6, 10]
    type: (primitive_type [6, 0] - [6, 3])
    declarator: (function_declarator [6, 4] - [6, 9]
      declarator: (identifier [6, 4] - [6, 7])
      parameters: (parameter_list [6, 7] - [6, 9])))

see the (comment [0, 18] - [0. 35]) node in there.

i am unsure if this is actually really fixable in the c grammar, as they just seem to be doing token stream parsing and don't actually parse strings. the reason comments can work there is because they are defined in the extras section.

i'll see if i can actually fix this over there, no promises though.