Weird syntax highlighting

[xingyu0618]

Description

Ruby LSP Information

VS Code Version

1.95.3

Ruby LSP Extension Version

0.8.13

Ruby LSP Server Version

0.22.0

Ruby LSP Addons

Ruby Version

3.3.5

Ruby Version Manager

rbenv

Installed Extensions

Click to expand

• cmake (0.0.17)
• cmake-tools (1.19.52)
• debugpy (2024.12.0)
• elixir-ls (0.24.2)
• manpages (0.2.0)
• python (2024.20.0)
• ruby-extensions-pack (0.1.12)
• ruby-lsp (0.8.13)
• sorbet-vscode-extension (0.3.37)
• vscode-clangd (0.1.31)
• vscode-pylance (2024.11.2)
• vscode-zig (0.5.9)
• xonsh (0.3.3)

Ruby LSP Settings

Click to expand

Workspace

{}

User

{
  "enableExperimentalFeatures": true,
  "enabledFeatures": {
    "codeActions": true,
    "diagnostics": true,
    "documentHighlights": true,
    "documentLink": true,
    "documentSymbols": true,
    "foldingRanges": true,
    "formatting": true,
    "hover": true,
    "inlayHint": true,
    "onTypeFormatting": true,
    "selectionRanges": true,
    "semanticHighlighting": true,
    "completion": true,
    "codeLens": true,
    "definition": true,
    "workspaceSymbol": true,
    "signatureHelp": true,
    "typeHierarchy": true
  },
  "featuresConfiguration": {},
  "addonSettings": {},
  "rubyVersionManager": {
    "identifier": "rbenv",
    "miseExecutablePath": ""
  },
  "customRubyCommand": "",
  "formatter": "auto",
  "linters": null,
  "bundleGemfile": "",
  "testTimeout": 30,
  "branch": "",
  "pullDiagnosticsOn": "both",
  "useBundlerCompose": false,
  "bypassTypechecker": false,
  "rubyExecutablePath": "",
  "indexing": {},
  "erbSupport": true,
  "useLauncher": false,
  "featureFlags": {}
}

Here just a method packages in file foo.rb, nothing more.

def packages
  pkgs=`pacman -Qe`.split("\n").map do |pkg_spec|
    pkg_name, pkg_ver = pkg_spec.split(' ')
    out = `pacman -Qi #{pkg_name}`
    out =~ /^Installed Size\s*: (.*?) (MiB|KiB|B)\n/
    binding.irb unless $~
    # num, unit=$~.captures
    size=Size.new(*$~.captures)
    Package.new(pkg_name, size)
  end
end

[vinistock]

Thank you for the report! This seems to be a bug in how we detect the backtick as a string opening. If you add a space after the equals, the highlighting is fixed.

# Broken
pkgs=`pacman -Qe`.split("\n").map do |pkg_spec|

# Okay
pkgs= `pacman -Qe`.split("\n").map do |pkg_spec|

[xingyu0618]

Does ruby-lsp use Prism as backend? This kind of syntax-highlighting problem happens when you assign string literal to a variable.
No matter it' single quote, double quote, or backticks. You must put a space before string literal.
Prism can handle it correctly. What causes ruby-lsp to render it wrong?

[vinistock]

Yes, the Ruby LSP uses Prism in the backend. However, highlighting in editors has two levels: regular highlighting based on TextMate grammars and semantic highlighting provided by the server.

We noticed that VS Code lags very severely if the server returns too many semantic tokens, so we try to optimize to return semantic tokens only in situations where the syntax is ambiguous and cannot be handled properly with the TextMate grammar.

In this scenario, we don't use Prism for highlighting variable assignments because they are not ambiguous syntax and can be accurately highlighted with the grammar. It's just a bug that we accidentally introduced.

#3131 will fix all of the issues reported in this issue.