`textDocument/semanticTokens/full` emits negative `deltaStart` for interpolations inside heredoc body

[flohessling]

Language Server Version

v0.38.5

Terraform Version

Terraform v1.14.6 on darwin_arm64

Client Version

Neovim v0.12.0-dev

Terraform Configuration

variable "items" {
  type = map(object({
    name       = string
    team_names = list(string)
  }))
}

resource "null_resource" "test" {
  for_each = var.items
  triggers = {
    message = <<-EOT
      ${join(" ", [for team in each.value.team_names : "@team-${team}"])}
      Pinging external service
      {{#is_warning}}
      Cluster for ${each.value.name} is approaching threshold
      {{/is_warning}}
    EOT
  }
}

Steps to Reproduce

1. Create a .tf file containing a heredoc (<<-EOT) with string interpolations (${...}) in the body (not the first line) — see the Terraform configuration above
2. Open the file in an LSP client
3. Observe the textDocument/semanticTokens/full response — the data array contains deltaStart values like 4294967255 (negative integers wrapped to uint32)

Expected Behavior

All deltaStart values in the textDocument/semanticTokens/full response should be non-negative uinteger values as required by the LSP specification. Reference tokens inside heredoc interpolations (e.g., each, value, name in ${each.value.name}) should have correct column deltas relative to the preceding token on the same line.

Actual Behavior

deltaStart values for reference tokens inside heredoc body interpolations overflow to large uint32 values (e.g., 4294967255 which is uint32(-41)). This is caused by token_encoder.go using the multi-line string token's Range.Start.Column from the first line of the heredoc as previousStartChar for tokens on a later line within the heredoc. Since the first-line column is higher than the interpolation token's column, the subtraction produces a negative value that wraps on uint32 cast.

Gist

https://gist.github.com/flohessling/f86b59f753286609b425c2266345131e

Workarounds

Disable semantic tokens for terraform-ls in the editor configuration.

For Neovim, add to the LSP on_attach callback:
client.server_capabilities.semanticTokensProvider = nil

References

• LSP: Infinite Hang + 100% CPU on typing in Insert Mode neovim/neovim#36257
• Report semantic tokens for HEREDOC (multi-line strings) #1220

Help Wanted

• I'm interested in contributing a fix myself

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment