Skip to content

fix(lsp): prevent negative deltaStart overflow in multi-line semantic token encoding#2095

Open
flohessling wants to merge 1 commit intohashicorp:mainfrom
flohessling:fix-semtok-multiline-deltastart-overflow
Open

fix(lsp): prevent negative deltaStart overflow in multi-line semantic token encoding#2095
flohessling wants to merge 1 commit intohashicorp:mainfrom
flohessling:fix-semtok-multiline-deltastart-overflow

Conversation

@flohessling
Copy link
Copy Markdown

@flohessling flohessling commented Mar 9, 2026

Summary

Fixes: #2094

Multi-line semantic tokens (e.g., heredoc strings with interpolations) produce negative deltaStart values that wrap to large uint32 numbers. This causes LSP clients like Neovim to hang with 100% CPU usage.

Root Cause

After a multi-line token is per-line split in encodeTokenOfIndex(), te.lastEncodedTokenIdx still points to the original token. The next token's previousStartChar is computed from the original token's Range.Start.Column (high value from the first line of the heredoc), not from the last emitted line. When the next token starts at a lower column, the subtraction produces a negative value that wraps on uint32 cast.

Fix

Adds lastEncodedLine and lastEncodedStartChar fields to TokenEncoder to track the actual last emitted position instead of deriving it from the original multi-line token's range.

Testing

Adds TestTokenEncoder_multiLineTokenFollowedBySameEndLineToken which reproduces the exact scenario: a multi-line string token followed by a reference token on the same line as the string's End.Line, with a lower column than the string's Start.Column.

PCI review checklist

  • I have documented a clear reason for, and description of, the change I am making.
  • If applicable, I've documented a plan to revert these changes if they require more than reverting the pull request.
  • If applicable, I've documented the impact of any changes to security controls.
    Examples of changes to security controls include using new access control methods, adding or removing logging pipelines, etc.

@flohessling flohessling requested a review from a team as a code owner March 9, 2026 15:54
@hashicorp-cla-app
Copy link
Copy Markdown

hashicorp-cla-app bot commented Mar 9, 2026

CLA assistant check
All committers have signed the CLA.

@hashicorp-cla-app
Copy link
Copy Markdown

CLA assistant check

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes

Have you signed the CLA already but the status is still pending? Recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

textDocument/semanticTokens/full emits negative deltaStart for interpolations inside heredoc body

1 participant