Skip to content

Fix multiline semantic token encoding in heredoc#2100

Open
arnaud-dezandee wants to merge 1 commit intohashicorp:mainfrom
arnaud-dezandee:fix/heredoc-semantic-token-encoding
Open

Fix multiline semantic token encoding in heredoc#2100
arnaud-dezandee wants to merge 1 commit intohashicorp:mainfrom
arnaud-dezandee:fix/heredoc-semantic-token-encoding

Conversation

@arnaud-dezandee
Copy link
Copy Markdown

@arnaud-dezandee arnaud-dezandee commented Mar 21, 2026

Summary

Alternative to #2095.

This fixes the same multiline semantic token encoding bug, but takes a slightly broader approach. In addition to fixing the bad delta anchor after multiline token emission, it also fixes how multiline token segments are sliced per line.

I hit this while reproducing broken Terraform heredoc highlighting in VS Code.

What changed

  • track the last emitted token position directly
  • compute multiline segment lengths from the correct per-line start offset
  • skip zero-length segments
  • add regression coverage for the heredoc interpolation case

Why not just #2095

#2095 fixes the overflow symptom. This PR is an alternative that also fixes the multiline slicing itself. While testing, I found the encoder was over-reporting the first emitted line length for multiline tokens that start late on a line, so I updated the existing multiline test as well.

Validation

  • reproduced locally in VS Code
  • verified the highlighting fix with a custom terraform-ls binary
  • ran focused tests
  • ran the full terraform-ls test suite

PCI review checklist

  • I have documented a clear reason for, and description of, the change I am making.
  • If applicable, I've documented a plan to revert these changes if they require more than reverting the pull request.
  • If applicable, I've documented the impact of any changes to security controls.
    Examples of changes to security controls include using new access control methods, adding or removing logging pipelines, etc.

@arnaud-dezandee arnaud-dezandee requested a review from a team as a code owner March 21, 2026 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant