fix: parse LSP messages by byte length to fix multi-byte UTF-8 (CI + real-Zed verified) - #11
Merged
Merged
Conversation
LSP Content-Length is defined in bytes, but data-parser.js parsed by
character. Any message containing multi-byte UTF-8 (e.g. a didOpen for an
.ets file with Chinese content) was mis-sliced: the parser grabbed too
many characters, pulling the next message's header into the JSON and
failing to parse — silently dropping the document so the language server
never received it, leaving the LSP dead for those files.
Restore Buffer-based parsing (Content-Length counted in bytes). Keep
setEncoding('utf8') in index.js and add a string->Buffer guard in parse()
so it works whether stdin yields strings or Buffers — the combination the
earlier reverts (a88ed44, 052b8be) never had together.
Verified in a real Zed editor against the real @arkts/language-server:
char-based drops the multi-byte didOpen; byte-based forwards it intact
with zero "Error parsing message" occurrences.
Also harden the initialize handler: forward the request as-is instead of
crashing with "Cannot read properties of undefined (reading 'tsdk')" when
initializationOptions is not configured.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Zed sends the lowercased language name ("arkts language") as the didOpen
languageId when no language_ids mapping is set. @arkts/language-server only
activates its ETS plugin for languageId === "ets" (and ts/js/json ids), so
every document was silently treated as plain text: definition, hover and
documentHighlight all returned empty results with no error. This was the
root cause of "go to definition does nothing" in Zed.
Verified end-to-end in real Zed: with the mapping, textDocument/definition
returns the correct LocationLink and the editor navigates to it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Zed only passes initializationOptions when the user configured lsp.arkts-language-server.initialization_options in settings; without them the server cannot finish initialize (it fails loading TypeScript) and Zed reports "Failed to start language server". Fall back to env vars (ZED_ETS_TSDK/TSDK, ZED_ETS_OHOS_SDK_PATH/OHOS_SDK_PATH) and then auto-detect the tsdk from the ohos-typescript package installed next to @arkts/language-server, so the server starts out of the box. A missing ohosSdkPath now degrades ArkUI typings instead of blocking startup. Also: swallow the response to the wrapper-injected ets/waitForEtsConfigurationChangedRequested request instead of forwarding it to the editor (which never issued it), and drop stdin setEncoding so the byte-based parser receives raw Buffers end to end (chunk splits inside multi-byte characters are covered by data-parser tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…iers) DefTest.ets places a resolvable reference at position 0:0 so a freshly opened editor can exercise textDocument/definition without moving the cursor; ChineseTest.ets covers multi-byte UTF-8 content including a Chinese identifier as the definition target. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- step 6 was missing the compiled tree-sitter grammar wasm; without grammars/arkts.wasm Zed fails to load the language entirely (no highlighting, no LSP). Document compiling it with Zed's cached wasi-sdk. - the Force* editor actions were never registered in element.rs, so the documented commands could not fire; document the registration step. - document the new ZED_AUTO_CMD_FILE automation channel and the two ways to bypass the worktree-trust dialog for headless runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v1.3+ refuses to initialize unless ets.sdkPath is an existing directory containing ets/build-tools/ets-loader/tsconfig.json (v1.2 accepted any value and merely warned). When no ohosSdkPath is configured, create a minimal placeholder skeleton under the OS temp dir instead of passing a nonexistent path, so the server still starts on both v1.2 and v1.3; ArkUI typings stay degraded until a real SDK path is configured. Verified against @arkts/language-server 1.3.10: initialize succeeds and definition/hover return correct results with the placeholder skeleton. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bump zed-ets-language-server to 3.0.0 and the extension to 0.3.0, and pin LANGUAGE_SERVER_VERSION to "3" so each extension release states exactly which wrapper major it is compatible with. Rework the update check to compare within the pinned major instead of against npm's overall latest: the old logic reinstalled the wrapper on every startup whenever npm's latest belonged to a different major than the pin (installed could never equal latest), adding a network round trip to every language server start. Now a wrapper of the right major is kept as-is unless a newer release exists within that same major, and a failed latest-version lookup (e.g. offline) no longer prevents startup when a compatible wrapper is already installed. Note: publish zed-ets-language-server@3.0.0 to npm before shipping extension 0.3.0, otherwise fresh installs have no 3.x to download. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
CI (
Test LSP Serverjob) was failing: two tests inlib/data-parser.test.jsunder UTF-8 byte length handling — the parse callback was never invoked.Root cause: LSP
Content-Lengthis defined in bytes, butdata-parser.jshad been reverted to parse by character (stdinBuffer = '',substring). Any message containing multi-byte UTF-8 (e.g.textDocument/didOpenfor an.etsfile with Chinese content) is mis-sliced — the parser grabs too many characters, pulls the next message's header into the JSON, andJSON.parsethrows. The document is silently dropped, so the language server never receives it → dead LSP for those files. The tests asserted the correct byte-based behavior, so they (correctly) failed.This has been reverted the wrong way several times (
c6cee0c→1083e2d→a88ed44, plus3b04512/052b8be). Each revert "worked" only because LSP startup traffic is ASCII.Fix
lib/data-parser.js— restore Buffer-based parsing (Content-Length counted in bytes) with all three pieces the earlier reverts never had together:Buffer.alloc(0),Buffer.concat,subarray,.toString('utf8'));process.stdin.setEncoding('utf8')inindex.js(its StringDecoder reassembles multi-byte chars split across TCP chunks);parse()(Buffer.isBuffer(data) ? data : Buffer.from(data, 'utf8')) — becausesetEncodingyields strings andBuffer.concat([buf, string])throws.index.js— harden theinitializehandler to forward the request as-is instead of crashing withCannot read properties of undefined (reading 'tsdk')wheninitializationOptionsisn't configured.Verification
@arkts/language-server, opening a Chinese-heavy.etsfile:didOpenis dropped; wrapper logsError parsing message: Unexpected non-whitespace character after JSON.didOpenforwarded intact, 0 parse errors, no crash.🤖 Generated with Claude Code