Skip to content

feat: add TextEncoder/TextDecoder scene globals#2542

Open
kuruk-mm wants to merge 1 commit into
mainfrom
feat/text-codec-globals-signed
Open

feat: add TextEncoder/TextDecoder scene globals#2542
kuruk-mm wants to merge 1 commit into
mainfrom
feat/text-codec-globals-signed

Conversation

@kuruk-mm

Copy link
Copy Markdown
Member

Scene runtimes have no TextEncoder/TextDecoder because the embedded deno_core runs without deno_web, so SDK scenes and libraries relying on the WHATWG text codecs crash at load (or fall back to slow bundled polyfills).

This adds two std-only ops (op_utf8_encode/op_utf8_decode) and a small JS shim module registered like the existing fetch/ws shims, installing WHATWG-shaped TextEncoder and TextDecoder on globalThis when absent.

  • TextDecoder accepts the WHATWG utf-8 label set with fatal and ignoreBOM support, throws RangeError for any other encoding and a loud TypeError for the unsupported { stream: true } option.
  • TextEncoder implements encode and encodeInto, including correct UTF-8 boundary truncation for encodeInto on short destinations.
  • Decoding uses Rust's from_utf8_lossy, whose maximal-subparts U+FFFD substitution matches the WHATWG spec exactly; encoding inherits USVString lone-surrogate replacement from deno_core's string conversion. No new dependencies.

Unit tests cover the Rust decode paths (cargo test text_encoding) and the JS shim (13 node:test cases mirroring ws.test.cjs). Full crate suite passes (170 tests), cargo fmt clean. The ops are pure std string/slice operations with no OS APIs, so this ships identically on desktop and the Android/iOS builds (default features include use_deno on all targets).

Scenes built with the current SDK keep working unchanged: the SDK installs its polyfill only when the global is absent, so the native-backed implementation wins automatically, including for already-published content.

Related: decentraland/js-sdk-toolchain#1452 (slims the SDK polyfill to a validating utf-8 codec), decentraland/bevy-explorer#958 (same exposure via deno_web).

Note for maintainers: the shim tests are runnable with node --test lib/src/dcl/js/js_modules/text_encoding.test.cjs and are meant to join ws.test.cjs in the static-checks node --test step; that one-line workflow edit is left out of this PR (pushed from a token without workflow scope).

Supersedes #2517 (same changes, re-committed with a signed commit).

Co-authored-by: Esteban Ordano <esteban@decentraland.org>
@kuruk-mm
kuruk-mm requested a review from a team as a code owner July 14, 2026 23:58
@kuruk-mm
kuruk-mm enabled auto-merge (squash) July 15, 2026 00:01
@github-actions

Copy link
Copy Markdown
Contributor

📦 Build Report

🤖 Android

Artifact Status
APK 📱 Download APK
AAB main/release builds only
Debug Symbols main/release builds only

Build Status: ✅ Success

🍏 iOS

The signed APK is uploaded to R2 on every build (link above). Add the build label (alias: build-ios) to build & ship iOS to TestFlight and post a Slack "Android build ready" notification.


🔗 Workflow Run: View logs

🔄 Updated: 2026-07-15 00:13:41 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant