XMLHttpRequest with responseType='arraybuffer' corrupts binary responses (fetch() is correct)

[alexskinner]

Before you start

• I searched existing issues. Related to Binary fetch and interception bodies are corrupted by UTF-8 conversion #716 but a different path — see the note below.
• I tested against main.
• This is not a security vulnerability.

obscura version or commit

9326880 (main)

OS and architecture

macOS arm64

How are you using obscura?

Embedded Rust library and CDP server.

Affected area

XMLHttpRequest response bodies.

Build variant

Render-enabled.

Reproduction

Serve a file containing every byte value and read it back both ways.

python3 -c "open('bytes.bin','wb').write(bytes(range(256)))"
python3 -m http.server 9977 --bind 127.0.0.1

const viaFetch = new Uint8Array(await (await fetch('bytes.bin')).arrayBuffer());

const viaXhr = await new Promise(res => {
  const x = new XMLHttpRequest();
  x.open('GET', 'bytes.bin', true);
  x.responseType = 'arraybuffer';
  x.onload = () => res(new Uint8Array(x.response));
  x.send();
});

console.log(viaFetch.length, viaXhr.length);   // expect 256 256

(Loopback needs OBSCURA_ALLOW_PRIVATE_NETWORK=1, or serve the fixture from anywhere else.)

Expected behavior

Both are 256 bytes and byte-for-byte identical to the file. Chrome 147:

fetch len 256 | xhr len 256 | diverge at 256
FETCH 78 79 7a 7b 7c 7d 7e 7f 80 81 82 83 84 85 86 87 88 89 8a 8b …
XHR   78 79 7a 7b 7c 7d 7e 7f 80 81 82 83 84 85 86 87 88 89 8a 8b …

Actual behavior

fetch() is correct. XMLHttpRequest returns 255 bytes, and every byte from 0x80 on is wrong:

fetch len 256 | xhr len 255 | diverge at 128
FETCH 78 79 7a 7b 7c 7d 7e 7f 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f …
XHR   78 79 7a 7b 7c 7d 7e 7f 01 c2 83 c4 85 c6 87 c8 89 ca 8b cc 8d ce 8f d0 …

Bytes 0x00–0x7F survive; everything above is mangled. The pattern identifies the
transform exactly — the body is being decoded as UTF-8 (leniently, treating any
high byte as a lead byte) and then re-encoded:

source bytes	lenient UTF-8 decode	re-encoded
82 83	(0x82 & 0x1F) << 6 | (0x83 & 0x3F) = U+0083	c2 83
84 85	U+0105	c4 85
86 87	U+0187	c6 87
88 89	U+0209	c8 89

So the ArrayBuffer holds the UTF-8 encoding of the decoded code points rather
than the original octets, and the length changes with the content.

Because ASCII passes through untouched, JSON and text over XHR look fine, which
is why this hides.

Impact

Any binary read over XMLHttpRequest. In particular every Emscripten/WASM
application, since the generated loader fetches .wasm and data files with
XMLHttpRequest + responseType = 'arraybuffer' rather than fetch(). A real
4 MB asset came back 77,065 bytes short with a corrupted header, and the
application aborted on a nonsense allocation derived from it — the failure
surfaces far from the cause.

Does headless Chrome behave the same?

No. Chrome 147 returns 256 identical bytes on both paths (output above).

Note on #716

#716 covers binary request bodies (fetch(..., { body }), XHR.send()) and
the CDP fulfill path. This is the response side of XMLHttpRequest, and it
is a separate code path: fetch() responses are already byte-correct here while
XHR responses are not. Same root cause — a String where Vec<u8> belongs —
so it may well fold into the same fix, and I'm happy for this to be closed into
#716 if you'd rather track it there.

[alexskinner]

Found the cause and it is contained: XMLHttpRequest is implemented over fetch
in crates/obscura-js/js/bootstrap.js, and it reads every response through
resp.text() before choosing a responseType:

const text = await resp.text();
...
case 'arraybuffer':
  xhr.response = new TextEncoder().encode(text).buffer;
case 'blob':
  xhr.response = new Blob([text]);

text() → TextEncoder().encode() is not a round-trip for bytes that are not
valid UTF-8, which is exactly the transform the byte dump in the issue shows.
fetch() is unaffected because it never takes that detour.

Reading the body as bytes once, and decoding to text only for the text-ish
response types, fixes it:

       xhr._setReadyState(2); // HEADERS_RECEIVED
 
-      const text = await resp.text();
+      const buffer = await resp.arrayBuffer();
       if (xhr._aborted) return;
 
+      const wantsText = xhr.responseType === '' || xhr.responseType === 'text'
+                     || xhr.responseType === 'json' || xhr.responseType === 'document';
+      const text = wantsText ? new TextDecoder().decode(buffer) : '';
+
       xhr.responseText = text;
       xhr._setReadyState(3); // LOADING
 
       switch (xhr.responseType) {
         ...
         case 'arraybuffer':
-          xhr.response = new TextEncoder().encode(text).buffer;
+          xhr.response = buffer;
           break;
         case 'blob':
-          xhr.response = new Blob([text]);
+          xhr.response = new Blob([buffer]);
           break;

blob had the same bug. Not decoding the body for the binary types also avoids
building a multi-megabyte string that nothing reads — and per spec
responseText is not defined for those types anyway (this leaves it ''
rather than throwing, which seemed the safer change).

Verified on the 256-byte fixture from the issue:

before   fetch len 256 | xhr len 255 | diverge at 128
after    fetch len 256 | xhr len 256 | diverge at 256      (matches Chrome 147)

And on a real 4,196,020-byte asset: XHR now returns all 4,196,020 bytes instead
of 4,118,955. An Emscripten application that previously aborted while parsing a
corrupted header now starts and runs normally, rendering to a 2D canvas and
responding to Input.dispatchKeyEvent.

Happy to open a PR if useful.