Skip to content

base64: Support multi bytes encoding characters - #1122

Merged
sonnyp merged 2 commits into
xmppjs:mainfrom
randax:fix/base64-utf8-encoding
Apr 13, 2026
Merged

base64: Support multi bytes encoding characters#1122
sonnyp merged 2 commits into
xmppjs:mainfrom
randax:fix/base64-utf8-encoding

Conversation

@randax

@randax randax commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • encode()/decode() in @xmpp/base64 use btoa()/atob(), which only handle Latin-1 (single-byte) encoding
  • Non-ASCII characters like ø (U+00F8) get encoded as a single byte (0xF8) instead of the correct UTF-8 sequence (0xC3 0xB8)
  • This silently corrupts SASL PLAIN credentials for users with non-ASCII usernames, causing authentication failures downstream

The fix replaces btoa()/atob() with TextEncoder/TextDecoder to properly handle UTF-8, and adds a test suite covering Nordic (æøå), German (äöüß), French (éèêëç), Cyrillic, CJK, Arabic, Hebrew, and emoji characters.

Reproduction

Any username containing characters outside ASCII will produce corrupted SASL PLAIN payloads. For example, with username øyvindranda:

import { encode } from "@xmpp/base64";

// Current (broken): btoa encodes ø as single byte 0xF8
encode("ø"); // => "+A==" (base64 of [0xF8])

// Fixed: TextEncoder encodes ø as UTF-8 bytes 0xC3, 0xB8
encode("ø"); // => "w7g=" (base64 of [0xC3, 0xB8])

When a server decodes the +A== payload, byte 0xF8 is not valid UTF-8, so the username gets mangled (e.g. replaced with U+FFFD), and JID construction or session binding fails silently.

Test plan

  • All 18 new tests in packages/base64/test/test.js pass
  • Full test suite passes (201 passed, 1 skipped, 0 failed)
  • ASCII-only credentials produce identical base64 output (backwards compatible)
  • Verified with real XMPP server: Nordic username øyvindranda authenticates successfully after fix

🤖 Generated with Claude Code

Øyvind Randa and others added 2 commits April 12, 2026 23:31
btoa()/atob() treat each character as a single Latin-1 byte, which
silently corrupts non-ASCII characters in SASL credentials. For
example, "ø" (U+00F8) gets encoded as byte 0xF8 instead of the
correct UTF-8 sequence 0xC3 0xB8, causing XMPP authentication
failures for users with non-ASCII usernames (Nordic æøå, German
umlauts, Cyrillic, CJK, etc.).

Use TextEncoder/TextDecoder to properly handle UTF-8 encoding and
decoding before passing to btoa()/atob().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sonnyp sonnyp changed the title fix(base64): use TextEncoder/TextDecoder for proper UTF-8 encoding base64: Support multi bytes encoding characters Apr 13, 2026

@sonnyp sonnyp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

I slightly improved the impl a7279c4

@sonnyp

sonnyp commented Apr 13, 2026

Copy link
Copy Markdown
Member

The new tests are passing, CI failures are unrelated and will be addressed separately.

@sonnyp
sonnyp merged commit 7035ced into xmppjs:main Apr 13, 2026
0 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants