Skip to content

utf-8 is incorrect when used in browsers or React Native #369

@ChALkeR

Description

@ChALkeR

In Node.js, UTF-8 decoding is fine

The issue is that in browsers:

  1. Buffer is polyfilled to http://npmjs.com/package/buffer, which is old and invalid
  2. string_decoder is polyfilled to https://npmjs.com/package/string_decoder, which is also old and invalid

And iconv-lite just uses those everywhere without checking

Example:

import iconv from 'iconv-lite'

console.log(escape(iconv.decode(Buffer.of(0xf0, 0x90, 0x80), 'utf8'))) // should be %uFFFD
console.log(escape(iconv.decode(Buffer.of(0xf0, 0x80, 0x80), 'utf8'))) // should be %uFFFD%uFFFD%uFFFD

Correct:

$ node iconv-test.js
%uFFFD
%uFFFD%uFFFD%uFFFD

But when run in browsers or React Native, it outputs:

%uFFFD
%uFFFD

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions