Skip to content

Drop utf8ToBytes and asciiToBytes in favor of TextEncoder #60

@coolaj86

Description

@coolaj86

TextEncoder and TextDecoder are already supported in Chrome and Firefox (not sure about MSIE and Safari).

var buf = new TextEncoder('utf-16le').encode("I ½ ♥ 💩");
var str = new TextDecoder('utf-16le').decode(buf);

If you use them you could drop a fair amount of code and only provide your UTF-8, UTF-16, etc conversions as a separate Polyfill module when the native TextEncoder and TextDecoder don't exist.

I haven't actually tested your UTF-8 converter, but if you get the same values as you get in node, then it's probably the best that there is. The full TextEncoder Polyfill seems to have much more code than I would intuitively expect and Mozilla's UTF-8 converter is actually incorrect (well it provides a correct encoding, but it's not the encoding that node and the W3C utilities use).

Anyway, then you could drop a bit of code.

I'm not sure how this compares in terms of performance, but if you wanted the polyfill code to be even smaller you can actually do this:

function utf8ToBinaryString() {
  var escstr = encodeURIComponent(str);
  // replaces any uri escape sequence, such as %0A, with binary escape, such as 0x0A
  var binstr = escstr.replace(/%([0-9A-F]{2})/g, function(match, p1) {
    return String.fromCharCode('0x' + p1);
  });

  return binstr;
}

function utf8ToBuffer(str) {
  var binstr = utf8ToBinaryString(str);
  var buf = new Uint8Array(binstr.length);
  Array.prototype.forEach.call(binstr, function (ch, i) {
    buf[i] = ch.charCodeAt(0);
  });
  return buf;
}

function utf8ToBase64(str) {
  var binstr = utf8ToBinaryString(str);
  return btoa(binstr);
}

I don't know if that still works for utf-16, but I have a hunch that it would.

I know I could have just used your module here, but I've been enjoying the headache of learning the hacky, non-hacky, and best-possible ways of doing this stuff in the browser.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions