CODEWITHSUNDEEP
javascriptcharacter-encodingfromcharcode
undefined
undefined

Reputation: 6463

Array.join inserts extra characters when joining strings created with String.fromCharCode

I have an array of bytes that I would to to encode to a string to pass to btoa in the browser. These bytes use the full 0-255 range. I encountered what at first seemed to be a bug with btoa, but it turns out to be a bug (or at least quite unexpected behavior) with javascript's Array.prototype.join. To illustrate the problem, I'll start with some base64-encoded data:

gACJNqQ0cg==

This can be decoded to a byte array as follows:

atob('gACJNqQ0cg==').split('').map(c => c.charCodeAt(0))
> [128, 0, 137, 54, 164, 52, 114]

Now, you would expect to be able to reverse the operation and get back the original string:

btoa([128, 0, 137, 54, 164, 52, 114].map(String.fromCharCode).join(''))

But instead, you get a much larger string:

gAAAAAEAiQIANgMApAQANAUAcgYA

Upon further investigation, the problem occurs when joining any strings created with String.fromCharCode:

'Hi'.split('').join('').length
> 2
'Hi'.split('').map(c => c.charCodeAt(0))
> [72, 105]
[72, 105].map(String.fromCharCode).join('').length
> 6
//what?

I see this behavior everywhere I have tried it: Chrome (60), Firefox (53), and Node (6.9.4). In the browser, you don't have simple alternatives such as node's new Buffer(array, 'binary').toString('base64') to work around this problem. How can I safely create a string from an array of byte values, which can be passed to btoa?

Upvotes: 0

Views: 577

Answers (2)

akuiper
akuiper

Reputation: 214967

Your code works for me if you specify an arrow function in map instead of directly passing the String.fromCharCode method to it:

console.log(btoa([128, 0, 137, 54, 164, 52, 114].map(x => String.fromCharCode(x)).join('')));

Upvotes: 1

undefined
undefined

Reputation: 6463

I found a simple workaround to this complicated problem: String concatenation. Here's an example:

let str = '';
[128, 0, 137, 54, 164, 52, 114].forEach(c => {
    str += String.fromCharCode(c);
});

str.length
> 7
btoa(str)
> 
).join('').length

It works, and it should perform well enough for strings that aren't huge. But I would love to see a better, cleaner, solution, or even just to understand more fully what's going on here with String.fromCharCode and Array.prototype.join.

Upvotes: 0

Related Questions