CODEWITHSUNDEEP
javascriptnode.jsstringutf-8
TKF
TKF

Reputation: 61

Browser buffer to string conversion is not same in browser and nodejs

I have encountered an interesting issue. I'm using node v8.1.4

I have the following buffer. 

[ 191, 164, 235, 131, 30, 28, 164, 179, 101, 138, 94, 36, 115, 176, 83, 193, 9, 177, 85, 228, 189, 193, 127, 71, 165, 16, 211, 132, 228, 241, 57, 207, 254, 152, 122, 98, 100, 71, 67, 100, 29, 218, 165, 101, 25, 17, 177, 173, 92, 173, 162, 186, 198, 1, 80, 94, 228, 165, 124, 171, 78, 49, 145, 158 ]

When i try to convert it to utf8 using nodejs and using browser i get different results. even length of string is not the same.

Is there a way to convert string to utf8 in browser same way as node js do?

It seems that some characters that some sequence which nodejs replace to U+FFFD are more lengthy than the replaced sequence in browser. so output utf8 string is different

Code i use in browser and in nodejs is same i have buffer object tmpString

  tmpString.toString('utf-8')

tmpString.toString('utf-8').length differs in browser and nodejs for the same source bytes.

In nodejs i use native buffer implementation, for browser webpack loads polyfill (feross/buffer i think)

i think more accurately would say that i try to interpret buffer bytes as UTF8 string.

Upvotes: 0

Views: 4248

Answers (2)

snnsnn
snnsnn

Reputation: 13600

If you are reading a file and sending its content directly on the server side via nodejs:

const content = await readFile(fullpath);
socket.clients.forEach(ws => ws.send(content));

You will get an array buffer as shown below:

{ type: 'Buffer', data: [35, 32, ...] }

This could be converted into string like so:

const encoder = new TextDecoder("utf-8");
const array = new Uint8Array(value.data);
const textContent = encoder.decode(array);

Upvotes: 0

jconder
jconder

Reputation: 746

Have you tried the TextEncoder/TextDecoder APIs? I've used them for converting strings in both nodejs and the browser and haven't seen any differences.

E.g.:

const encoder = new TextEncoder('utf-8');
const decoder = new TextDecoder('utf-8');

const foo = 'Hello world!';
const encoded = encoder.encode(foo);
console.log(encoded);

const decoded = decoder.decode(encoded);
console.log(decoded);

Upvotes: 7

Related Questions