Reputation: 23
If I receive a UTF-8 string via a socket (or for that matter via any external source) I would like to get it as a properly parsed string object. The following code shows what I mean
var str='21\r\nJust a demo string \xC3\xA4\xC3\xA8-should not be anymore parsed';
// Find CRLF
var i=str.indexOf('\r\n');
// Parse size up until CRLF
var x=parseInt(str.slice(0, i));
// Read size bytes
var s=str.substr(i+2, x)
console.log(s);
This code should print
Just a demo string äè
but as the UTF-8 data is not properly parsed it only parses it up to the first Unicode character
Just a demo string ä
Would anyone have an idea how to convert this properly?
Upvotes: 2
Views: 12173
Reputation: 149734
You should use utf8.js which is available on npm.
var utf8 = require('utf8');
var encoded = '21\r\nJust a demo string \xC3\xA4\xC3\xA8-foo bar baz';
var decoded = utf8.decode(encoded);
console.log(decoded);
Upvotes: 1
Reputation: 36
It seems you could use this decodeURIComponent(escape(str))
:
var badstr='21\r\nJust a demo string \xC3\xA4\xC3\xA8-should not be anymore parsed';
var str=decodeURIComponent(escape(badstr));
// Find CRLF
var i=str.indexOf('\r\n');
// Parse size up until CRLF
var x=parseInt(str.slice(0, i));
// Read size bytes
var s=str.substr(i+2, x)
console.log(s);
BTW, this kind of issue occurs when you mix UTF-8 and other types of enconding. You should check that as well.
Upvotes: 2