CODEWITHSUNDEEP
javascriptstringbinarynon-unicode
Kane
Kane

Reputation: 579

Converting non-unicode binary strings to byte array in Javascript

I read a binary file using ajax from a web server. The HTTP response is of contentType: 'application/octet-stream' and contains a binary string that is just a string of bytes (not unicode), for example (in hex):

0x00 0x08 0x17 0xA1 0x01

Note: in C this would be represented as 5 bytes in memory:

char buf[5] = {0, 8, 23, 161, 1}

...but in Javascript that is a string which ASCII representation is something like " � " (I can't actually paste it properly as not all characters have a printable representation).

I now need to convert this into an array of characters or integers so that I can access the numerical value of each character in the string. However iterating through the example string using the charCodeAt() function returns:

[0] 0
[1] 8
[2] 23
[3] 65533
[4] 1

because charCodeAt() decodes unicode characters, and 0xA1 is not recognised as valid unicode character so a Replacement Character (65533) is used instead.

I would like to get the following:

[0] 0
[1] 8
[2] 23
[3] 161
[4] 1

How to achieve this?

Here is the code snippet:

$.ajax({
url: url,
type: "get",
success: function(data) { // data contains binary representation of 0x00 0x08 0x17 0xA1 0x01
    var byteTab = []
    for (var n = 0; n < data.length; ++n) {
        byteTab.push(data.charCodeAt(n))
    }
})

Upvotes: 3

Views: 600

Answers (2)

Kane
Kane

Reputation: 579

Following the suggestion from Haus 's answer I found some more solutions to the problem so I will share my findings here.

  1. My preferred solution is to use ArrayBuffer to access raw data. Unfortunately Ajax does not seem to support it as per this article, although there is an option to add support (which I could not get to work), and there is no straightforward way to create an ArrayBuffer out of string. However XMLHttpRequest supports ArrayBuffer as responseType as explained here so I changed my code to the following, which does what I want:

    var xhr = new XMLHttpRequest();
xhr.open('GET', url);
xhr.responseType = 'arraybuffer';  
xhr.onload = function() {
    if (this.status == 200) {
        var byteTab=[]
        var uint8View = new Uint8Array(this.response)
        for (var n = 0; n < uint8View.byteLength; n++) {
            byteTab.push(uint8View[n])
        }

    } 
};
xhr.send();

  2. Another solution is to get to ArrayBuffer via using Blob, which is explained with an example in this answer

  3. Another option is to use Blob in conjunction with FileReader.readAsBinaryString() along the lines of the example here. Using readAsBinaryString() converts the blob to a unicode string which can then be parsed using charCodeAt()

Upvotes: 1

Haus
Haus

Reputation: 1492

Update: I'm not so sure [default] Ajax is the right tool for the job in this case. Regardless of file size, it is generally a good practice to use streams instead to accommodate future scalability, as your current method loads everything at once.

This article goes over how to create a custom ajax transport that wraps an XmlHttpRequest to load the data into an array buffer rather. You could do this, sequentially push the bytes into your main array, and go from there.

Old: I'm not sure if I properly understood your data representation, but I believe you can just use parseInt() to turn a hex string into decimal:

var data = ['0x00', '0x08', '0x17', '0xA1', '0x01'];
var parsed = [];

for(var i = 0; i < data.length; i++) {
	parsed.push(parseInt(data[i], 16));
}

console.log(parsed);
console.log(parseInt('0xA1', 16))

If this is not what you mean, please comment and I will try to update my answer with a more specific implementation.

Upvotes: 1

Related Questions