CODEWITHSUNDEEP
node.jsbinary
arslan
arslan

Reputation: 2224

node js, how to write/read integers to binary file?

I am new to this, so might be asking a basic question. I have an array of floats

[1.3, 2.0, 10, 9.8, ... ] . //let's say length = 10

which I want to store in a binary file. I want to write in the following format

 10 1.3 2.0 10 9.8, ...

so that when i load that file later, I first read the length 10, then read floats 10 times.

I think that saves space and should be faster than just writing/reading it as text.

So far, I found a bit info about binary file in node.js.

  var fs = require('fs');
  var wstream = fs.createWriteStream('myBinaryFile');
  var buffer = crypto.randomBytes(100);
  wstream.write(buffer);
  wstream.end();

where I did not know how I can write my numbers into the buffer and write the buffer to file.

Also, I read some documents that says buffer in node only have integer 0~255, so does that mean, it cannot handle integers larger than 255?

For example, How can I write/read an array 

  [11000, 24535, 92833, 45353, ... ]

to/from a binary file.

I think I need a bit guidance, so here I am seeking for help. Please point out what i should do, thanks!

Upvotes: 2

Views: 5679

Answers (2)

Robel Robel Lingstuyl
Robel Robel Lingstuyl

Reputation: 1591

To be a bit more specific here is an answer to what you are looking to accomplish.

    const values = [0, 4764, 9243, 13167, 16303, 18461, 19513];

    var wstream = fs.createWriteStream(testDatFile, { encoding: 'binary' });
    wstream.on('finish', function() {
      console.log('file has been written');
    });

    console.log('buffer.from', Buffer.from(Int16Array.from(values).buffer));
    wstream.write(Buffer.from(Int16Array.from(values).buffer));
    wstream.end()

Upvotes: 1

slebetman
slebetman

Reputation: 113876

You need to use a Buffer (or the standard Uint8Array but since this is node.js I'll use Buffer). You can basically figure out what you need to do by reading the docs but I'll describe what you need to do to give you a head start.

First there are several decisions you need to make.

  1. Do you want your integer to be 8 bits or 16 bits or 32 bits or 48 bits? (yes, node supports 48 bit ints but not 64 - probably because numbers in js are doubles with 52 bit mantissa so a 64 bit int can't be accurately converted to a number). Also, if you want floating point numbers do you want floats (32 bits) or doubles (64 bits)?

  2. Do you want to store the number on disk as big endian or little endian? FWIW, programmers tend to prefer little endian and hardware designers tend to prefer big endian. Doesn't really matter but be consistent. Don't mix endianness in a single file.

Now that you've made up your mind let's do an example. Say for example you want to store two 32 bit ints and one double in a file. The space you'd need is 32 + 32 + 64 = 128 bits = 16 bytes. So you need a 16 byte Buffer (yes, buffers operate at byte level, you're basically temporarily promoted to be a C programmer):

var b = Buffer.alloc(16);

Now let's store those numbers. Let's say you've decided to store them as little endian:

var x = 12345;
var y = 54321;
var z = 1.234;

b.writeInt32LE(x,0);  // write x to bytes 0-3
b.writeInt32LE(y,4);  // write y to bytes 4-7
b.writeDoubleLE(z,8); // write z to end of buffer

Now you can write the value of the buffer b to file using any of the fs write functions. Note that you should set the encoding to "binary" because the default encoding of fs functions is utf8 which may try to parse your data and corrupt it.

To read the values from file you'd do the same in reverse. Use the buffer.read... functions. Again, be sure to set the encoding to "binary" (or was it "buffer"?) when reading the file from disk.

Upvotes: 6

Related Questions