NodeJS: how to read (at most) the first N bytes from a file?

Question

In NodeJS, what is a concise, robust, and elegant way to read at most the first N bytes from a file?

• If there's less data then I don't want an error to be thrown.
• If there's more data then I don't want that to be read into memory.

Ideally without installing external packages. Maybe involving async iterators which seem to be new (non-experimental) since NodeJS 12?

Bonus: how do I read the first N characters, providing a specific codec (such as utf-8)?

Edit: based on my Internet research the answer to this indeed rather basic question is far from obvious, which is why I ask. I know how to do this in other language environments, and I am new to the NodeJS ecosystem.

Answer 1

Since v17.5.0 and v16.15.0, you can easily do it using iterator helper methods.

Example:

import fs from 'node:fs';

// Gets an array of byte values
const first8Bytes = await fs.createReadStream(filepath)
  .flatMap(chunk => new Uint8Array(chunk))
  .take(8)
  .toArray();

if (first8Bytes.length < 8) {
  throw new Error("The file has less than 8 bytes.");
}

// Read as a single 64-bit signed int
const number = new BigInt64Array(first8Bytes)[0];

⚠ Note: The methods .flatMap(), .take(), and .toArray() are still experimental as of the date of this answer.

Answer 2

I have been using the following approach for a while now in NodeJS 12 (with TypeScript):

async function readFirstNBytes(path: fs.PathLike, n: number): Promise<Buffer> {
  const chunks = [];
  for await (let chunk of fs.createReadStream(path, { start: 0, end: n-1 })) {
    chunks.push(chunk);
  }
  return Buffer.concat(chunks);
}

Answer 3

I think you need to work with readable streams and read event.

Attach a handler and keep checking the chunk.length compared to your accumulated read to determine if you should continue.

Will read the max that is stipulated, or the minimum considering the file length.

Demo to get you thinking:

const fs = require('fs');
const path = require('path');
const Promise = require('bluebird');

peek(path.resolve('./random/stream.js'), 100, 100)
    .then(result => {
        console.info(result.length);
    });

function peek(filePath, size, max, encoding) {
    return new Promise((resolve, reject) => {
        const stream = fs.createReadStream(filePath, {
            encoding: encoding || 'utf8'
        });

        let length = 0;
        let content = '';
        stream.on('readable', () => {
            let chunk;
            while (length < max && (chunk = stream.read(size)) !== null) {
                length += chunk.length;
                content += chunk.toString();

                if ((length + size) >= max) {
                    size = max - length;
                }
            }

            return resolve({
                length: length,
                content: content
            });
        });
    });
}