CODEWITHSUNDEEP
javascriptnode.jsfile-io
Ark-kun
Ark-kun

Reputation: 6787

How to read a text file line by line in JavaScript?

I need to read a text file line by line in JavaScript.

I might want to do something with each line (e.g. skip or modify it) and write the line to another file. But the specific actions are out of the scope of this question.

There are many questions with similar wording, but most actually read the whole file into memory in one step instead of reading line by line. So those solutions are unusable for bigger files.

Upvotes: 17

Views: 16751

Answers (3)

bobbyg603
bobbyg603

Reputation: 3840

Here's a function from the MDN docs that shows you how to use a ReadableStream to read a File line-by-line. This is compatible with both node.js and web browsers. This example uses fetch, but if you already have a File, you can call stream() and getReader() instead.

async function* makeTextFileLineIterator(fileURL) {
  const utf8Decoder = new TextDecoder("utf-8");
  let response = await fetch(fileURL);
  let reader = response.body.getReader();
  let { value: chunk, done: readerDone } = await reader.read();
  chunk = chunk ? utf8Decoder.decode(chunk, { stream: true }) : "";

  const re = /\r\n|\n|\r/gm;
  let startIndex = 0;

  for (;;) {
    const result = re.exec(chunk);
    if (!result) {
      if (readerDone) {
        break;
      }
      const remainder = chunk.substr(startIndex);
      ({ value: chunk, done: readerDone } = await reader.read());
      chunk =
        remainder + (chunk ? utf8Decoder.decode(chunk, { stream: true }) : "");
      startIndex = re.lastIndex = 0;
      continue;
    }
    yield chunk.substring(startIndex, result.index);
    startIndex = re.lastIndex;
  }
  if (startIndex < chunk.length) {
    // last line didn't end in a newline char
    yield chunk.substr(startIndex);
  }
}

for await (let line of makeTextFileLineIterator(urlOfFile)) {
  processLine(line);
}

Upvotes: 2

Larry
Larry

Reputation: 678

With Node.js a new function was added in v18.11.0 to read files line by line

  • filehandle.readLines([options])

This is how you use this with a text file you want to read

import { open } from 'node:fs/promises';
myFileReader();
async function myFileReader() {
    const file = await open('./TextFileName.txt');
    for await (const line of file.readLines()) {
        console.log(line)
    }
}

To understand more read Node.js documentation here is the link for file system readlines(): https://nodejs.org/api/fs.html#filehandlereadlinesoptions

Upvotes: 25

Ark-kun
Ark-kun

Reputation: 6787

The code to read a text file line by line is indeed surprisingly non-trivial and hard to discover.

This code uses NodeJS' readline module to read and write text file line by line. It can work on big files.

const fs = require("fs");
const readline = require("readline");

const input_path = "input.txt";
const output_path = "output.txt";

const inputStream = fs.createReadStream(input_path);
const outputStream = fs.createWriteStream(output_path, { encoding: "utf8" });
var lineReader = readline.createInterface({
  input: inputStream,
  terminal: false,
});
lineReader.on("line", function (line) {
  outputStream.write(line + "\n");
});

Upvotes: 14

Related Questions