CODEWITHSUNDEEP
javascriptnode.jsasynchronous
Numant Santz
Numant Santz

Reputation: 180

Array doesnt have expected value - Nodejs

I am running a script which looks into a directory and lists files, then checks for the file type to process, if the extension matches then the file is read and each line of the file (.col which is just a txt file renamed) is inserted into an array.

Now after the file is read and the array is populated I would like to use the array and do some further processing, e.g create a db record. I am missing something really basic here because on each console log I do as below I always get the full items (in my array) of the contents of all files.

So to make it a bit simpler: array is empty. Then file is read and processed and array now has array[0]=line 0 of file array[0]=line 1 of file etc

    const fs = require('fs');
const readline =require('readline');
var files = fs.readdirSync('/home/proj/data');
var path = require('path');
var model=[];
var lineReader=[];

for(var i=0; i<files.length; i++) {

    if(path.extname(files[i]) === ".col") {
       lineReader[i] = readline.createInterface({
            input: require('fs').createReadStream(files[i])
         });  

        lineReader[i].on('line', function (line) {  
          model.push(line); 
        }).on('close', async function() {
          console.log(model); 
      });

}

}

Instead the script is run and array[] holds all lines of all files that match the extension.

Your help is greatly appreciated and anyone is allowed to scorch my JS as I am pretty sure I am missing something basic here.

Upvotes: 0

Views: 148

Answers (2)

vsemozhebuty
vsemozhebuty

Reputation: 13792

If your files are not very big and sync methods are OK, you can simplify the code this way:

'use strict';

const fs = require('fs');
const path = require('path');

const model = [];

fs.readdirSync('/home/proj/data')
  .filter(name => path.extname(name) === '.col')
  .forEach((name) => {
    model.push(...fs.readFileSync(name, 'utf8').split('\n'));
  });

console.log(model);

Upvotes: 1

Aritra Chakraborty
Aritra Chakraborty

Reputation: 12542

So, you want to read the files in parallel (because that's what your program does) and put it in an array of arrays? You can make the reading file mechanism a promise and use it using Promise.all. Here is an example to get you started.

const fs = require('fs');
const readline = require('readline');
var files = fs.readdirSync('./');
var path = require('path');

function readFile(fileName) {
    return new Promise(resolve => {
        const array = [];
        const lineReader = readline.createInterface({
            input: fs.createReadStream(files[i])
        });
        lineReader.on('line', function (line) {
            array.push(line);
        }).on('close', async function () {
            //do some proc
            console.log(array);
            resolve(array);
        });
    });
}
const readFilePromises = [];
for (var i = 0; i < files.length; i++) {
    if (path.extname(files[i]) === ".js") {
        readFilePromises.push(readFile(files[i]));
    }
}

Promise.all(readFilePromises) //or await Promise.all([..])
    .then(data => {
        console.log(data);//will be array of arrays
    })

If you want a single Array you can always flatten the result using data.flat()

Upvotes: 2

Related Questions