CODEWITHSUNDEEP
node.jscsv
user5797064
user5797064

Reputation:

node js having trouble with parsing of csv file useing fast-csv libary

I'm using the fast-csv node library to parse a csv-file that looks like this

var stream = fs.createReadStream('./tmp_dir/'+files.fileUploaded.name);

var csvStream = csv()
    .on("data", function(data){
         console.log(data);
    })
    .on("end", function(){
         console.log("done");
    });

stream.pipe(csvStream);

I wonder if it is possible to do the following things

Upvotes: 0

Views: 2034

Answers (2)

zoubida13
zoubida13

Reputation: 1759

You put the link to the library github but you didn't read the doc!

It's all in here !

var stream = fs.createReadStream('./tmp_dir/'+files.fileUploaded.name);
var errorCount = 0;
var csvStream = csv.parse({strictColumnHandling: true, headers: true})
.on("data", function(data){
     console.log(data);
}).on("data-invalid", function(){
     console.log("error");
     errorCount += 1;
})    
.on("end", function(){
     console.log("done");
});

stream.pipe(csvStream);

For the timing you can just get the date when you read the first line (use a counter or a toggle isFirstLine that you set to false after read) and when you hit the finish event, and you'll have timing.

Upvotes: 1

Soubhik Mondal
Soubhik Mondal

Reputation: 2666

1

To see how many rows in the csv file that could not be parsed?

In the docs, it is mentioned that the event data-invalid is emitted when invalid row-data is encountered. I'm not sure but perhaps you can use that.

var csvStream = csv()
  .on("data-invalid", function(data){
     // handle the erroneous data
  })
  .on("data", function(data){
     console.log(data);
  })
  .on("end", function(){
     console.log("done");
  });

2

Record start and end time for parsing of the file?

You could try this:

var startTime = (new Date()).getTime();

var csvStream = csv()
  .on("end", function(){
     console.log("done");

     var endTime = (new Date()).getTime();

     console.log("Time taken : ", (endTime - startTime));

  });

Upvotes: 0

Related Questions