CODEWITHSUNDEEP
javascriptnode.jstypescriptcsvpapaparse
redlightfilms
redlightfilms

Reputation: 233

How can I use papaparse on Node JS?

I have been trying to get papaparse running on nodejs for a very long time now and I keep failing. I want to try to load my local CSV and then give this papaparse? How to do that? My code does not work.

import papa from "papaparse";
import fs from "fs";

export const convertCSV = async (res: Response) => {
  const file = await fs.createReadStream("../files/test.csv");
  papa.parse(file, {
    header: true,
    complete: function (results, file) {
      console.log("Complete", results.data.length, "records.");
    },
  });
};

results.data.length is always 0.

My CSV file is located in the files folder, which is located in the src folder:

src/files/test.csv

Upvotes: 4

Views: 6861

Answers (5)

user1527225
user1527225

Reputation: 1089

To get streaming working on Nodejs on a Mac, this worked for me:

Usage: % node this-file.mjs < csv-file.csv

/* Documentation
   https://www.papaparse.com/docs#config
   https://github.com/mholt/PapaParse/blob/master/README.md#papa-parse-for-node
*/

import Papa from 'papaparse'

Papa.parse(process.stdin, {
  header: true,
  step: function(results) {
    process.stdout.write(`${JSON.stringify(results.data)}\n`)
  },
})

Upvotes: 0

Colin Cheung
Colin Cheung

Reputation: 250

Ran into this issue with papa-parse, the docs weren't very clear. Inspired by Toshino's answer

import Papa from 'papaparse';
import { Readable } from 'stream';

export const parseCsvFromReadable = async (
  readable: Readable,
): Promise<Papa.ParseResult<unknown>> => {
  return new Promise((resolve, reject) => {
    Papa.parse(readable, {
      complete: results => {
        if (results.errors.length > 0) {
          reject(new Error(JSON.stringify(results.errors)));
        }
        resolve(results);
      },
      error: error => reject(error),
      header: true,
    });
  });
};

Upvotes: 1

Toshinou Kyouko
Toshinou Kyouko

Reputation: 334

As Erick says in their comment, the download option is not supported by Node. However, it does tell us the Node interface accepts a ReadableStream instead of a File.

This works for me:

Papa.parse(fs.createReadStream("sample/result.csv"), {
    complete: function(results) {
        console.log(results["data"]);
    }
});

Upvotes: 2

Erick Petrucelli
Erick Petrucelli

Reputation: 14892

From the Papaparse README, it supports Node in this way:

Papa Parse can parse a Readable Stream instead of a File when used in Node.js environments (in addition to plain strings). In this mode, encoding must, if specified, be a Node-supported character encoding. The Papa.LocalChunkSize, Papa.RemoteChunkSize, download, withCredentials and worker config options are unavailable.

Papa Parse can also parse in a node streaming style which makes .pipe available. Simply pipe the Readable Stream to the stream returned from Papa.parse(Papa.NODE_STREAM_INPUT, options). The Papa.LocalChunkSize, Papa.RemoteChunkSize, download, withCredentials, worker, step and complete config options are unavailable. To register a callback with the stream to process data, use the 'data' event like so: stream.on('data', callback) and to signal the end of stream, use the 'end' event like so: stream.on('end', callback).

Upvotes: 3

Nathan Mc Grath
Nathan Mc Grath

Reputation: 308

Your code is fine, the problem is in the file path. fs.createReadStream needs an absolute path, or a path that's relative to the root of your project.

Assuming this is the project structure:

files/
  - test.csv
src/
  - convert-csv.ts
package.json
tsconfig.json

Then you can use a path that's relative to the root: './files/test.csv', or if you really want to use a path that's relative to the file then you can use path.resolve(__dirname, '..', 'files', 'test.csv').

See NodeJS docs for path.resolve, it will use the path segments to resolve an absolute path for you.

Upvotes: 1

Related Questions