CODEWITHSUNDEEP
node.jsjson
rajat meena
rajat meena

Reputation: 363

How to read large json file in node js?

I have a JSON file about 300MB in size and I'm trying to read it but it's not working.

How can I read a large JSON file? If anyone can guide me with a small piece of code then it would be great.

I have already tried fs.readFile but no luck, it's working fine for smaller files but not for large ones.

Below is the code I have tried so far:

app.get('/getData', function (req, res) {
  fs.readFile('./uploads/test.json', function (err, data) {
      if (err) throw err
      var jsonData = data;
      var jsonParsed = JSON.parse(jsonData);
      res.json(jsonParsed);
    });
});

Upvotes: 0

Views: 3215

Answers (1)

Ashen Gunaratne
Ashen Gunaratne

Reputation: 455

Speculating from the code snippet, I believe there is no requirement to do any modification (i.e. filtering) to the JSON data prior dispatching the response. If that is the case, parsing of the JSON would be unnecessary and Node.js's built-in readable streams could be used to provide more efficiency.

const fs = require('fs');
const path = require('path');
const express = require('express');
const app = express();

app.get('/data', (req, res) => {
  res.setHeader('Content-Type', 'application/json');
  fs.createReadStream(path.join(__dirname, 'public', 'citylots.json')).pipe(res);
});

app.listen(process.env.PORT || 8080);

Upvotes: 3

Related Questions