CODEWITHSUNDEEP
node.jslocalizationio
ragulka
ragulka

Reputation: 4342

fs.readFileSync seems faster than fs.readFile - is it OK to use for a web app in production?

I know that when developing in node, you should always try to avoid blocking (sync) functions and go with async functions, however, I did a little test to see how they compare.

I need to open a JSON file that contains i18n data (like date and time formats, etc) and pass that data to a class that uses this data to format numbers, etc in my view.

It would be kind of awkward to start wrapping all the class's methods inside callbacks, so if possible, I would use the synchronous version instead.

console.time('one');
console.time('two');
fs.readFile( this.dir + "/" + locale + ".json", function (err, data) {
  if (err) cb( err );
  console.timeEnd('one');
});
var data = fs.readFileSync( this.dir + "/" + locale + ".json" );
console.timeEnd('two');

This results in the following lines in my console:

two: 1ms
one: 159ms

It seems that fs.readFileSync is about 150 times faster than fs.readFile - it takes about 1 ms to load a 50KB JSON file (minified). All my JSON files are around 50-100KB.

I was also thinking maybe somehow memoizing or saving this JSON data to the session so that the file is read-only once per session (or when the user changes their locale). I'm not entirely sure how to do that, it's just an idea.

Is it okay to use fs.readFileSync in my case or will I get in trouble later?

Upvotes: 51

Views: 38188

Answers (5)

Peter Lyons
Peter Lyons

Reputation: 146014

No, it is not OK to use a blocking API call in a node server as you describe. Your site's responsiveness to many concurrent connections will take a huge hit. It's also just blatantly violating the #1 principle of node.

The key to node working is that while it is waiting on IO, it is doing CPU/memory processing at the same time. This requires asynchronous calls exclusively. So if you have 100 clients reading 100 JSON files, node can ask the OS to read those 100 files but while waiting for the OS to return the file data when it is available, node can be processing other aspects of those 100 network requests. If you have a single synchronous call, ALL of your client processing stops entirely while that operation completes. So client number 100's connection waits with no processing whatsoever while you read files for clients 1, 2, 3, 4, and so on sequentially. This is Failville.

Here's another analogy. If you went to a restaurant and were the only customer, you would probably get faster service if a single person sat you, took your order, cooked it, served it to you, and handled the bill without the coordination overhead of dealing with the host/hostess, server, head chef, line cooks, cashiers, etc. However, with 100 customers in the restaurant, the extra coordination means things happen in parallel and the overall responsiveness of the restaurant is increased way beyond what it would be if a single person were trying to handle 100 customers on their own.

Upvotes: 84

Anupam Maurya
Anupam Maurya

Reputation: 2211

Yes, it's correct, to deal with the asynchronous way in a server-side environment. But if their use case is different like to generating the build as in client-side JS project, meanwhile reading and writing the JSON files for different flavors.

It doesn't affect that much. Although we needed a rapid manner to create a minified build for deployment (here synchronous comes into the picture). for more info and library

Upvotes: 1

buuuudzik
buuuudzik

Reputation: 11

After a lot of time and a lot of learn & practice I've tried once more and I've found the answer and I can show some example:

const fs = require('fs');

const syncTest = () => {
    let startTime = +new Date();
    const results = [];
    const files = [];

    for (let i=0, len=4; i<len; i++) {
        files.push(fs.readFileSync(`file-${i}.txt`));
    };

    for (let i=0, len=360; i<len; i++) results.push(Math.sin(i), Math.cos(i));
    console.log(`Sync version: ${+new Date() - startTime}`);
};

const asyncTest = () => {
    let startTime = +new Date();
    const results = [];
    const files = [];

    for (let i=0, len=4; i<len; i++) {
        fs.readFile(`file-${i}.txt`, file => files.push(file));
    };

    for (let i=0, len=360; i<len; i++) results.push(Math.sin(i), Math.cos(i));

    console.log(`Async version: ${+new Date() - startTime}`);
};

syncTest();
asyncTest();

Upvotes: 1

buuuudzik
buuuudzik

Reputation: 11

I've tried to check the real, measurable difference in a speed between fs.readFileSync() and fs.readFile() for downloading 3 different files which are on SD card and I've added between this downloads some math calculation and I don't understand where is the difference in speed which is always showed on node pictures when node is faster also in simple operation like downloading 3 times the same file and the time for this operation is close to time which is needed for downloading 1 time this file.

I understand that this is no doubtly useful that server during downloading some file is able to doing other job but a lot of time on youtube or in books there are some diagrams which are not precise because when you have a situation like below async node is slower then sync in reading small files(like below: 85kB, 170kB, 255kB).

var fs = require('fs');

var startMeasureTime = () => {
  var start = new Date().getTime();
  return start;
};

// synch version
console.log('Start');
var start = startMeasureTime();

for (var i = 1; i<=3; i++) {
  var fileName = `Lorem-${i}.txt`;
  var fileContents = fs.readFileSync(fileName);
  console.log(`File ${1} was downloaded(${fileContents.length/1000}KB) after ${new Date().getTime() - start}ms from start.`);

  if (i === 1) {
    var hardMath = 3*54/25*35/46*255/34/9*54/25*35/46*255/34/9*54/25*35/46*255/34/9*54/25*35/46*255/34/9*54/25*35/46*255/34/9;  
  };
};

// asynch version
setImmediate(() => {
  console.log('Start');
  var start = startMeasureTime();

  for (var i = 1; i<=3; i++) {
    var fileName = `Lorem-${i}.txt`;
    fs.readFile(fileName, {encoding: 'utf8'}, (err, fileContents) => {
      console.log(`File ${1} was downloaded(${fileContents.length/1000}KB) after ${new Date().getTime() - start}ms from start.`);
    });

    if (i === 1) {
      var hardMath = 3*54/25*35/46*255/34/9*54/25*35/46*255/34/9*54/25*35/46*255/34/9*54/25*35/46*255/34/9*54/25*35/46*255/34/9;  
    };
  };
});

This is from console:
Start
File 1 was downloaded(255.024KB) after 2ms from start.
File 1 was downloaded(170.016KB) after 5ms from start.
File 1 was downloaded(85.008KB) after 6ms from start.
Start
File 1 was downloaded(255.024KB) after 10ms from start.
File 1 was downloaded(85.008KB) after 11ms from start.
File 1 was downloaded(170.016KB) after 12ms from start.

Upvotes: -1

lab419
lab419

Reputation: 1258

You are blocking the callback of the asynchronous read with your synchronous read, remember single thread. Now I understand that the time difference is still amazing, but you should try with a file that is much, much longer to read and imagine that many, many clients will do the same, only then the overhead will pay off. That should answer your question, yes you will run into trouble if you are serving thousands of requests with blocking IO.

Upvotes: 12

Related Questions