CODEWITHSUNDEEP
node.jsunixconcurrencylocking
Alexander Mills
Alexander Mills

Reputation: 100010

Locking a file for read/write with Node.js

I am looking for the ability to do the following sequence of events:

I see the value in merely writing atomically, but are there any suggestions for

1. locking a file
2. reading from it
3. writing to it
4. releasing the lock

how do we do this (with Node.js)? Let's say this file stores simple JSON data.

Upvotes: 3

Views: 5484

Answers (2)

Simon Rigét
Simon Rigét

Reputation: 2895

You must ensure "atomic locks" From inside node, you can use symlinks to achieve this. Here is an example of a simple counter:

const fs = require('fs');

// the lock function must be a recursive timer
const flock = (file_name, resolve, reject) => {
  fs.symlink(file_name, 'locked_' + file_name, (err) => {
    if(err)
      if(err.code == 'EEXIST')
        setTimeout(() => {
          flock(file_name, resolve, reject)
        }, 13);
      else
        reject(err);
    else
      resolve();
  });
}

const counter_increment = async () => {
  file_name = 'counter.txt';

  // Lock file
  await new Promise((resolve, reject) => {
    flock(file_name, resolve, reject);
  });

  // Read file
  let count = await new Promise((resolve, reject) => {
    fs.readFile(file_name, 'utf8', (err, data) => {
      if (err) resolve(0);
      resolve(parseInt(data));
    });
  });

  // Write file
  await new Promise((resolve, reject) => {
    fs.writeFile(file_name,"" + ++count + " ", (err) => {
      if (err) reject(err);
      resolve();
    });
  });

  // Unlock file
  fs.unlink('locked_' + file_name, (err) => {
    if (err) console.error(err);
  });

  return count;
};

counter_increment();

This is slow compared to a real OS implementation of file lock, but it's working for production.

The node file system library is rather immature at this point. (As of node 11) This functionality should have been standard by now, but isn't.

The module fs.ext https://github.com/baudehlo/node-fs-ext/ adds the real OS file locking. Unfortunately there is a bug in the way i handles multiple requests, that makes it unusable in production. Further more, the maintainers don't seem to have the time to fix it.

All other modules I have come across, doesn't do a very good job of it. Even the very popular ones.

Upvotes: 1

Alexander Mills
Alexander Mills

Reputation: 100010

For even this simple use case, I believe you do need a locking mechanism if there are other processes concurrently accessing the file.

I tried out Lockfile and Warlock and was not impressed about their default settings and how difficult it was to fine tune them.

So I wrote a different kind of mutex library called "live-mutex" which allows for no polling and is a completely evented client-broker model. It's main claim to fame is that it's about 15x faster than both lockfile and warlock.

Check it out: Github.com/oresoftware/live-mutex

Upvotes: 1

Related Questions