CODEWITHSUNDEEP
node.jsredisatomicnode-redis
Lorenzo Borelli
Lorenzo Borelli

Reputation: 127

An alternative to redis transactions for atomicity in node

I have an express webserver that performs some operations, in Redis, which need to be atomic, on a key whose value is a list. More specifically, this is more or less the structure of my code

redisclient.lrange(key, 0, -1 (error, items) => {
    .
    .
    .
    //some slightly complex code that updates the list of strings
    //obtain newItems from items
    .
    .
    .
    redisclient.del(key);
    //push the newly updated list
    for (let item of newItems){
        redisclient.rpush(key,item);
    }
});

The problem here is that, from my understanding, in order to make these operations atomic I would need to use a Lua script. I don't know anything about Lua though, and converting my JS logic in a Lua script wouldn't be trivial.

Node is single-threaded, but isn't there any alternative to avoid race conditions between different clients in a code like this?

Upvotes: 2

Views: 1606

Answers (1)

LeoMurillo
LeoMurillo

Reputation: 6754

You can use a transaction so that the DEL and RPUSH commands are executed atomically. See CLIENT.MULTI([COMMANDS]).

You can add a WATCH for your key if you want the transaction not to execute if your list was modified while you processed it. See OPTIMISTIC LOCKS. But here, you need recovery/retry logic in case it fails.

To use WATCH, you first start watching, then read the list with LRANGE, do your manipulation, then do MULTI, DEL, RPUSH, EXEC. The EXEC will fail if the list was modified between WATCH and EXEC.

client.watch(key, function( err ){
    if(err) throw err;

    client.lrange(key, 0, -1, function(err, result) {
        if(err) throw err;

        // Process the result

        client.multi()
            .del(key)
            .rpush(key, newItems)
            .exec(function(err, results) {

                /**
                 * If err is null, it means Redis successfully attempted 
                 * the operation.
                 */ 
                if(err) throw err;

                /**
                 * If results === null, it means that a concurrent client
                 * changed the key while we were processing it and thus 
                 * the execution of the MULTI command was not performed.
                 * 
                 * NOTICE: Failing an execution of MULTI is not considered
                 * an error. So you will have err === null and results === null
                 */
            });
    });
});

RPUSH in Redis supports multiple elements at once. Consider pushing multiple elements at once by using the [send_command][3] directly, ES6 spread syntax, or passing the array directly (depends on what version are you running).

That said, consider using Lua scripts. Here a little push to get you started:

EVAL "local list1 = redis.call('LRANGE', KEYS[1], 0, -1) for ix,elm in ipairs(list1) do list1[ix] = string.gsub(elm, 'node', 'nodejs') end redis.call('DEL', KEYS[1]) redis.call('RPUSH', KEYS[1], unpack(list1)) return list1" 1 myList

Here a friendly view of the Lua script:

local list1 = redis.call('LRANGE', KEYS[1], 0, -1)
for ix,elm in ipairs(list1) do
    list1[ix] = string.gsub(elm, 'node', 'nodejs')
end
redis.call('DEL', KEYS[1])
redis.call('RPUSH', KEYS[1], unpack(list1))
return list1

This simply replaces node by nodejs in all elements of the list. See more on string manipulation at the String Library Tutorial.

Upvotes: 3

Related Questions