CODEWITHSUNDEEP
.net-coreredistransactionslockingstackexchange.redis
Bercovici Adrian
Bercovici Adrian

Reputation: 9380

How to make multiple operations atomic using StackExchange.Redis

Hello i have the following problem:

I have a hash that contains strings.This hash will get queried by multiple users.

When a user comes with a Key to first check if it exists in this hash and if it does not, add it.

How can I make the operations "check if hash exists", "add if not exists" atomic? Reading the redis documentation it seems Watch is what I need. Basically start a transaction and end it if the variable changes.

I have tried using Condition.HashNotExists to no avail:

class Program {

        public static async Task<bool> LockFileForEditAsync(int fileId) {
            var database = ConnectionMultiplexer.Connect(CON).GetDatabase();
            var exists = await database.HashExistsAsync("files", fileId); //this line is for  shorting the transaction if hash exists 

            if (exists) {
                return false;
            }

            var tran = database.CreateTransaction();
            tran.AddCondition(Condition.HashNotExists("files", fileId));
            var setKey = tran.HashSetAsync("files", new HashEntry[] { new HashEntry(fileId, 1) });
            var existsTsc = tran.HashExistsAsync("files", fileId);

            if (!await tran.ExecuteAsync()) {
                return false;
            }

            var rezult = await existsTsc;
            return rezult;
        }

        public const string CON = "127.0.0.1:6379,ssl=False,allowAdmin=True,abortConnect=False,defaultDatabase=0";

        static async Task Main(string[] args) {
            int fid = 1;
            var locked = await LockFileForEditAsync(fid);
        }
    }

If I connect via redis-cli and issue in the cli : hset files {fileId} 1 , right BEFORE I issue the ExecuteAsync (in the debugger) I am expecting this transaction to fail since I placed the Condition. However it does not happen so.

How can I basically use redis commands place something like a lock on the two operations:

  1. Check if hashentry exists
  2. Add hashentry

Upvotes: 1

Views: 3162

Answers (1)

LeoMurillo
LeoMurillo

Reputation: 6774

Bad news, good news, and for everything else...

Bad news

This doesn't work because SE.Redis pipelines the transaction. This means all the transaction commands are sent to the server simultaneously when ExecuteAsync() is called. However, conditions are evaluated first.

tran.AddCondition(Condition.HashNotExists("files", fileId)); translates to:

WATCH "files"
HEXISTS "files" "1"

And these two commands are sent first when ExecuteAsync() is called to evaluate the condition. If the condition holds true (HEXISTS "files" "1" = 0) then the rest of the transaction commands are sent.

This effectively ensures no false positives, because if the files key is modified in between (while SE.Redis is evaluating the condition), the WATCH will make the transaction fail.

The problem is the false negatives. For example, the transaction will also fail if another field of the hash was set while SE.Redis is evaluating the condition. The WATCH "files" makes it so.

I tested this by having a redis-benchmark -c 5 HSET files 2 1 running when ExecuteAsync() was called. The condition passed, but the transaction failed although the field "1" did not exist, because the field "2" was set in between.

I verified using the MONITOR command in a separate redis-cli window. This is handy to troubleshoot expectations not met or simply to see what is really going to the server and when.

A WATCH is not helpful when you care about a field of a hash, as it would create false negatives when other fields are touched.

A solution to this would be to use a regular key instead (files:1).

Good news

There is a command to do exactly what you want: HSETNX.

This simplifies your LockFileForEditAsync() completely:

    public static async Task<bool> LockFileForEditAsync(int fileId)
    {
        var database = ConnectionMultiplexer.Connect(CON).GetDatabase();
        var setKey = await database.HashSetAsync("files", fileId, 1, When.NotExists);
        return setKey;
    }

Notice the When.NotExists parameter. It results in HSETNX "files" "1" "1" command being sent.

For everything else...

You may use Lua scripts in situations like this, where you want to have some conditional action done atomically, like in how to use spop command with count if set have that much (count) element in set.

It seems like you are trying to do a distributed lock. See Distributed locks with Redis for other aspects you may want to consider. See What is distributed Atomic lock in caches drivers? for a nice story on this one.

Upvotes: 1

Related Questions