CODEWITHSUNDEEP
redissynchronization
Lorenzo Belli
Lorenzo Belli

Reputation: 1847

How to synchronise multiple writer on redis?

I have multiple writers overwriting the same key in redis. How do I guarantee that only the chosen one write last?

Can I perform write synchronisation in Redis withour synchronise the writers first?

Background: In my system a unique dispatcher send works to do to various workers. Each worker then write the result in Redis overwrite the same key. I need to be sure that only the last worker that receive work from the dispatcher writes in Redis. 

Upvotes: 5

Views: 8201

Answers (3)

Lorenzo Belli
Lorenzo Belli

Reputation: 1847

Use an ordered set (ZSET): add your entry with a score equal to the unix timestamp, then delete all but the top rank.

A Redis Ordered set is a set, where each entry also has a score. The set is ordered according to the score, and the position of an element in the ordered set is called Rank.

In order:

  1. Remove all the entries with score equal or less then the one you are adding(zremrangebyscore). Since you are adding to a set, in case your value is duplicate your new entry would be ignored, you want instead to keep the entry with highest rank. 
  2. Add your value to the zset (zadd)
  3. delete by rank all the entries but the one with HIGHEST rank (zremrangebyrank)
  4. You should do it inside a transaction (pipeline)

Example in python:

# timestamp contains the time when the dispatcher sent a message to this worker
key = "key_zset:%s"%id
pipeline = self._redis_connection.db.pipeline(transaction=True)
pipeline.zremrangebyscore(key, 0, t)  # Avoid duplicate Scores and identical data
pipeline.zadd(key, t, "value")
pipeline.zremrangebyrank(key, 0, -2)
pipeline.execute(raise_on_error=True)

Upvotes: 4

Heera Jaiswal
Heera Jaiswal

Reputation: 771

You can use redis pipeline with Transaction.

Redis is single threaded server. Server will execute commands syncronously. When Pipeline with transaction is used, server will execute all commands in pipeline atomically.

Transactions

MULTI, EXEC, DISCARD and WATCH are the foundation of transactions in Redis. They allow the execution of a group of commands in a single step, with two important guarantees: All the commands in a transaction are serialized and executed sequentially. It can never happen that a request issued by another client is served in the middle of the execution of a Redis transaction. This guarantees that the commands are executed as a single isolated operation.

A simple example in python

with redis_client.pipeline(transaction=True) as pipe:
    val = int(pipe.get("mykey"))
    val = val*val%10
    pipe.set("mykey",val)
    pipe.execute()

Upvotes: 0

Tuan Anh Tran
Tuan Anh Tran

Reputation: 7237

If I were you, I would use redlock.

Before you write to that key, you acquire the lock for it, then update it and then release the lock.

I use Node.js so it would look something like this, not actually correct code but you get the idea.

Promise.all(startPromises)
        .bind(this)
        .then(acquireLock)
        .then(withLock)
        .then(releaseLock)
        .catch(handleErr)


function acquireLock(key) {
    return redis.rl.lock(`locks:${key}`, 3000)
}

function withLock(lock) {
    this.lock = lock
    // do stuff here after get the lock
}

function releaseLock() {
    this.lock.unlock()
}

Upvotes: 0

Related Questions