CODEWITHSUNDEEP
pythonredis
SangminKim
SangminKim

Reputation: 9136

When is redis pipeline exceute() method blocked?

I implement a Redis wrapper class like below.

import redis
import threading
import time

class IntervalRedis(object):
    @classmethod
    def init(cls, interval=0.1, host='localhost', port=6379, db=0):
        cls._r = redis.Redis(host=host, port=port, db=db)
        cls.r = cls._r.pipeline()
        cls.t = threading.Thread(target=cls._intervalExecute)
        cls.interval = interval

        cls.t.start()  


    @classmethod
    def _intervalExecute(cls):

        while True:
            print "1"
            cls.r.execute()     # blocked at here after some loop cycle.
            print "2"
            time.sleep(cls.interval)
            print "3"

if __name__ == "__main__":
    print "start"
    IntervalRedis.init()
    count = 0

    while True:
        count+=1
        IntervalRedis.r.lpush("foo", count)
        time.sleep(0.01)
        if count == 10000000:
            break;

    print "end"

In the main statement, it accesses redis pipline object and run lpush commands, looping while statement.

But real execution on the commands is run in _intervalExecute at regular interval with pipline execute().

I think it Must run by when count reachs 10000000.

However, When I run this code, it is blocked after some loop cycle. (sometimes 3 cycles, sometimes 5 cycles randomly)

Could you suggest me?

Upvotes: 0

Views: 913

Answers (1)

Didier Spezia
Didier Spezia

Reputation: 73216

It is not safe to access the same Redis connection from different threads. Each thread is supposed to have its own connection to play with (or use a pool of connections).

In your example, you have two threads sharing the same pipeline and connection.

Upvotes: 3

Related Questions