CODEWITHSUNDEEP
pythonperformanceredis
user3446151
user3446151

Reputation: 169

Redis pipelines suitable for real time get set?

I have several keys setting up with real time values. Since it is said pipelines are much faster, is it really suitable to run

r= redis.StrictRedis()
pipe = r.pipeline()
for i in range(100000):
    pipe.set(i, i+1).execute()

There are 1 lakh keys with the values changing every second. I'll require to run execute() because I'll be retrieving the data parallely in real time too.

Is it suitable to use pipelines in this case in terms of performance and CPU load?

Upvotes: 2

Views: 2774

Answers (1)

Tw Bert
Tw Bert

Reputation: 3809

You are calling execute on each iteration. So you are not using the pipeline at all, only adding overhead.

If you batch the execute() bundled per 5000 items (for example), yes, this is suitable and very fast.

Keep in mind that StrictRedis.pipeline is no real pipeline. It's a client-side queue, which uses the server-side pipeline interface at execute(). See here if interested.

Hope this helps, TW

Upvotes: 1

Related Questions