CODEWITHSUNDEEP
pythonperformanceredis
krishna chaitanya
krishna chaitanya

Reputation: 93

Redis value fetch for 3000 keyhash takes about 10 seconds(python 3.5)

I am new to REDIS world. I am trying to fetch values for 3000 keys from REDIS. Each hash has 6 values that I want to fetch. I am using Python 3.5 to to make connection to REDIS one time and then loop through my key hashs to get their respective values from REDIS. However, currently it is taking about 10 seconds to get the values for these 3000 rows. I am using the below code to get data from REDIS. Can you please help me in speeding up the fetch. Is there a way to send all the keys at once and get the values related to them? I am using python 3.5 for this. 

redis_pool = redis.ConnectionPool(host='XYZ',
                                  port='XY',
                                  db='X')
r = redis.Redis(connection_pool=red_pool)

field1 = r.hmget(primary_key, "Field1")

Upvotes: 3

Views: 1377

Answers (1)

GuangshengZuo
GuangshengZuo

Reputation: 4677

You could try pipeline to speed up your query.

r = redis.Redis(connection_pool=red_pool)
pipe = r.pipeline()
for key in keys_list: 
    pipe.hget(key, "field1")
results = pipe.execute()

The results will be the list of every hget reply. you could refer redis-py's readme's pipeline section to learn more about how to use pipeline in python redis client.

Upvotes: 6

Related Questions