CODEWITHSUNDEEP
pythondjangoredishiredisdjango-redis
afriedman111
afriedman111

Reputation: 2341

How to optimize a bulk query to redis in django - hiredis

I am porting a rest/graphql api from a Java project to Django in Python. We are using redis in both. We have one endpoint that is rather large (returns several MB). In this endpoint we construct a key and if that key exists in redis we return that data and skip past the other logic in the endpoint. I have deployed the api to a development environment. The Java version of this endpoint returns data between 4-5 seconds faster than the python version. I recognize some of this is that Java is a compiled language vs Python which is interpreted. I'm aware that there are a host of other differences, but I'm trying to see if there is any low hanging fruit that I can pick to speed up my Python api.

I ran across an optimized library named hiredis. The website says the library speeds up multi-bulk replies, which is my use case. I repeatedly hit an endpoint that returns a large amount of data. Hiredis is supposedly a parser written in C. I have redis-py (v4.5.4) and django-redis installed. I read that I need to pip install hiredis. One source says hiredis will automatically be detected and just work in my environment but another says I need to add a PARSER_CLASS property to my CACHES setting like so:

CACHES = {
    'default': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': 'redis://127.0.0.1:6379/1',
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
            'PARSER_CLASS': 'redis.connection.HiredisParser',
        }
    }
}

I have tried both ways and I do not see a speed up. I'm certain the data is being cached in redis and the key is found. The data is returning from redis and skipping over the rest of the logic. Here are my questions

  1. Am I able to use hiredis with Django?
  2. What do I need to do to enable it?
  3. Is there a way I can verify that the hiredis parser is being used?
  4. Are there any other ways I can speed up my redis connection?

Upvotes: 1

Views: 150

Answers (0)

Related Questions