CODEWITHSUNDEEP
javaredisjedis
perplexedDev
perplexedDev

Reputation: 835

Efficient way to get all the key value pair from redis cache using Jedis

I have a redis cache database that holds 80k records. I need to get this entire data( key and value) in my application. I am using Jedis Client and here is the code I have used:

private static JedisPool jedisPool;
public RedisClient(String ip, int port, int timeout, String password) {
        try {
            if (jedisPool == null) {
                JedisPoolConfig poolConfig = new JedisPoolConfig();
                jedisPool = new JedisPool(poolConfig, ip, port, timeout, password);
            }
        } catch (Exception e) {
            log.error("Malformed server address", e);
        }
    }

 public String get(final String key) {
        try (Jedis jedis = jedisPool.getResource()) {
           String value = jedis.get(key);
            return value;
        } catch (Exception ex) {
            log.error("Exception caught in get", ex);
        }
        return null;
    }


 public void populateRedisMap(RedisClient redisClient) {
        RedisIterator iterator = redisClient.iterator(Integer.parseInt(scanCount), matchPattern, scan);

        List<String> keys = iterator.next();
        while (iterator.hasNext()) {
            keys.addAll(iterator.next());
        }

        log.info("Keys count:{}", keys.size());

        for (String key : keys) {
            if(redisClient.get(key) instanceof String) {
                String value = redisClient.get(key);
                redisMap.put(key, value);
            }
            else{
                Object keyValue = redisClient.get(key);
                log.info("Value {0} of Type {1} found for key {2}", keyValue, keyValue.getClass(), key);
            }
        }

        log.info("Records count in Redis Map:{}", redisMap.size());
    }

The above code works. I get 80k keys and it loops through each key to get the value and the hash map is populated with key and value. I haven't added the code for the Iterator here, I didn't think it is relevant to my question but I can added it if needed.

My issue here is the performance, it takes a long time (>30 mins) for this hash map to be populated. The iterator part to get the list of keys takes few seconds, it is the for loop to get the values that take time. My question is, is there a more efficient way to get all the key value data from redis cache? I couldn't find any other command I could use in Redis documentation but wanted to check if I am missing something.

Upvotes: 1

Views: 8252

Answers (1)

sazzad
sazzad

Reputation: 6267

You can chunk your keys into arrays[Note1] of keys and call mget.

    public List<String> get(final String[] keys) {
        try (Jedis jedis = jedisPool.getResource()) {
            List<String> values = jedis.mget(keys);
            return values;
        } catch (Exception ex) {
            log.error("Exception caught in mget", ex);
        }
        return null;
    }

Note1: The arrays can be of size 100 or 1000 or any other size depending on your system and preference.

Upvotes: 2

Related Questions