CODEWITHSUNDEEP
mysqlmemoryredisjedisnosql
theja_swarup
theja_swarup

Reputation: 167

How to ensure memory consumption is not scaling with the size of the redis database

I am currently using Jedis for fetching my data from redis database server. I need to push this data on to MySQL. The redis server has millions of records in it. Using the below statements I can copy the data in to a set:

String pattern = "users*";//All keys starting with users
Set<String> users = jedis.keys(pattern)//Read all the keys in to set

Now my users set has all the records. But there may be million records that match my pattern in redis database. This will eventually consume all my memory. How can I do some thing like below

for(All the keys that match my pattern){    
   Set<String> set = get current to (current+10000) records from server
   ...Code to push 10K records to MySQL...
   current = current + 10001;
}

Or please suggest if there is an elegant approach for this

Upvotes: 0

Views: 109

Answers (1)

Nick Bondarenko
Nick Bondarenko

Reputation: 6351

Yes, you can. Use SCAN command for that. Look at Jedis Scan Test for examples of usage.

Upvotes: 1

Related Questions