CODEWITHSUNDEEP
javacachingmemcached
Débora
Débora

Reputation: 5952

Cache multiple byte arrays for single key

I have to cache large byte arrays. Most of the time, the size of the byte array exceeds the max size of accepted size of underlying caching service. For example, Memcache accepts only objects upto 1MB.
Therefore, My proposed way is to split the large byte arrays into small size arrays and cache them separately. Furether more, when the perticular key is requested, all cached byte arrays should be retrieved. I found some ways in php. but no for java implementation. Does any one to do such thing in Java ? any resource code snip please.

Upvotes: 0

Views: 659

Answers (1)

ragnor
ragnor

Reputation: 2528

Have you considered to split your array into smaller peaces and cache each part under dedicated key? When you want to fetch an array just fetch each parts a merge into single array.

put into cache:

String cacheKey = ...;
int maxItems = 10000;
int totalCount = myObjects.length;
MyObject [] myObjects = ...;
for (int i = 0; i < (totalCount  / maxItems) + 1; i++) {
  int endIndex = totalCount  > (maxItems * (i + 1)) ? maxItems * (i + 1) : totalCount;
  MyObject [] part = Arrays.copyOfRange(myObjects, i * maxItems, endIndex);
  cache.put(cacheKey + i, part);
}

get from cache:

String cacheKey = ..'
int partNr = 0;
MyObject [] part;
MyObject [] myObjects = new MyObject[0];
do {
 part = cache.get(cacheKey + partNr);
 partNr++;
 // from guava
 if (part != null) {
   ObjectArrays.concat(myObjects, part, MyObject.class);
 }
} while (part != null && part.length > 0)

It can be optimized using multiget to fetch all parts at once and using multiset to put all parts into cache at once.

Upvotes: 1

Related Questions