CODEWITHSUNDEEP
javahazelcasthazelcast-imap
jnmts
jnmts

Reputation: 21

Is hazelcast distributed-loading blocking?

Currently, I have an IMap backed with a MapStore. This IMap is loaded on startup and have millions of entries. When the map is fully loaded, the clients retrieves entries using Imap.values() with predicates. This loading work fine for me on startup.

Now, I have to reload peridiocally all the data (e.g., every 2 hours). I'm using IMap.loadAll(true) to force the reload. The MapStore (loadAllKeys() + loadAll(keys)) takes about 4 minutes to finish the complete load. It seems that in these 4 minutes, all the clients that send queries to the imap via predicates are blocked until the reloading is finished.

In my case, I can't block the cluster 4 minutes (neither a second in fact).

  1. is really the Imap.loadAll blocking the predicates queries?
  2. is possible configure the imap as non-blocking or exists a non-blocking alternative to IMap.loadAll().

I'm trying it with HC 3.7.2, with 2 nodes and java 8.

Upvotes: 2

Views: 794

Answers (1)

tom.bujok
tom.bujok

Reputation: 1642

It's a sensible use case, but it's currently unsupported for a couple of reasons. The loading takes place on the partition threads and thus blocks them - it gives consistency, the map won't serve any requests until the loading has finished.

For your use case of "reloading" it would have to happen not on partition threads so that the requests can be served.

Could you create a feature request on github? It looks like sth we could address in the next release.

Upvotes: 1

Related Questions