Hazelcast: Slowness/Blocking when search results are big

Question

We are seeing slowness/blocking when search results are big. Server is receiving around 10000 requests/minute. Queries with smaller search results are faster. Slowness/blocking observed when search results are big.

I have enabled a slow operation detector to log slow operation taking more than 1 seconds. No slow operation detected while running search so query execution on server side should be faster.     So far I tried below optimization:

1. Increased client socket buffer size from 32 kb to 1024 kb. Not seeing improvement in performance.
2. Double the members of nodes. Not seeing improvements.

I am using hazelcast portable serialization. Hazelcast version 3.11.

I am using IMap.getAll to fetch multiple objects. Hazelcast slow operation detector is enable to detect query taking more than 1 seconds. It's not detecting this query so assuming query is fast.

It's faster if result has small objects but slow when results contains big object with getAll.  

What other optimization can be done?

Answer 1

Duplicate of Google Group question