CODEWITHSUNDEEP
javahazelcast-imaphazelcast-jet
Parthi
Parthi

Reputation: 123

How to control number of processors during Hazelcast Imap filter

I am new to Hazelcast. When Imap filter with predicate is triggered on a HazelCast Jet pipeline, in a VDI I see the porcessors count is only 2 and on my machine I see 12 porcessors.

When there is only 2 processor, filter job is completing very fast. But when I see 12 processors on my machine, filter job keeps running for longer time and keeps logging a line of Hazelcast healthmonitor log and times out. I assume that this is the problem.

From my search, I understand that Hazelcast create number of processors based on CPU. How can I control the this processor count.

Below is the line of log when Hazelcast jet client is created and IMap filter is applied on a pipeline.

Pardon me, I couldn't give clear description of issue with limited understanding of Hazelcast.

Jun 08, 2021 9:31:20 PM com.hazelcast.internal.diagnostics.HealthMonitor
INFO: [*xxx.xx.xxx.xxx*]:5701 [jet] [3.1] **processors=12**, physical.memory.total=32.0G, physical.memory.free=224.8M, swap.space.total=2.0G, swap.space.free=716.5M, heap.memory.used=110.4M, heap.memory.free=182.1M, heap.memory.total=292.5M, heap.memory.max=7.1G, heap.memory.used/total=37.74%, heap.memory.used/max=1.52%, minor.gc.count=4, minor.gc.time=75ms, major.gc.count=2, major.gc.time=129ms, load.process=3.01%, load.system=93.62%, load.systemAverage=9.20, thread.count=78, thread.peakCount=78, cluster.timeDiff=0, event.q.size=0, executor.q.async.size=0, executor.q.client.size=0, executor.q.client.query.size=0, executor.q.client.blocking.size=0, executor.q.query.size=0, executor.q.scheduled.size=0, executor.q.io.size=0, executor.q.system.size=0, executor.q.operations.size=0, executor.q.priorityOperation.size=0, operations.completed.count=392, executor.q.mapLoad.size=0, executor.q.mapLoadAllKeys.size=0, executor.q.cluster.size=0, executor.q.response.size=0, operations.running.count=0, operations.pending.invocations.percentage=0.00%, operations.pending.invocations.count=2, proxy.count=8, clientEndpoint.count=0, connection.active.count=0, client.connection.count=0, connection.count=0

Upvotes: 0

Views: 317

Answers (1)

Oliv
Oliv

Reputation: 10812

The processors=12 is a red herring I guess - it's just the value of Runtime.availableProcessors().

From my search, I understand that Hazelcast create number of processors based on CPU

By this you probably mean the number of Jet parallel processors. You can control the number of parallel processors by setting localParallelism(n) in your Pipeline.

A line from Health Monitor is logged when there's unusual load on the system. It can be caused by some other process in the system maxing out the CPU. Your question doesn't give enough information to guess.

Upvotes: 0

Related Questions