CODEWITHSUNDEEP
javaperformanceparallel-processing
Allzhere
Allzhere

Reputation: 1370

Max Independent JVM Processes

I have java deployed on a unix system at /home/apps/java. export JAVA_HOME="/home/apps/java"

There are some shell scripts that run in parallel (currently : 5 in number) with the following commands: $JAVA_HOME/bin/java -Xmx512M com.home.prac.SendNotification

The Java program performs a db operation and sends out a notification. There are several objects created here and each execution peaks heap to about 300 MB. I have not shared code here as the my query is not related to it. Hence, i have just briefed the purpose. My understanding is that separate independent processes of JVM will be initiated that carry out executions of Java programs. They have their separate heaps and memory assignment.

However, i am stuck with the following queries:

  1. How many of these parallel instances can i run considering they are all taking 300 MB in heap?
  2. There should a max cap (At OS level) to this which freezes after certain number of instances?
  3. I can't just go ahead and initiate 100 or 1000 parallel processes?
  4. I have a follow up question but would like to clarify the concept here first.

Upvotes: 2

Views: 1720

Answers (3)

Stephen C
Stephen C

Reputation: 719199

1) How many of these parallel instances can i run considering they are all taking 300 MB in heap?

You probably should create more instances than you have physical memory to support. However, you can't just multiply the number of instances by the heap size, because a JVM needs more memory than that. Rather, you need to use monitoring tools (like top) to figure out how much memory each instance is actually using.

Note that Linux will allow you to create more processes than there is physical memory, provided that there is sufficient swap space. However, if the processes are all active, they effectively compete for memory, and the system spends a significant percentage of its time waiting while pages belonging to one process are "swapped out" and other pages are "swapped in".

2) There should a max cap (At OS level) to this which freezes after certain number of instances?

There is most likely an upper limit on the number of processes that can exist, but this is unlikely to be an issue. This issue will be competition for resource; e.g. CPU cycles, physical memory, memory bandwidth, disc bandwidth and network bndwidth.

3) I can't just go ahead and initiate 100 or 1000 parallel processes?

That would be a bad idea. If you over-allocate memory, and there are too many active processes competing for memory, two things are likely to happen:

  • The system performance will drop drastically due to "virtual memory thrashing".

  • On Linux, the "oom killer" will start killing processes in order preven the system from locking up completely. The processes killed might be your JVM instances ... or they might be something else.

Upvotes: 2

Vishal Biyani
Vishal Biyani

Reputation: 4407

Answers to questions with same number:

1) In theory there is no limit. The only thing you are limited by is memory/RAM available. If you have let's say 3GB RAM then (your processes)*512 + System needed memory should be 3GB. This is because heap is allocated at startup and if heap allocation fails - your JVM process won't start. But again this is dependent on how much memory is being consumed by a process at given point in time

2) Not that I am aware of - your OS details please?

3) Check (1)

Upvotes: 0

DaveH
DaveH

Reputation: 7335

  1. Depends on how much available RAM you have on your machine. If you have 1G, then running 7 instances will be tricky......

  2. Once you run out RAM, I presume that Unix will start swapping the JVM out to disc ( his is a guess ... ). If that starts to happen, your processes will not necessarily die, but you will see a marked degradation in performance.

  3. If you do that, you will be in for a disappointment .... There comes a point where swap gets exhausted as well, at which point you will probably have succeeded in crashing the whole unix box.

Have you considered altering the code to try to get the processes to run as separate threads within the same JVM? It's likely to reduce the memory footprint significantly.

Upvotes: 0

Related Questions