Why doesn't the JVM compile the entire program up front, instead of compiling it piece-by-piece?

Question

For this thread Herbert Schildt writes:

It is important to understand that it is not practical to compile an entire Java program into executable code all at once, because Java performs various run-time checks that can be done only at run time.

What run-time checks does he mean?

Please explain the reasons of compiling bytecode piece-by-piece, not the entire program.

Answer 1

Faster start up times and to be able to make smarter optimizations after profiling the code during execution.

This article gives an excellent answer to this question: https://advancedweb.hu/2016/05/27/jvm_jit_optimization_techniques/

The JVM starts out interpreting everything on-the-fly. It keeps track of how many times a function is called and if it passes a certain limit it compiles that function and caches this machine code. Every time that function is called after that it executes this machine code. If it passes another threshold it can be compiled again but with even more aggressive optimizations performed.

The goal is to balance the up-front cost of compilation. Compiling everything up-front will cause longer start-up times and the compiler cannot make the best possible optimizations without seeing the code run. Longer running programs have the potential for better optimizations than shorter running programs.

It can also compile just the body of a loop if it is executed so many times.

Answer 2

That reasoning is incorrect. It is possible to compile an entire Java program into executable code all at once, and in fact gcj does exactly that.

The actual reasons Java runtimes typically don't do up-front compilation:

• As part of Java's "write once, run anywhere" philosphy, compiled code is distributed as platform independent bytecode instead of platform-specific native code.

• Compiling bytecode to native code at start-up is sub-optimal for a few reasons. First, it would incur a pretty hefty start-up cost especially for larger programs. Also, modern JITs are able to make use of runtime profiling to produce better optimizations than would be possible with an up-front compiler. Optimizations generally involve making trade-offs: For example, inlining removes function-call overhead at the expense of larger code (which can, in turn lead to slower code due to bad cache performance or even swapping). When you have runtime profiling information at compile time you can more intelligently decide which trade-offs make sense.

• Java APIs have a history of relying on the ability to load bytecode at runtime. (Think of anything that relies on Class.forName and/or custom Classloaders, for example, including applets, servlet containers, etc.) To support this some kind of run-time compiler (or interpreter) is necessary. Doing up-front compilation would mean you'd either have to remove support for this, or treat it as a special case.

Answer 3

What he is saying is that it's impractical to compile all the bytecode to machine language at runtime. You could precompile everything, but that's not the approach that JIT takes.

For one thing, there is no knowing how large the program is. People would get fairly upset at a 30 minute startup as it compiled every library it could find (A given Java program isn't in a single file, it has access to everything in the classpath)

For another, even if you told the system exactly what components your program would use, there is no telling how much of your program could be used for a given run--people would get more upset at a 30 minute startup to run a command line program with parameters consisting of "--help"

Finally it can do some great tricks by compiling while it's running. with a method like this:

public testMeh(int param) {
    if(param == 35) 
        do bunches of crap;
    else if (param > 5)
        do much more crap;
    else 
        return;
    }

The compiler can call it once or twice and on the fly recognize that values 5 and under just return. If this is called all the time with a value of 2 it can replace the ENTIRE method call with if (param != 2) testMeh(param);

which eliminates the entire method call for that number. Later it can figure out that not calling that method means certain member variables can't change and that can collapse other portions of the code to nothing.

This is just plain hard as hell if you precompile stuff. I mean you could write exception code everywhere as you recognize patterns but your code would quickly become a nightmare.

Now if you are asking why not precompile the entire program when you compile it into bytecode--that's a different question and not what the quote was addressing at all--but you can do that to. It's been done and works pretty well. You trade portability and runtime flexibility for a quicker start time.

Answer 4

One possible reason: Java runtime is very dynamic environment. Classes are loaded and unloaded all the time. Depending on classes currently in use, certain optimizations are possible or impossible. For example, if you have interface A and single implementation ImplA, all calls to methods in interface A may be changed to use direct calls to ImplA methods, without doing any virtual method stuff. Once another class AnotherImplA is loaded, this kind of optimization is not possible anymore. Once AnotherImplA is unloaded, it's possible again.

There are lot of optimizations in JVM that use data gathered during runtime. Doing all optimizations up-front would miss lot of opportunities.

Check also recent talk from Cliff Click, A JVM Does That?

Answer 5

There could be several reasons to compile it piece by piece(those are the first two that come to my mind):

1. Optimisation of a code that is used many times, not all of the code needs to be recompiled, but only the specific part.
2. Acquisition of classes over the network - sometimes you want to avoid acquiring all the code, since it costs in bandwidth.

And I don't know if this site is accurate, but I learned a lot from it: http://www.artima.com/insidejvm/ed2/index.html