CODEWITHSUNDEEP
javajitjmh
mmirwaldt
mmirwaldt

Reputation: 883

Why use JMH if you can switch off JIT?

I wonder why I should use JMH for benchmarking if I can switch off JIT? Is JMH not suppressing optimizations which can be prevented by disabling JIT?

Upvotes: 1

Views: 793

Answers (2)

Aleksey Shipilev
Aleksey Shipilev

Reputation: 18857

TL;DR; Assess the Formula 1 performance by riding a bycicle at the same track.

The question is very odd, especially if you ask yourself a simple follow-up question. What would be the point of running the benchmark in the conditions that are drastically different from your production environment? In other words, how would a knowledge gained running in interpreted mode apply to real world?

The issue is not black and white here: you need optimizations to happen as they happen in the real world, and you need them broken in some carefully selected places to make a good experimental setup. That's what JMH is doing: it provides the means for constructing the experimental setups. JMH samples explain the intricacies and scenarios quite well.

And, well, benchmarking is not about fighting the compiler only. Lots and lots of non-compiler (and non-JVM!) issues need to be addressed. Of course, it can be done by hand (JMH is not magical, it's just a tool that was also written by humans), but you will spend most of your time budget addressing simple issues, while having no time left to address the really important ones, specific to your experiment.

Upvotes: 10

fge
fge

Reputation: 121790

The JIT is not bulletproof and almighty. For instance, it will not kick in before a certain piece of code is run a certain number of times, or it will not kick in if a piece of bytecode is too large/too deeply buried/etc. Also, consider live instrumentation which, more often than not, prevents the JIT from operating at all (think profiling).

Therefore the interest remains in being able to either turn it off or on; if a piece of code is indeed time critical, you may want to know how it will perform depending on whether the JIT kicks in.

But those situations are very rare, and are becoming more and more rare as the JVM (therefore the JIT) evolves.

Upvotes: 0

Related Questions