CODEWITHSUNDEEP
javajmh
qiGuar
qiGuar

Reputation: 1804

How to choose jmh configuration parameters

I want to write performance tests using jmh to measure some parts of code of my application. I've read jmh samples on how to write tests and watched some talks on youtube about jmh and performance testing.

What I don't understand is how to choose correct values for @Warmup, @Measurement and @Fork values. In most samples it's 1 Fork, 10 iterations with 1 second duration.

I understand, that there's no silver bullet for configuration, but what should be my thought process when choosing what values I should put?

Upvotes: 4

Views: 1478

Answers (2)

Imaskar
Imaskar

Reputation: 2959

Just a little addition to @Eugene's answer.

@Fork is used to check for optimizations of the hot code. Sometimes you might see very different results in different forks. That's because JIT compiler swaps some code with a more optimized one based on usage. So I personally run at least 3 forks.

Upvotes: 0

Eugene
Eugene

Reputation: 120998

I don't know if anyone could recommend parameters that are actually as close as you might need - probably not even the guys that wrote the tool. This is extremely specific to the use case that you are testing, the OS, the JVM, etc.

I personally do it a few times with different parameters and capture the result each time so that I could analyze it. You have some Error +/- (for Average time at least) field that is my first indication. If it's too big (and that is something you would determine), I usually increase the @Warmup and @Measurement; that's not a golden rule, but this is what I do. Here is a sample I use:

 @BenchmarkMode(Mode.AverageTime)
 @OutputTimeUnit(TimeUnit.NANOSECONDS)
 @Warmup(iterations = 5, time = 2, timeUnit = TimeUnit.SECONDS)
 @Measurement(iterations = 5, time = 2, timeUnit = TimeUnit.SECONDS)
 @State(Scope.Thread)

I might increase the iterations and time little by little to see if I get close to constant results with previous runs.

I usually run all the benchmarks under the same warmup, so that results are not biased on that.

The thing to get here, is that you might get good or bad results, or slow and fast - but that is only the output, not the reason. Understanding the reason and running jmh under perfasm or xperfasm and many other profilers, is an entire different kitchen, way out of my league...

Upvotes: 3

Related Questions