CODEWITHSUNDEEP
c++performancex86-64icc
Gaston
Gaston

Reputation: 607

Possible causes for icc (2019) with -O3 -march=native on Xeon Gold 6126 producing slower exe than -O3 -xCORE-AVX512?

The purpose of the question is to ask about possible causes regarding the program's behaviour as a function of icc 2019's compilation flags, considering two phenomena and the information provided in the notes below.

A program can run three types of simulations, let's name them S1, S2 and S3.

Compiled (and ran) on Intel Xeon Gold 6126 nodes the program has the following behaviour, expressed as

A ± B

where A is the mean time, B is the standard deviation, and the units are microseconds.

When compiled with -O3:

When compiled with -O3 -march=native:

When compiled with -O3 -xCORE-AVX512:

Two conclusions:

  1. -xCORE-AVX512 produces code that is more performant than -march=native
  2. the program's simulation called S3 DECREASES its performance when compiled considering the architecture.

Note1: the standard deviation is huge, but repeated tests yield always similar values for the mean that leave the overall ranking unchanged.

Note2: the code runs on 24 processors and Xeon Gold 6126 has 12 physical cores. It's hyper-threading but each two threads per core DO NOT share memory.

Note3: the functions of S3 are "very sequential", i.e. cannot be vectorized.

There is no MWE. Sorry, the code is huge and cannot be posted here.

EDIT: print-related outliers were to blame for the large deviation. The means were slightly changed but the trend and hierarchies remains.

Upvotes: 1

Views: 157

Answers (2)

MSalters
MSalters

Reputation: 179991

Your premise is wrong. The differences in all cases are a tiny fraction of one standard deviation. A statistically significant result typically is > 2 standard deviations.

Of course, if we see a duration of 104.7612 ± 108.7875, we know that the expected runtime cannot be normally distributed, since that would imply a >16% chance of finishing before it starts! (negative runtime). But there are other distributions with long tails where the standard deviation can be bigger than the mean. Without knowing the exact distribution, I'm not entirely sure if the "> 2 standard deviations" rule of thumb holds up, but <0.1 standard deviation difference is definitely not significant.

[edit] New figures are S1 differs by about .3 sigma, S2 by about .05 sigma and S3 by about 1.2 sigma. Individually that's still inconclusive, and there's still not enough code to say whether there's a correlation. Certainly can't simply add them up, but even if you did you still wouldn't hit 2 sigma.

Upvotes: 3

Gaston
Gaston

Reputation: 607

An acceptable possible explanation was outlined in the comments, it read:

Tiny differences in tuning choices for code-gen might result in alignment differences that end up mattering more. Especially How can I mitigate the impact of the Intel jcc erratum on gcc? on Skylake-family CPUs if -march=native or -xCORE-AVX512 didn't enable a workaround option.

Further readings:

32-byte aligned routine does not fit the uops cache

Intel JCC Erratum - should JCC really be treated separately?

Upvotes: 0

Related Questions