CODEWITHSUNDEEP
assemblyfloating-pointsseavxdenormal-numbers
Bikineev
Bikineev

Reputation: 1777

cmpeqpd sometimes returns wrong values

For some reason, sometimes in my program I see that

cmpeqpd xmm3,xmm0

where xmm0 == {0x2cd000000000, 0x2cd000000000} and xmm3 == {0x0, 0x2011d0800000000} happens to return {0xffffffffffffffff, 0x0} in xmm3, which is, well, wrong, since (double)0x0 is not equal to (double)0x2cd000000000.

I've noticed that it happens only occasionally. I've recorded the execution of the program with rr so that to reproduce this consistently. Funnily enough, in a super reduced simple program I can't reproduce this issue anymore. I wonder, are there any hidden microarchitectural states that can change cmpeqpd (cmppd) behavior?

Just a note, I checked that upper 128bits in the corresponding ymm registers are zeroed.

Upvotes: 3

Views: 103

Answers (1)

user555045
user555045

Reputation: 64904

0x2CD000000000 (or 0x00002CD000000000 to make it more obvious what the exponent is doing) is denormal. There is indeed a setting that would make a denormal compare equal to zero: the DAZ flag

The DAZ flag may have gotten set at some point, and it would explain why a simplified program no longer showed this behaviour.

Upvotes: 7

Related Questions