CODEWITHSUNDEEP
floating-pointsseavxfloating-point-exceptions
jcai
jcai

Reputation: 3593

SSE/AVX floating point convert exceptions

Suppose an SSE register contains one or more packed values not representable as a 32-bit int (such as Inf or NaN), and a convert-to-int is invoked, such as _mm_cvtpd_epi32 / cvtpd2dq.

  1. Is it safe, i.e. is the behavior defined?
  2. Does it break control flow, or merely raise a flag?
  3. What is the result of the conversion, if defined?

Upvotes: 3

Views: 640

Answers (1)

Stephen Canon
Stephen Canon

Reputation: 106287

The answers to these questions can mostly be found in the Intel® 64 and IA-32 Architectures Software Developer’s Manual:

CVTPD2DQ

... If a converted result is larger than the maximum signed doubleword integer, the floating-point invalid exception is raised, and if this exception is masked, the indefinite integer value (80000000H) is returned.

It's not spelled out clearly in this section, but this also applies to infinities and NaNs. So:

  1. Yes, the behavior is defined.
  2. Under the default floating-point environment (i.e. with the invalid exception masked in MXCSR), it does not break control flow; it only sets a flag.
  3. The result is 0x80000000.

Upvotes: 4

Related Questions