CODEWITHSUNDEEP
floating-pointieee-754
geza
geza

Reputation: 29962

Understanding the usefulness of denormalized floating point numbers

Reading Goldberg's What Every Computer Scientist Should Know About Floating-Point Arithmetic, I found something I don't really understand.

He states that having denormalized numbers is good, because x = y if and only if x-y==0. Then he gives the example:

if (x != y) then z = 1/(x-y)

Now, suppose that x-y is a denormalized number. Then there is a high chance that 1/(x-y) will become inf. Which is the same result, if we didn't have denormalized numbers in the first place.

Even, if I want to execute a division, and avoid inf results, then it is more convenient, if we don't have denormalized numbers:

if (x-y) then z = 1/(x-y) // here, we know that z is not inf

I cannot achieve the same with denormalized numbers, as x-y might not be zero, but a denormalized number, and then the 1/(x-y) divison will result in inf. So, here, denormalized numbers are actually cause trouble.

Why is it a good property to have x=y if and only if x-y=0?

When denormalized numbers are useful?

Upvotes: 0

Views: 555

Answers (1)

Eric Postpischil
Eric Postpischil

Reputation: 222866

Some points from William Kahan on gradual underflow:

  • If a floating-point format has subnormal values, the gaps between representable values do not increase as numbers decrease. This helps when writing proofs, as one can assert that, if the difference between x0 and its nearest neighbor is u, then, for any x ≤ x0, the difference between x and its nearest neighbor is at most u. Then you can go on and conclude that the error produced by some sequence of computations is at most some value as long as the input is within bounds. Without subnormal values, you would have to write exceptions where you consider cases where intermediate values underflowed, different your proof into multiple cases, exclude certain parts of the domain fro the proof, and so on.
  • x±y cannot underflow since a subnormal difference is exact.
  • A sum of products suffers less rounding error from subnormal terms as long as at least one term is normal.

The paper does not claim that gradual underflow is superior in every situation, but that it has proven to be preferable overall.

While you give this example:

if (x-y) then z = 1/(x-y) // here, we know that z is not inf

that has restricted utility, as it is not true in if (x-y) then z = 4/(x-y).

Upvotes: 2

Related Questions