CODEWITHSUNDEEP
floating-pointprecision
tonyjosi
tonyjosi

Reputation: 845

Floating point error in ULPs (units in the last place)

This question is based on the paper What Every Computer Scientist Should Know About Floating-Point Arithmetic by David Goldberg. In the section 1.2 Relative Error and Ulps (page number 8), the paper gives an equation to calculate error in, ULP (units in the last place):

| d.d…d - (z/(β^e)) | β^(p-1)

Where d.d…d * (β^e) is the floating point representation of the number z and, β and p are the base and precision of the representation.

Later in the paper (page number 9) there is an example evaluating the result of the operation 10.1 - 9.93 which results in 0.2 when using a floating point representation of β = 10 and p = 3, it says the result 0.2 has an error of 30 ULPs and every digit in the result is wrong. But I couldn't use the above equation to arrive at 30 ULPs error, the value that I get while calculating the error using above equation is 0.3 ULPs.

This is how I did the calculation:

(0.02 - (0.17/(10^1)))*10^(3-1) = 0.3      // 0.17 is the actual difference

Am I missing something?

Upvotes: 5

Views: 1857

Answers (1)

chux
chux

Reputation: 154127

The infinite precision difference of 10.1 - 9.93 is 0.17 .

The machine with β = 10 and p = 3 returned a difference of 0.200.
Notice 3 significant base 10 digits as p = 3 .

The value of the error is |0.17 - 0.2| or 0.03 .

The ULP of 0.200 on this machine is 0.001 .

The correct answer is 0.03/0.001 or 30 ULPs from 0.2 .

the value that I get while calculating the error using above equation is 0.3 ULPs.

Likely as the error is 0.3 ULPs of 10.1 away. ULP is not a constant for a given machine but a function ULP(x) per value.

Upvotes: 3

Related Questions