CODEWITHSUNDEEP
pythonnumpynumericnumerical-methods
Bull
Bull

Reputation: 11941

Is Python's epsilon value correct?

According to Wikipedia:

Machine epsilon is defined as the smallest number that, when added to one, yields a result different from one

In Python, epsilon can be found using sys.float_info.epsilon and returns a value equivalent to 2^-52. However, I can add any number greater than 2^-53 to 1 and still get a result different to one. But by the above definition of epsilon, adding any value less than epsilon to one should give one. Does this mean that sys.float_info.epsilon is returning an incorrect value, or that Python is using some other definition of epsilon?

The following code illustrates this, with the floating point numbers printed out in hex format.

import sys
import numpy
print 'epsilon=%g' % sys.float_info.epsilon
# output: 2.22045e-16
epsilon = sys.float_info.epsilon
print 'epsilon(hex) = %s' % float.hex(epsilon)
# output:  0x1.0000000000000p-52

one = numpy.float64(1.0)

delta = float.fromhex('0x1.fffffffffffffp-53')
print 'delta = %s' % float.hex(delta)

print 'epsilon - delta = %s' % (float.hex(epsilon-delta))
#output: 0x1.0000000000000p-105

print '\n1.0 + epsilon = %s' % (float.hex(one+numpy.float64(epsilon)))
#output: 0x1.0000000000001p+0

print '\n1.0 + delta = %s' % (float.hex(one+numpy.float64(delta)))
#output: 0x1.0000000000001p+0
# since delta is smaller than epsilon, I expected 0x1.0000000000001p+0

delta1 = float.fromhex('0x1.0000000000001p-53')
print '\n1.0 + %s = %s' % (float.hex(delta1), float.hex(one+delta1))
#output: 0x1.0000000000001p+0
# since delta is smaller than epsilon, I expected 0x1.0000000000001p+0

delta2 = float.fromhex('0x1.0000000000000p-53')
# note: delta2 = epsilon / 2.0
print '\n1.0 + %s = %s' % (float.hex(delta2), float.hex(one+delta2))
# 0x1.0000000000000p+0

The resulting output is

epsilon=2.22045e-16
epsilon(hex) = 0x1.0000000000000p-52
delta = 0x1.fffffffffffffp-53
epsilon - delta = 0x1.0000000000000p-105

1.0 + epsilon = 0x1.0000000000001p+0

1.0 + delta = 0x1.0000000000001p+0

1.0 + 0x1.0000000000001p-53 = 0x1.0000000000001p+0

1.0 + 0x1.0000000000000p-53 = 0x1.0000000000000p+0

Upvotes: 6

Views: 18060

Answers (2)

Brenton LeMesurier
Brenton LeMesurier

Reputation: 41

That definition is wrong: machine epsilon is the number eps such that 1+eps is the next number after 1. Thus with standard rounding, u = eps/2 is the smallest number that when added to 1 gives a value greater than 1. This quantity u is often called the unit roundoff or machine unit. (For IEEE64, eps=2^(-52), u = 2^(-53).)

By the way, for most practical purposes, the unit roundoff u is more useful than eps: for example, u is the maximum relative error when a value is rounded to a normalized machine number.

Reference: Higham, Accuracy and Stability of Numerical Algorithms, pages 37, 38.

Upvotes: 4

Blckknght
Blckknght

Reputation: 104712

I think that what you you are seeing is how Python's float type handles rounding when it runs out of precision. The Wikipedia text you quoted describing epsilon appears to leave this part out.

In your example, 1 + delta gets rounded up to 1 + epsilon. Even though a float can specify the differences between delta and epsilon, it cannot represent the difference between 1 + delta and 1 + epsilon. As you have noticed (with your delta2 test), the largest number that gets rounded down to 1 rather than up to 1 + epsilon appears to be 1 + epsilon/2.

So, a correct definition of what epsilon means in Python might be:

epsilon is the smallest positive floating point number such that (1 + epsilon) - 1 equals epsilon.

Upvotes: 7

Related Questions