CODEWITHSUNDEEP
pythonnumpyfloating-pointprecision
Charlie
Charlie

Reputation: 326

Machine Epsilon in Python

A manual that I am currently studying (I am a newbie) says:

"Numbers which differ by less than machine epsilon are numerically the same"

With Python, machine epsilon for float values can be obtained by typing

eps = numpy.finfo(float).eps

Now, If I check

1 + eps/10 != 1

I obtain False.

But If I check

0.1 + eps/10 != 0.1

I obtain True.

My latter logical expression turns to be False if I divide eps by 100. So, how does machine epsilon work? The Python documentation just says

"The smallest representable positive number such that 1.0 + eps != 1.0. Type of eps is an appropriate floating point type."

Thank you in advance.

Upvotes: 18

Views: 30216

Answers (2)

Joe Kington
Joe Kington

Reputation: 284880

In this case, you actually don't want np.finfo. What you want is np.spacing, which calculates the distance between the input and the next larger number that can be exactly represented.

Essentially, np.spacing calculates "eps" for any given number. It uses the number's datatype (native python floats are 64-bit floats), so a np.float32 or np.float16 will give a different answer than a 64-bit float.

For example:

import numpy as np

print 'Float64, 1.0 -->', np.spacing(1.0)
print 'Float64, 1e12 -->', np.spacing(1e12)
print 'Float64, 1e-12 -->', np.spacing(1e-12)
print ''
print 'Float32, 1.0 -->', np.spacing(np.float32(1.0))
print 'Float32, 1e12 -->', np.spacing(np.float32(1e12))
print 'Float32, 1e-12 -->', np.spacing(np.float32(1e-12))

Which yields:

Float64, 1.0 --> 2.22044604925e-16
Float64, 1e12 --> 0.0001220703125
Float64, 1e-12 --> 2.01948391737e-28

Float32, 1.0 --> 1.19209e-07
Float32, 1e12 --> 65536.0
Float32, 1e-12 --> 1.0842e-19

Upvotes: 15

tobias_k
tobias_k

Reputation: 82949

Floating point numbers have a certain precision, to a few decimal places in scientific notation. The larger the number, the larger the least significant digit in that representation, and thus the larger the "epsilon" that could contribute to that number.

Thus, the epsilon is relative to the number it is added to, which is in fact stated in the documentation you cited: "... such that 1.0 + eps != 1.0". If the "reference" number is smaller by, e.g. one order of magnitude, then eps is smaller, too.

If that was not the case, you could not calculate at all with numbers smaller than eps (2.2e-16 in my case).

Upvotes: 11

Related Questions