CODEWITHSUNDEEP
pythonpython-3.xnumpymathrounding
John
John

Reputation: 527

Getting 'large' errors that seem too big to be from rounding alone

When I use Python, I'm getting some significant rounding errors that seem too large for simply floating point problems. I have the following variables:

p = 2.2*10**9
m = 0.510999*10**6

I then send it through the following:

b = 1/np.sqrt((m/p)**2 + 1) = 0.99999997302479693

I then use this value through another equation that should return p:

p = (1/np.sqrt(1-b**2)) * m * b = 2200000008.1937...

Aforementioned gives a difference in p of 8.19... (error in the 9th decimal place if scientific notation is used), which seems to be way too big to be simply a matter of rounding.

I've tried using Decimal().sqrt() with arbitrarily high precision for all the calculations and I get a difference in p of 1.8935..., which is only marginally better.

Is there a better way of getting higher precision?

Upvotes: 3

Views: 126

Answers (1)

Lutz Lehmann
Lutz Lehmann

Reputation: 26040

It is the operation

sqrt(1+x)

that loses you that much precision. Or really the 1+x part of it. As your x=(m/p)**2 has the magnitude 1e-6, you lose about 5-6 digits of the 15-16 valid decimal digits of x, so that only 9-10 valid digits remain. And in the reconstruction you see exactly that, (only) the leading 9 digits are correct.

Upvotes: 7

Related Questions