CODEWITHSUNDEEP
c#
user414396
user414396

Reputation: 11

Errors in finding the Product of two double values in VS8 C#

I am using C# on VS8 professional. I have a piece of code I originally wrote in Pascal then C then C# so has been used without a hitch until now. The fragment of code looks like this:-

*bool InverseTransverseMercator(coord enl)
{
            double nu, rho, k0;
            double x, y, x2, x3, x4, x5, x6, x7;
            double phif, phi, cosphif, sinphif, lam, n, n2, n3, n4;
            double t, t2, t3, t4, t6, nu2, nu3, nu5, nu7, neta2;
            double a0, a1, a2, a3, p7, p8, p9, p10, p11, p12, p12a;
            coord cc; //has a structure public double x, y, z;
            cc = enl;
            x = cc.x;
            y = cc.y;
            x2 = x * x;
            x3 = x * x2;//I stop the debugger at the start of this line and examine some values.
...*

Here are the values from the debugger that have me really confused:-

x = 23500.0  - original value
x * x = 552250000.0 (right hand side of x2=x * x  correct)
x2 = 552249984.0 (left hand side of x2 = x * x INCORRECT!)

As you can see the x * x value is correct yet the value in the variable to which it is assigned (X2) is not. In fact the answers differ by 16. These are all double variables and I would have expected rounding at the 16th significant digit. I have tried renaming the variables, using Math.Pow(x,2), using the 'decimal' type and all have the problem that the value for x2 is not what it should be. I tried rearranging the lines and adding in more dummy lines without a difference. Searching through my code I have no global property name x2. Interestingly the value does change to near the exact answer as if a colossal rounding error was being done. I must be doing something rather dumb and stupid here but I just can't see it. I would be most grateful for any advice from someone on this issue.

Upvotes: 1

Views: 86

Answers (1)

Hans Passant
Hans Passant

Reputation: 941465

You've most likely got some unmanaged code in your project that changed the FPU control word, switching it to single precision mode. The debugger evaluates the expression on its own thread, it didn't get messed with.

That unmanaged code is typically DirectX. Review its CreateDevice() function with the D3DCREATE_FPU_PRESERVE option. If you didn't write this code then you'll have to talk to your software vendor. One trick that might work is intentionally throwing and catching an exception, not 100% sure.

Upvotes: 1

Related Questions