CODEWITHSUNDEEP
.netdecimal
TheSean
TheSean

Reputation: 4566

How do I compare two decimals to 10 decimal places?

I'm using decimal type (.net), and I want to see if two numbers are equal. But I only want to be accurate to 10 decimal places. For example take these three numbers. I want them all to be equal.

0.1123456789
0.11234567891
0.11234567899

The 10 decimal limit is coming from my database, so I have to assume that the first number was already rounded, and therefore I can't simply round the others because the last one will round up.

I really just want to truncate the to 10 decimal places, but can't see how to do this either.

Upvotes: 6

Views: 4685

Answers (5)

R. Martinho Fernandes
R. Martinho Fernandes

Reputation: 234654

What about multiplying by 10^10 and dropping the fractional part?

decimal x2 = Math.Truncate(x * 10000000000);
decimal y2 = Math.Truncate(y * 10000000000);
Assert.Equals(x2, y2);

EDIT: Changed to Math.Truncate by Aaron's suggestion. Thanks.

Upvotes: 8

zeocrash
zeocrash

Reputation: 665

Multiply by 10^10 Rather than converting to an integer (which is 32 bit) it might be worth using an int64. An int 32 has a limit of approximatly 2 billion which when multipled out gives a range between +2.1 and -2.1 on your decimal value, an int64 has a much larger range that, when multiplied out gives you a range of approximatly +922,000,000 to -922,000,000 on the decimal value

http://msdn.microsoft.com/en-us/library/system.int32.aspx

http://msdn.microsoft.com/en-us/library/system.int64.aspx

Upvotes: 0

Russell Newquist
Russell Newquist

Reputation: 2666

The same way you'd compare floating point numbers. Here's some pseudocode, because I don't know the .NET call for absolute value, but it will essentially look like this (modify the constant for the precision needed):

if( Math.Abs( value1 - value2 ) < 0.0000000001 )
{
  // blah blah
}

Upvotes: 11

Bobby
Bobby

Reputation: 11576

Maybe this works for you:

If Decimal.Round(yourDec1, 10, MidpointRounding.AwayFromZero) = Decimal.Round(yourDec2, 10, MidpointRounding.AwayFromZero) Then

Bobby

Upvotes: -1

James Black
James Black

Reputation: 41838

Multiply by 10000000000, convert to an int, then divide by the same number.

That way you truncate the the excess digits.

You may want to add .5 * 10^-11 in order to round properly before truncating.

Upvotes: 1

Related Questions