CODEWITHSUNDEEP
c#.netfloating-point
ediblecode
ediblecode

Reputation: 11971

Float is converting my values

I have a method that tests a value is within the range allowed on fields. If it is outside the range returns null and if inside returns the value.

internal float? ExtractMoneyInRangeAndPrecision(string fieldValue, string fieldName, float min, float max, int scale, int lineNumber)
    {          
        float returnValue;

        //Check whether valid float if
        if (float.TryParse(fieldValue, out returnValue))
        {
            //Check whether in range
            if (returnValue >= min && returnValue <= max)
            {
                int decPosition = 0;
                decPosition = fieldValue.IndexOf('.');

                if (
                    (decPosition == -1) ||
                    ((decPosition != -1) && (fieldValue.Substring(decPosition, fieldValue.Length - decPosition).Length -1 <= scale))
                    )
                {
                    return returnValue;
                }
            }
        }            

        return null;
    }

Here is my unit test:

[TestMethod()]
    [DeploymentItem("ImporterEngine.dll")]
    public void ExtractMoneyInRangeAndPrecisionTest_OutsideRange()
    {
        MockSyntaxValidator target = new MockSyntaxValidator("", 0);
        string fieldValue = "1000000";
        string fieldName = "";
        float min = 1;
        float max = 999999.99f;
        int scale = 2;
        int lineNumber = 0;
        float? Int16RangeReturned;

        Int16RangeReturned = target.ExtractMoneyInRangeAndPrecision(fieldValue, fieldName, min, max, scale, lineNumber);

        Assert.IsNull(Int16RangeReturned);
    }

As you can see the max is 999999.99 but when the method takes it in it changes it to 1,000,000

Why is this?

Upvotes: 0

Views: 814

Answers (4)

James Johnson
James Johnson

Reputation: 46047

Floating-point types (as defined in C#) are approximate. For precision you should always use decimal.

From MSDN:

The decimal keyword indicates a 128-bit data type. Compared to floating-point types, the decimal type has more precision and a smaller range, which makes it appropriate for financial and monetary calculations. The approximate range and precision for the decimal type are shown in the following table.

http://msdn.microsoft.com/en-us/library/364x0z75.aspx

There seems to be some dispute about what qualifies as a floating-point type in C#. While decimal does qualify as a floating-point type by actual definition, it is not defined as such according to the MSDN specification.

http://msdn.microsoft.com/en-us/library/9ahet949.aspx

Upvotes: 1

Jeffrey Sax
Jeffrey Sax

Reputation: 10313

The float type doesn't have enough precision to do what you want to do. I would recommend using the decimal type. Floats can be accurate to 7 decimal digits at most, and you're using 8 here. Decimal can have up to 28 digits, which is more than enough for any amount. Moreover, unlike float, the value the compiler uses and the value you write will always be the same.

Here's the long explanation:

Floats (single-precision floating-point numbers) are stored as an integer times a power of two, where the integer is in a certain range (between 2^23 and 2^24).

When you write a decimal number in your code, the compiler interprets this as the number in this form that is closest to the number you wrote. Sometimes the match is exact (99999.75). In other cases, your number needs to be rounded to the closest floating-point number. This is what happened here:

99999.99 = 2^19 * 1.907348613739013671875
         = 2^19 * 2^-23 * (2^23 * 1.907348613739013671875)
         = 2^-4 * 15999999.84

The closest integer to 15999999.84 is 16000000, so the rounded value is

(float)99999.99 = 2^-4 * 16000000 
                = 1000000

The big advantage of the decimal type is that is represented as a 96 bit integer times a power of 10, so decimal numbers with up to 28 digits can be represented exactly, without any rounding. What you see is what you get.

The biggest disadvantage of decimal is that it is significantly slower, but in a situation like this where you're converting strings to numbers, this is not a factor.

Upvotes: 1

KeithS
KeithS

Reputation: 71573

http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

In short, because of the way floating-point numbers represent real numbers, the number you assign to a float is not always the number you get back out. The value you specify is converted to the nearest value that can be represented in scientific notation with a magnitude determined by a base of 2.

In the case of 999999.99, the nearest number that can be represented as a float with the same number of sig figs is 7.6293945 * 217 = 999999.99504, which when rounded to the same sig figs is 1,000,000.00. This may not be the EXACT case, but error like this is inherent in the use of floats.

Do not use floating-point types in situations where the accuracy of the number at a given level of precision is required. Instead, use the decimal type, which will retain the precision of values entered.

Upvotes: 4

drdwilcox
drdwilcox

Reputation: 3951

Not every string of digits can be converted to a float. Without checking, I would say that 999999.99 is one such number. A decimal would solve this.

Upvotes: 1

Related Questions