CODEWITHSUNDEEP
javaandroid
WmBurkert
WmBurkert

Reputation: 295

multiplication error

If I multiply 12 x 25.4 i get 304.7 whereas I expect 304.8. I see that using odd numbers x 25.4 I get the correct answers but using even numbers always seem to be off by 0.1. I am writing a scientific app that is heavily based on formulas so any insight would be helpful. 

if (((m1_sqs1_spinner.getSelectedItem().toString().equals("in"))))  

    { // start square in inches
         double m1_sqs1_eng = new Double(m1_sqs1.getText().toString()); 

         double square_effective_dia_inch = m1_sqs1_eng;

         double square_effective_dia_mm = square_effective_dia_inch * 25.4;
         m1_ed_mm.setText(Double.toString(square_effective_dia_mm));

    } // end square in inches

Upvotes: 0

Views: 1178

Answers (2)

Hovercraft Full Of Eels
Hovercraft Full Of Eels

Reputation: 285460

I don't know what programming language you're using, but 12 * 25.4 gives me 304.8 when I do this using Java:

public static void main(String[] args) {
   double result = 12 * 25.4;

   System.out.printf("Result: %.1f%n", result );
}

But seriously -- how are you displaying the result obtained? Are you using one of the many numeric formatting options available in Java (one of which is displayed above)? Also do you understand the limits to precision when using 64-bit IEEE 754 floating point (double) variables? They are pretty accurate and useful for most floating-point applications, but not applications that have strict precision requirements such as financial calculations.

Upvotes: 1

jbrookover
jbrookover

Reputation: 5150

Floating point number types in Java are approximations. Try adding 1 + 2 + 3 + 4 as float type and you might end up with 10.00000003.

To ensure accurate math, try using the java.math.BigDecimal type. It is a memory hog, but is accurate.

Upvotes: 2

Related Questions