CODEWITHSUNDEEP
javainteger-division
nmokc
nmokc

Reputation: 41

Division by integer in Java yields incorrect value of 2147483

This is the method I am calling to essentially calculate the mean of 1000 integers:

public void setMeanSampleSize(ResultData[] R) {

    double temp = 0;
    for(int i = 0;i < R.length;i++){

        temp = temp + R[i].getTotal();
    }
    MeanSampleSize = (int) temp/R.length;
}

The variable Total which is part of the ResultData object is an integer.

In 99.99% of the cases, this works correctly but in some odd cases, the code yields an incorrect result of 2147483 for MeanSampleSize (which is eerily similar to 2,147,483,647 the 32-bit integer max).

Can anyone help understand why this division will work sometimes and give completely erroneous result of 2147483 on other occasions?

Thanks in advance!

Upvotes: 1

Views: 485

Answers (1)

T.J. Crowder
T.J. Crowder

Reputation: 1073968

You're building up your total in a double, and then doing this:

MeanSampleSize = (int) temp/R.length;

which is

MeanSampleSize = ((int) temp)/R.length;

As Radiodef said, if temp's value is greater than Integer.MAX_VALUE, (int)temp will be Integer.MAX_VALUE (2147483647), which when divided by 1000 (with integer division) is ... 2147483.

You probably wanted to divide first (as a double), then truncate to int:

MeanSampleSize = (int) (temp / R.length);

Worth noting the various comments, though, that doing this with a double probably isn't your best bet unless getTotal returns a double and you want those fractional values to accumulate. But it may be worth using a long (and adjusting your parens as above).

Upvotes: 2

Related Questions