Cjueden
Cjueden

Reputation: 1200

qt returns bad math

mlAnswer = ( ( ( degreesPLato->text().toInt() * 1000000 ) * 3800 ) / answer );

is the code in quesition mlAnswer out puts -8223, while my calculator puts out 228000

debug output

12 * 1000000 * 3800 / 200000 = -8223

all data types are ints Please tell me what I'm doing wrong.

Upvotes: 0

Views: 92

Answers (2)

Yuushi
Yuushi

Reputation: 26040

You can fix this by reordering your operations:

12 * 1000000 * 3800 / 200000

Will overflow an int, however:

12 * 1000000 / 200000 * 3800

will not.

Note that this will only give the same answer if the numerator is an integer multiple of the denominator. Using LL is a better solution on platforms that support it, but if you are constrained to a 4 byte int type, this will at least stop overflow in more situations.

Upvotes: 1

Benjamin Lindley
Benjamin Lindley

Reputation: 103713

12 * 1000000 * 3800 = 45.6 billion.

This is out of range for a 4 byte signed integer, which is what int usually is. Try using long long instead.

The default type of an integer literal is int, unless the number is too big to fit in an int. As long as you are doing math operations between ints, the results remain as ints. 12 is an int, 1000000 is an int, and 3800 is an int. When you multiply them together, the result is still an int, even though it no longer fits. Add the LL suffix to make the integer literal a long long. i.e. 12LL, 1000000LL, 3800LL, etc...

Upvotes: 5

Related Questions