CODEWITHSUNDEEP
c++overflowaddition
CommanderDJ
CommanderDJ

Reputation: 31

Integer overflow despite large enough type?

A bit of code I'm working on has stumped me. I appear to be dealing with integer overflow, which I admit I haven't encountered before, but I've tried several things to fix it and none of them have worked so far. I'm using MSVC++ 2010.

The relevant code is very simple, and is as follows:

int64_t numerator = cdfVal * (maxIntensity - 1);

This is failing for me for the case where cdfVal (this is just a value read from an array, nothing fancy) is 8667003 and maxIntensity - 1 is 255. A simple calculation shows that the solution ought to be 2210085765, whereas I'm getting the value -2084881531. I understand that this is a case of integer overflow, but I thought 64 bits would be large enough to hold the result. I switched to uint64_t, but in that case I got a result that was twenty digits long, which is obviously incorrect! I've tried using Boost's multiprecision integers but got similar results with those as well.

I feel like this is something really simple I'm missing.

Upvotes: 2

Views: 83

Answers (1)

Angew is no longer proud of SO
Angew is no longer proud of SO

Reputation: 171107

The type of numerator is irrelevant to the type in which the expression is evaluated. C++ evaluates expressions in the "biggest type of the operands involved" (disregarding some details).

You mentioned cdfVal and maxIntensity are both of type int. 1 is of type int as well. Which means the whole expression is evaluated in type int (and overflows there). The resulting int is then promoted to int64_t and assigned to numerator.

If you want to get around this, either declare at least one of the variables as int64_t, or cast at least one of them to the desired type. Like this:

int64_t numerator = static_cast<int64_t>(cdfVal) * (maxIntensity - 1);

To be consistent and a little more explicit, you can of course cast both:

int64_t numerator = static_cast<int64_t>(cdfVal) * static_cast<int64_t>(maxIntensity - 1);

Upvotes: 3

Related Questions