CODEWITHSUNDEEP
pythonpython-3.xnumpypython-3.6absolute-value
jk_sri
jk_sri

Reputation: 101

Python abs() function failing on negative number

I'm working in python3.6 on linux and came across a pretty obvious failure of the abs() function. My variable x ended up as a very large negative number (possibly -inf), but the absolute value abs() function still returned a negative number, which shouldn't be possible. I put in a quick fix for my code by just adding 0.1 to the input of abs() but.... am I misunderstanding how abs() should be used?

$> x
-9223372036854775808

$> abs(x)
-9223372036854775808

$> np.abs(x)
-9223372036854775808

$> abs(x+.1)
9.223372036854776e+18

$> np.abs(x+.1)
9.223372036854776e+18

EDIT: Solved below, but it boils down to x being a numpy.int64 and not just int, unbeknownst to me.

Upvotes: 2

Views: 2607

Answers (1)

ShadowRanger
ShadowRanger

Reputation: 155363

You didn't think to mention it (I inferred it from your tests with np.abs), but it's important that x is a numpy.int64 (or equivalent signed 64 bit type). That specific value, in two's complement, has no positive equivalent, so abs just produces the same value again (it could be made to raise an exception, but numpy stuck with the low level C behavior where it returns the original value in this case).

Convert it to a true Python int first, e.g. abs(int(x)) and it will work.

Explanation of why it works this way:

The bit pattern of -9223372036854775808 is 0x8000_0000_0000_0000 (highest bit only is set, underscores for readability). Two's complement negation is handled algorithmically by flipping all the bits and then adding one, with carry, so the conversion changes 0x8000_0000_0000_0000 to 0x7fff_ffff_ffff_ffff (all bits flipped), then adds 1, which carries the whole length of the field (since every bit but the high bit is set), producing 0x8000_0000_0000_0000 again. That same bit pattern does actually correspond to the bit pattern an unsigned 64 bit quantity equal to 9223372036854775808 would have, but given it's interpreted as signed, it continues to be interpreted as the most negative value, not one higher than the most positive value int64 value (which can't be represented as an int64).

Upvotes: 8

Related Questions