CODEWITHSUNDEEP
pythonnumpytype-conversion
hawk_sf
hawk_sf

Reputation: 55

Numpy's dtype conversion algorithm

How does numpy scale values, when you convert an array from a float dtype to an integer dtype, if you have an array with a max value higher than what the integer type can hold?

In [9]: data_array.dtype
Out[9]: dtype('<f4')

In [11]: data_array.max()
Out[11]: 32767.0

In [16]: test = np.asarray(data_array, dtype=np.int8)

In [17]: test.max()
Out[17]: 127

In [18]: data_array.max()/test.max()
Out[18]: 258.00787

How did numpy arrive at a scale factor of 258?

Thanks for the help.

Upvotes: 4

Views: 458

Answers (2)

ecatmur
ecatmur

Reputation: 157314

They're not the same element of the array.

Numpy converts from floating to integer types by converting to int and then truncating the binary representation, so 32767.0 will convert to the integer 32767 (0x7fff) and then to 0xff, which is -1 in int8.

The 127 is coming from another array element whose integer value is congruent to 127 modulo 256.

Upvotes: 3

Joran Beasley
Joran Beasley

Reputation: 113940

its not a scale ... its just using 8 bits for a signed integer ...

32767 & 0b11111111 = 0b11111111 = 255 (unsigned)

int 8 is signed so it only gets 7 bits with the 8th being the sign bit

0b01111111 = 127

Upvotes: 2

Related Questions