CODEWITHSUNDEEP
pythonpandasnumpy
BhishanPoudel
BhishanPoudel

Reputation: 17144

Confused by pandas dtype conversion to np.float16 value 2053 becomes 2052

I was trying to reduce memory consumption by downcasting float data types.

I checked the range of np.float16:

np.finfo(np.float16)
finfo(resolution=0.001, min=-6.55040e+04, max=6.55040e+04, dtype=float16)

This shows -6.55040e+04 < 2053 < 6.55040e+04

Now:

s = pd.Series([2051,2052,2053,2054])
s.astype(np.float16)

0    2052.0
1    2052.0
2    2052.0
3    2054.0

Why is it so?

UPDATE
Borrowing documentation from np.finfo

numpy.finfo
class numpy.finfo[source]
Machine limits for floating point types.

Parameters: 
dtype : float, dtype, or instance

Kind of floating point data-type about which to get information.

Attributes

eps (float) The smallest representable positive number such that 1.0 + eps != 1.0. Type of eps is an appropriate floating point type.
epsneg  (floating point number of the appropriate type) The smallest representable positive number such that 1.0 - epsneg != 1.0.
iexp    (int) The number of bits in the exponent portion of the floating point representation.
machar  (MachAr) The object which calculated these parameters and holds more detailed information.
machep  (int) The exponent that yields eps.
max (floating point number of the appropriate type) The largest representable number.
maxexp  (int) The smallest positive power of the base (2) that causes overflow.
min (floating point number of the appropriate type) The smallest representable number, typically -max.
minexp  (int) The most negative power of the base (2) consistent with there being no leading 0’s in the mantissa.
negep   (int) The exponent that yields epsneg.
nexp    (int) The number of bits in the exponent including its sign and bias.
nmant   (int) The number of bits in the mantissa.
precision   (int) The approximate number of decimal digits to which this kind of float is precise.
resolution  (floating point number of the appropriate type) The approximate decimal resolution of this type, i.e., 10**-precision.
tiny    (float) The smallest positive usable number. Type of tiny is an appropriate floating point type.

Upvotes: 5

Views: 1104

Answers (1)

gmds
gmds

Reputation: 19885

It's not about min or max, which determine the lowest and highest values a float16 can take respectively, but resolution, or the least difference between two values before they are considered identical.

finfo shows that the resolution of float16 is 0.001, or 4 significant figures. The digit 2 in your case is the 4th significant figure.

Upvotes: 4

Related Questions