CODEWITHSUNDEEP
pythonnumpynandtype
KoKo
KoKo

Reputation: 379

Convert nan to Zero when numpy dtype is "object"

I have a numpy array that contains nan. I attempted to convert those nans to zeros using 

 X_ = np.nan_to_num(X_, copy = False)

but it didn't work. I suspect its because dtype of X_ is object. I attempted to convert that to float64 using 

X_= X_.astype(np.float64)

but that didn't work either

Is there a way to convert nan to zero when dtype is object?

Upvotes: 2

Views: 2012

Answers (4)

HeyWatchThis
HeyWatchThis

Reputation: 23433

The "object" dtype was causing me a problem too. But your astype(np.float64) actually did work for me. Thanks!

print("Creating a numpy array from a mixed type DataFrame can create an 'object' numpy array dtype:")
A = np.array([1., 2., 3., np.nan]); print('A:', A, A.dtype)
B = pd.DataFrame([[1., 2., 3., np.nan,],  [1, 2, 3, '4']]
                  ).to_numpy();  print('B:', B, B.dtype, '\n')

print('Converting vanilla A is fine:\n', np.nan_to_num(A, nan=-99), '\n')
print('But not B:\n', np.nan_to_num(B, nan=-99), '\n')
print('Not even this slice of B, \nB[0, :] : ', B[0, :])
print(np.nan_to_num(B[0, :], nan=-99), '\n')

print('The astype(np.float64) does the trick here:\n', 
      np.nan_to_num(B[0, :].astype(np.float64), nan=-99), '\n\n')

Output:

Creating a numpy array from a mixed type DataFrame can create an 'object' numpy array dtype:
A: [ 1.  2.  3. nan] float64
B: [[1.0 2.0 3.0 nan]
 [1.0 2.0 3.0 '4']] object 

Converting vanilla A is fine:
 [  1.   2.   3. -99.] 

But not B:
 [[1.0 2.0 3.0 nan]
 [1.0 2.0 3.0 '4']] 

Not even this slice of B, 
B[0, :] :  [1.0 2.0 3.0 nan]
[1.0 2.0 3.0 nan] 

The astype(np.float64) does the trick here:
 [  1.   2.   3. -99.]

Upvotes: 0

Paul Panzer
Paul Panzer

Reputation: 53029

If your array contains only "reasonable" (see below) elements then you can use the following work around:

np.where(X_==X_,X_,0)

By reasonable I mean that an element e satisfies e==e with the sole exception of nan. This should for example be the case if no userdefined classes are used as elements.

Upvotes: 0

Carsten
Carsten

Reputation: 3048

Seems that because of the object type, a conversion to float does not work. Might be a bit hacky, but you could try to convert to str:

X_.astype(str).replace('np.NaN', 0).astype(float)

Upvotes: 0

one
one

Reputation: 2585

a  = [np.nan]
b = np.array(a)
c = np.nan_to_num(b)
print(b)
print(c)

result:

[nan]
[0.]

it works. check you X_ format.

Upvotes: 0

Related Questions