CODEWITHSUNDEEP
pythonscikit-learn
lte__
lte__

Reputation: 7576

Python/sklearn - preprocessing.MinMaxScaler 1d deprecation

I'd like to scale a column of a dataframe to have values between 0 and 1. For this I'm using a MinMaxScaler, which works fine, but is sending me mixed messages. I'm doing:

x = df['Activity'].values #returns a numpy array
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
df['Activity'] = pd.Series(x_scaled)

Message numero uno for this code is a warning:

DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.

Okay, so apparentyl having 1d arrays is gonna be a no-no soon, so let's try to reshape it as advised:

x = df['Activity'].values.reshape(-1, 1)

Now the code doesn't even run: Exception: Data must be 1-dimensional is thrown. So I'm confused. 1d is going to be deprecated soon, but the data also has to be 1d?? How to do this safely? What's the issue here?

EDIT as requested by @sascha

x looks like this: 

array([ 0.00568953,  0.00634314,  0.00718003, ...,  0.01976002,
        0.00575024,  0.00183782])

And after reshaping:

array([[ 0.00568953],
       [ 0.00634314],
       [ 0.00718003],
       ..., 
       [ 0.01976002],
       [ 0.00575024],
       [ 0.00183782]])

The whole warning:

/usr/local/lib/python3.5/dist-packages/sklearn/preprocessing/data.py:321: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
  warnings.warn(DEPRECATION_MSG_1D, DeprecationWarning)
/usr/local/lib/python3.5/dist-packages/sklearn/preprocessing/data.py:356: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
  warnings.warn(DEPRECATION_MSG_1D, DeprecationWarning)

The error when I reshape:

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-132-df180aae2d1a> in <module>()
      2 min_max_scaler = preprocessing.MinMaxScaler()
      3 x_scaled = min_max_scaler.fit_transform(x)
----> 4 telecom['Activity'] = pd.Series(x_scaled)

/usr/local/lib/python3.5/dist-packages/pandas/core/series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    225             else:
    226                 data = _sanitize_array(data, index, dtype, copy,
--> 227                                        raise_cast_failure=True)
    228 
    229                 data = SingleBlockManager(data, index, fastpath=True)

/usr/local/lib/python3.5/dist-packages/pandas/core/series.py in _sanitize_array(data, index, dtype, copy, raise_cast_failure)
   2918     elif subarr.ndim > 1:
   2919         if isinstance(data, np.ndarray):
-> 2920             raise Exception('Data must be 1-dimensional')
   2921         else:
   2922             subarr = _asarray_tuplesafe(data, dtype=dtype)

Exception: Data must be 1-dimensional

Upvotes: 9

Views: 23195

Answers (1)

Ilya V. Schurov
Ilya V. Schurov

Reputation: 8047

You can simply drop pd.Series:

import pandas as pd
from sklearn import preprocessing
df = pd.DataFrame({'Activity': [ 0.00568953,  0.00634314,  0.00718003, 
                                0.01976002, 0.00575024,  0.00183782]})
x = df['Activity'].values.reshape(-1, 1) #returns a numpy array
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
df['Activity'] = x_scaled

or you can explicitly get the first column of x_scaled:

df['Activity'] = pd.Series(x_scaled[:, 0])

Upvotes: 24

Related Questions