CODEWITHSUNDEEP
machine-learningencodingscikit-learnsklearn-pandasordinal
jeffhale
jeffhale

Reputation: 4032

How to map categorical data to category_encoders.OrdinalEncoder in python pandas dataframe

I'm trying to use category_encoders.OrdinalEncoder to map categories to integers in a pandas dataframe. But I'm getting the following error without any other helpful hints.

TypeError: 'NoneType' object is not iterable

Code runs fine without the attempted mapping, but I'd like the mapping.

Code:

import category_encoders as ce

ordinal_cols = [
    "ExterQual",
]

ordinal_cols_mapping = [{
    "ExterQual": {
        'Ex': 5,
        'Gd': 4,
        'TA': 3, 
        'Fa': 2,
        'Po': 1,
        'NA': NaN
    }},
]

encoder = ce.OrdinalEncoder(  mapping = ordinal_cols_mapping, return_df = True, cols = ordinal_cols,)  

df_train = encoder.fit_transform(train_data)
print(df_train)

What am I dong wrong?

mapping: list of dict a mapping of class to label to use for the encoding, optional.

http://contrib.scikit-learn.org/categorical-encoding/ordinal.html

Full Stack Trace:

---------------------------------------------------------------------------
TypeError                                 
Traceback (most recent call last)
<ipython-input-56-4944c8d41d07> in <module>()
    150 # use the Ordinal Encoder to map the ordinal data to interval and then fit transform
    151 encoder = ce.OrdinalEncoder( return_df = True, cols = ordinal_cols, mapping = ordinal_cols_mapping)  #NaNs get -1, mapping = ordinal_cols_mapping removed due to error
--> 152 X = encoder.fit_transform(X)

/opt/conda/lib/python3.6/site-packages/sklearn/base.py in fit_transform(self, X, y, **fit_params)
    515         if y is None:
    516             # fit method of arity 1 (unsupervised transformation)
--> 517             return self.fit(X, **fit_params).transform(X)
    518         else:
    519             # fit method of arity 2 (supervised transformation)

/opt/conda/lib/python3.6/site-packages/category_encoders/ordinal.py in fit(self, X, y, **kwargs)
    130             cols=self.cols,
    131             impute_missing=self.impute_missing,
--> 132             handle_unknown=self.handle_unknown
    133         )
    134         self.mapping = categories

/opt/conda/lib/python3.6/site-packages/category_encoders/ordinal.py in ordinal_encoding(X_in, mapping, cols, impute_missing, handle_unknown)
    249             for switch in mapping:
    250                 X[str(switch.get('col')) + '_tmp'] = np.nan
--> 251                 for category in switch.get('mapping'):
    252                     X.loc[X[switch.get('col')] == category[0], str(switch.get('col')) + '_tmp'] = str(category[1])
    253                 del X[switch.get('col')]

TypeError: 'NoneType' object is not iterable

Example data:

0    0
1    1
2    0
3    1
4    0
Name: ExterQual, dtype: int64

Upvotes: 7

Views: 12978

Answers (2)

dennisobrien
dennisobrien

Reputation: 1198

In case all of your columns to encode are already pandas categoricals, you can construct a mapping like this.

In [82]:
from category_encoders.ordinal import OrdinalEncoder
import pandas as pd
from pandas.api.types import CategoricalDtype

# define a categorical dtype
platforms = ['android', 'ios', 'amazon']
platform_category = CategoricalDtype(categories=platforms, ordered=False)

# create a dataframe
df = pd.DataFrame([
    {'id': 1, 'platform': 'android'},
    {'id': 2, 'platform': 'ios'},
    {'id': 3, 'platform': 'amazon'},
])
# apply the categorical dtype
df['platform'] = df['platform'].astype(platform_category)

# create a mapping from all categorical columns that can be used with OrdinalEncoder
categorical_columns = list(df.select_dtypes(['category']).columns)
category_mapping = [
    {'col': column_name, 'mapping': list(zip(df[column_name].cat.categories, df[column_name].cat.codes))} 
    for column_name in categorical_columns
]

# pass this as the mapping
cat_encoder = OrdinalEncoder(cols=categorical_columns, mapping=category_mapping)
cat_encoder

Out[82]:
OrdinalEncoder(cols=['platform'], drop_invariant=False,
        handle_unknown='impute', impute_missing=True,
        mapping=[{'col': 'platform', 'mapping': [('android', 0), ('ios', 1), ('amazon', 2)]}],
        return_df=True, verbose=0)

Upvotes: 2

Vivek Kumar
Vivek Kumar

Reputation: 36599

You are using the 'mapping' param wrong.

The format should be:

'mapping' param should be a list of dicts where internal dicts should contain the keys 'col' and 'mapping' and in that the 'mapping' key should have a list of tuples of format (original_label, encoded_label) as value.

Something like this:

ordinal_cols_mapping = [{
    "col":"ExterQual",    
    "mapping": [
        ('Ex',5), 
        ('Gd',4), 
        ('TA',3), 
        ('Fa',2), 
        ('Po',1), 
        ('NA',np.nan)
    ]},
]

Then no need to set the 'cols' param separately. Column names will be used from the 'mapping' param.

Just do this:

encoder = OrdinalEncoder(mapping = ordinal_cols_mapping, 
                         return_df = True)  
df_train = encoder.fit_transform(train_data)

Hope that this makes it clear.

Upvotes: 5

Related Questions