CODEWITHSUNDEEP
pythonpandastensorflowkeras
Danny Tuppeny
Danny Tuppeny

Reputation: 42453

How to use ordered categorical columns in keras ("could not convert string to float: 'CATEGORY'")

I'm working on the Kaggle House Prices competition and the dataset has a lot of categorical data. I'm trying to set some them as ordered categories like this:

for col in ordered_category_rating_cols:
    data[col] = data[col].astype(pd.api.types.CategoricalDtype(ordered=True, categories = ["GLQ", "ALQ", "BLQ", "Rec", "LwQ", "Unf", "NA"]))

However when I get to passing the data into model.fit() is throws this error (full stack is below):

ValueError: could not convert string to float: 'GLQ'

By stripping out a bunch of columns, I narrowed it down to one - but if I print the dtype for that, it looks correct:

> train_x["BsmtFinType1"].dtype
> CategoricalDtype(categories=['GLQ', 'ALQ', 'BLQ', 'Rec', 'LwQ', 'Unf', 'NA'], ordered=True)

I've searched high and low, but can't find any solution to this. Do I need to do something to tell Keras to treat the categories as floats?

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-144-c86afee8eb19> in <module>()
      4     batch_size=128,
      5     epochs=6,
----> 6     validation_split=0.1
      7 )
      8 

3 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
    778           validation_steps=validation_steps,
    779           validation_freq=validation_freq,
--> 780           steps_name='steps_per_epoch')
    781 
    782   def evaluate(self,

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training_arrays.py in model_iteration(model, inputs, targets, sample_weights, batch_size, epochs, verbose, callbacks, val_inputs, val_targets, val_sample_weights, shuffle, initial_epoch, steps_per_epoch, validation_steps, validation_freq, mode, validation_in_fit, prepared_feed_values_from_dataset, steps_name, **kwargs)
    361 
    362         # Get outputs.
--> 363         batch_outs = f(ins_batch)
    364         if not isinstance(batch_outs, list):
    365           batch_outs = [batch_outs]

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in __call__(self, inputs)
   3275         tensor_type = dtypes_module.as_dtype(tensor.dtype)
   3276         array_vals.append(np.asarray(value,
-> 3277                                      dtype=tensor_type.as_numpy_dtype))
   3278 
   3279     if self.feed_dict:

/usr/local/lib/python3.6/dist-packages/numpy/core/numeric.py in asarray(a, dtype, order)
    536 
    537     """
--> 538     return array(a, dtype, copy=False, order=order)
    539 
    540 

ValueError: could not convert string to float: 'GLQ'

Upvotes: 1

Views: 641

Answers (2)

tawab_shakeel
tawab_shakeel

Reputation: 3749

The way I use to convert categorical columns to data

import pandas as pd
df = pd.DataFrame(data={"gender":["male","female"]})
df['gender'] = df['gender'].astype('category').cat.codes


  gender
0   1
1   0

in case multiple columns contains categorical data

category_columns = list(df.select_dtypes(['category']).columns)
df[category_columns] = df[category_columns].apply(lambda x: x.cat.codes)

Upvotes: 2

BENY
BENY

Reputation: 323396

You can convert the category data to their codes which is same to sklearn.preprocessing.LabelEncoder

df.col=df.col.cat.codes

Upvotes: 1

Related Questions