Pandas error "AttributeError: 'DataFrame' object has no attribute 'add_categories'" when trying to add catorical values?

Question

I was getting an error earlier 'ValueError: fill value must be in categories' when working with a dataframe. After researching, it appears I need to add categorical options for each value that is a category but I'm getting the below error:

  catgoricalValues = ['embarked', 'sex', 'pclass']

  df[catgoricalValues] = df[catgoricalValues].astype('category')
  df[catgoricalValues] = df[catgoricalValues].add_categories(df[catgoricalValues].unique())  # add options for catgorical values


AttributeError: 'DataFrame' object has no attribute 'add_categories'

What am I doing wrong?

Answer 1

• The error occurs because pandas.Series.cat.add_categories is a Series method, and df[['embarked', 'sex', 'pclass']] is a DataFrame.
• Use pd.Categorical
• pandas: Categorical data
• Some of the titanic dataset columns contain NaNs, which can't be categories.
  • Use .dropna() when creating the categories.

single column

df['embarked'] = pd.Categorical(df['embarked'], categories=df['embarked'].dropna().unique())

multiple columns

# looping through the columns
for col in ['embarked', 'sex', 'pclass']:
    df[col] = pd.Categorical(df[col], categories=df[col].dropna().unique())

# alternatively with .apply
df[['embarked', 'sex', 'pclass']] = df[['embarked', 'sex', 'pclass']].apply(lambda x: pd.Categorical(x, x.dropna().unique(), ordered=True))

• Appending new categories

# create a sample series
s = pd.Series(["a", "b", "c", "a"], dtype="category")

# add a category
s = s.cat.add_categories([4])