CODEWITHSUNDEEP
pythonscikit-learn
T0r0nt0
T0r0nt0

Reputation: 97

ColumnTransformer generating a TypeError when trying to fit_transform pipeline in sklearn

im hoping someone here will be able to help me debug a part of my code. I am trying to come up with a predictive model for the Ames, Iowa housing set for a Kaggle competition and im having an issue implementing my pipeline as I keep getting an error. here is the code I am trying to run

from sklearn.preprocessing import OneHotEncoder
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer


num_attributes = list(train_set.select_dtypes(exclude=['object'])) #to select all num columns, we exclude any column with object types
cat_attributes = list(train_set.select_dtypes(include=['object'])) #here we select all columns with object types

cat_pipeline = ([
    ('imputer', SimpleImputer(fill_value='none', strategy='constant')),
    ('one_hot', OneHotEncoder())
])

full_pipeline = ColumnTransformer([
    ('num', StandardScaler(), num_attributes),
    ('cat', cat_pipeline, cat_attributes)
])

train_set_prepared = full_pipeline.fit_transform(train_set)

and this is the error message I am getting

--------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-14-abf9d30bdc2b> in <module>
     20 ])
     21 
---> 22 train_set_prepared = full_pipeline.fit_transform(train_set)

~\Anaconda3\envs\ml_book\lib\site-packages\sklearn\compose\_column_transformer.py in fit_transform(self, X, y)
    470         """
    471         X = _check_X(X)
--> 472         self._validate_transformers()
    473         self._validate_column_callables(X)
    474         self._validate_remainder(X)

~\Anaconda3\envs\ml_book\lib\site-packages\sklearn\compose\_column_transformer.py in _validate_transformers(self)
    277                                 "transform, or can be 'drop' or 'passthrough' "
    278                                 "specifiers. '%s' (type %s) doesn't." %
--> 279                                 (t, type(t)))
    280 
    281     def _validate_column_callables(self, X):

TypeError: All estimators should implement fit and transform, or can be 'drop' or 'passthrough' specifiers. '[('imputer', SimpleImputer(add_indicator=False, copy=True, fill_value='none',
              missing_values=nan, strategy='constant', verbose=0)), ('one_hot', OneHotEncoder(categorical_features=None, categories=None, drop=None,
              dtype=<class 'numpy.float64'>, handle_unknown='error',
              n_values=None, sparse=True))]' (type <class 'list'>) doesn't.

I know the issue is specifically the cat_pipeline. does anyone know what the issue could be? thanks for the help

Upvotes: 2

Views: 2201

Answers (1)

T0r0nt0
T0r0nt0

Reputation: 97

I figured it out. I forgot to initiate the Pipeline in cat_pipeline this is what it should say

cat_pipeline = Pipeline([ # HERE
    ('imputer', SimpleImputer(fill_value='none', strategy='constant')),
    ('one_hot', OneHotEncoder())

Upvotes: 3

Related Questions