CODEWITHSUNDEEP
pythonmachine-learningkaggle
mchd
mchd

Reputation: 3163

ValueError while implementing the train_test_split

I'm following through the Machine Learning tutorial on Kaggle and I have a ValueError despite following the tutorial line by line. I'm trying to practice data validation with splitting. This is my code:

import pandas as pd
from sklearn.tree import DecisionTreeRegressor
from sklearn.model_selection import train_test_split


main_file_path = '../input/train.csv'
data = pd.read_csv(main_file_path)

y = data.SalePrice

data_predictors = ['LotArea', 'YearBuilt', '1stFlrSF', '2ndFlrSF', 'FullBath', 'BedroomAbvGr', 'TotRmsAbvGrd']
x = data[data_predictors]

train_x, val_x, train_y, val_x = train_test_split(x, y,random_state = 0)

data_model = DecisionTreeRegressor()
data_model.fit(train_x,train_y)

data_prediction = data_model.predict(val_x)
print(mean_absolute_error(val_y, data_prediction))

The error is pointing at this line:

data_prediction = data_model.predict(val_x)

I'm a beginner in ML learning so I compared my code with that of the authors and the implementations were the same.

Full stack trace:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-48f37072f996> in <module>()
     17 data_model.fit(train_x,train_y)
     18 
---> 19 data_prediction = data_model.predict(val_x)
     20 print(mean_absolute_error(val_y, data_prediction))

/opt/conda/lib/python3.6/site-packages/sklearn/tree/tree.py in predict(self, X, check_input)
    410         """
    411         check_is_fitted(self, 'tree_')
--> 412         X = self._validate_X_predict(X, check_input)
    413         proba = self.tree_.predict(X)
    414         n_samples = X.shape[0]

/opt/conda/lib/python3.6/site-packages/sklearn/tree/tree.py in _validate_X_predict(self, X, check_input)
    371         """Validate X whenever one tries to predict, apply, predict_proba"""
    372         if check_input:
--> 373             X = check_array(X, dtype=DTYPE, accept_sparse="csr")
    374             if issparse(X) and (X.indices.dtype != np.intc or
    375                                 X.indptr.dtype != np.intc):

/opt/conda/lib/python3.6/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
    439                     "Reshape your data either using array.reshape(-1, 1) if "
    440                     "your data has a single feature or array.reshape(1, -1) "
--> 441                     "if it contains a single sample.".format(array))
    442             array = np.atleast_2d(array)
    443             # To ensure that array flags are maintained

ValueError: Expected 2D array, got 1D array instead:

Upvotes: 4

Views: 2805

Answers (1)

enumaris
enumaris

Reputation: 1938

Although the error arose from the line you pointed out, the actual problem is in this line:

    train_x, val_x, train_y, val_x = train_test_split(x, y,random_state = 0)

Notice that you have two val_x in there. The second val_x should be val_y. What happened was, you set val_x, which should be a 2-D array of inputs, to what should have been y values which are 1-D arrays of predictions - thereby getting that ValueError saying you input a 1-D array where a 2-D array was expected.

Upvotes: 2

Related Questions