CODEWITHSUNDEEP
cross-validationskorch
Xiao Zhao
Xiao Zhao

Reputation: 23

Output of cross_val_predict

I used skorch to train my model which is defined as below,

net_reg = NeuralNetRegressor(
    Network_binaryDC,
    batch_size=32,
    lr=0.01,
    max_epochs=1000,
    criterion=nn.MSELoss,
    optimizer=torch.optim.Adam,
    train_split=None,
    iterator_train__shuffle=True
)

When I use the following two method (with or without 5-CV) to predict result, I found the result from 5-CV is much worse.

Method 1

net_reg.fit(X, y)
y_pred = net_reg.predict(X)
mse_train = mean_squared_error(y, y_pred)
r2_train = r2_score(y, y_pred)

Method 2

y_pred = cross_val_predict(net_reg, X, y, cv=5)
mse_train = mean_squared_error(y, y_pred)
r2_train = r2_score(y, y_pred)

Here is the output.

Method 1

epoch    train_loss     dur
-------  ------------  ------
...
996       23.6809  0.0090
997       23.8153  0.0080
998       24.9554  0.0090
999       25.1953  0.0090
1000       28.5202  0.0100

mse_train: 27.771873
r2_train: 0.9892790619950554

Method 2

epoch    train_loss     dur
-------  ------------  ------
...
996       36.4650  0.0090
997       31.8118  0.0090
998       31.5955  0.0100
999       28.3348  0.0100
1000       30.0020  0.0080

mse_train: 2985.767
r2_train: 0.23378432943403352

The train_loss in the end of my epochs are nearly the same for two methods, but why the mse and r2 are much worse when using 5-CV with cross_val_predict?

Upvotes: 2

Views: 186

Answers (0)

Related Questions