CODEWITHSUNDEEP
machine-learningneural-networkartificial-intelligence
tristndev
tristndev

Reputation: 447

Train-Validation-Test Split with Neural Networks and Grid Search

I am working in a machine learning setting, using Neural Networks (NN).

Theoretically, I am familiar with the general practice of partitioning the available dataset into three parts:

I want to select a good network architecture using grid search (my problem is not too complex, and datasets not too big, so this is feasible), but I am unsure on how to apply this three-way split to my problem.

Fitting a neural network alone (e.g. with keras), so just learning the weights, allows splitting your dataset into training and validation sets (using an EarlyStopping callback even requires this).

This means, doing grid search, I will already fit the model using the training and validation set (i.e., the validation set will have an influence on how the model looks). This would mean, the former test set somehow becomes the new validation set, that I would use to pick the best performing architecture in the grid search.

Assessment of model performance could then be done in a second step, using cross validation on the complete dataset.

Is this approach correct or do I have it mixed up?

Upvotes: 1

Views: 735

Answers (1)

joek47
joek47

Reputation: 126

This means, doing grid search, I will already fit the model using the training and validation set (i.e., the validation set will have an influence on how the model looks).

Yes, grid search helps to find the best hyper parameters using cross-validation splitting strategy.

This would mean, the former test set somehow becomes the new validation set, that I would use to pick the best performing architecture in the grid search.

Use your test set to see the actual performance. Grid search will return the best hyper parameters for your model based on cross-validation sets.

Upvotes: 1

Related Questions