CODEWITHSUNDEEP
pythonmachine-learningkerasconv-neural-network
akhetos
akhetos

Reputation: 706

conv net save weight and new test set

i'm using conv net for image classification.

There is something I dont understand theoretically

For training I split my data 60%train/20%validation/20% test

I save weight when metric on validation set is the best (I have same performance on training and validation set).

Now, I do a new split. Some data from training set will be on test set. I load the weight and I classify new test set.

Since weight have been computed on a part of the new test set, are we agree to says this is a bad procedure and I should retrain my model with my new training/validation set?

Upvotes: 0

Views: 31

Answers (2)

Thibault Bacqueyrisses
Thibault Bacqueyrisses

Reputation: 2331

The all purpose of having a test set is that the model must never see it until the very last moment.
So if your model trained on some of the data in your test set, it becomes useless and the results it will gives you will have no meaning.

So basicly:

  • 1.Train on your train set
  • 2.Validate on your validation set
  • 3.Repeat 1 and 2 until you are happy with the results
  • 4.At the very end, finally test your model on the test set

Upvotes: 1

Olivier Cruchant
Olivier Cruchant

Reputation: 4037

yes, for fair evaluation no sample in the test set should be seen during training

Upvotes: 2

Related Questions