CODEWITHSUNDEEP
gpucross-validationautoencoderk-fold
Satarnejad
Satarnejad

Reputation: 1

Facing ResourceExhaustedError while training an autoencoder using k-fold cross validation

I'm trying to train an stacked denoising autoencoder. Also, since I have a small data set, I implement 10-fold cross validation to find the best hyperprameters. In every fold I build a new model and after the third fold I face this error:

ResourceExhaustedError: {{function_node >_wrapped__Fill_device/job:localhost/replica:0/task:0/device:GPU:0}} OOM when allocating tensor with shape[2700,18930] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:Fill]

To solve this error I need to restart the kernel but then I lose everything. How can I solve this issue or how can I remove the model that I built in the previous fold from memory?

Upvotes: 0

Views: 8

Answers (0)

Related Questions