CODEWITHSUNDEEP
deep-learningkeras
user8483278
user8483278

Reputation:

The last steps of each epochs take too long time

I'm using Keras. When I run model.fit_generator(...), it goes 1 step per about 1.5 second, but the last step takes a few minutes.

Epoch 1/50
30/31 [============================>.] - ETA: 0s - loss: 2.0676 - acc: 0.2010

Why?

Upvotes: 1

Views: 1124

Answers (2)

Sonali Dasgupta
Sonali Dasgupta

Reputation: 61

I faced this issue while training a CNN , and found that decreasing the image dimensions speeds up the training. The processing time is reduced due to reduced input dimension during both forward pass and backpropagation (while updating weights). If for example, you are using a CNN for image classification, image size of 64*64 would be processed much faster than of size 256*256, though obviously at the cost of losing out information due to lower resolution.

Upvotes: 0

Dr. Snoopy
Dr. Snoopy

Reputation: 56357

This happens because you are giving validation data to Keras, through a parameter in model.fit or model.fit_generator.

After each epoch, Keras takes the validation data and evaluates the model on this data, which implies one forward pass for each validation data point, which might take a lot of time and might seem that Keras is stuck, but it is necessary when training a model.

Upvotes: 4

Related Questions