CODEWITHSUNDEEP
tensorflowoptimizationneural-networkkerasprediction
Denym
Denym

Reputation: 211

Keras taking very long time to make first prediction following model.load()

I am loading a model in keras with model.load() and am finding that the first prediction is taking more than 10x longer to calculate than follow on predictions, any ideas why this could be occurring or suggestions to make the load-initialise-first prediction cycle speed up would be greatly appreciated.

I am using Tensorflow backend with CPU processing.

Thanks for the help, Denym

Upvotes: 11

Views: 6566

Answers (1)

Denym
Denym

Reputation: 211

Ok so I have found the answer that works for me:

if you are loading many models simultaneously don't use the keras model.load function, save your structure as a json/yaml and the weights as a .h5 and load as per the keras examples.

the model.load function is much quicker when dealing with less than 5 models however load times exponentially increase the more models you simultaneously load.

loading from json and weights from .h5 was 10x faster when loading 100 models simultaneously, and while there is some slow down per model when loading structure and weights method it is linear rather than exponential, making it significantly faster when loading many models at once.

Upvotes: 8

Related Questions