CODEWITHSUNDEEP
pythondeep-learningkeras
MBK
MBK

Reputation: 33

Training and saving very large models in Keras

I'm building an LSTM model with attention in Keras for multi-label classification, but there are thousands of possible output labels, each with its own sigmoid prediction layer and unique attention MLP layer. Is it possible to train and save such a large model? I'm getting the following h5py RuntimeError: Unable to create attribute (Object header message is too large).

Upvotes: 2

Views: 1187

Answers (1)

NKSHELL
NKSHELL

Reputation: 1119

You may already know about HDF5's header limitation. Look here for more info.

So I ran to the same problem, and I solved it with a little trick. Change your layers' names to some small strings before saving them. I did that like this: 

for i, m in enumerate(model.layers):
    m.name = 'n' + str(i)

And it worked (don't let that 'n' confuse you. I just wanted my layer names to start with a character, instead of a number). Note that layer names should be unique, and str(i) solves that. If you need your layers' names after loading your model later, you can create a dictionary for them and save the dictionary in a text file. After loading your model, read the dictionary from the text file and use it to map current layer names to the original ones. For example, create the dictionary like this:

dic = {}
for i, m in enumerate(model.layers):
    dic['n' + str(i)] = m.name
    m.name = 'n' + str(i)

and use it later like this:

for m in model.layers:
    m.name = dic[m.name]

Upvotes: 1

Related Questions