CODEWITHSUNDEEP
pythontensorflowkerassaveresnet
Jeanne Chaverot
Jeanne Chaverot

Reputation: 157

ValueError: Unable to create dataset (name already exists) while saving tensorflow model

I am trying to save the trained model below.

resnet = ResNet50V2(input_shape=(im_size,im_size,3), weights='imagenet', include_top=False)
headModel = AvgPool2D(pool_size=(3,3))(resnet.output)
headModel = Flatten(name="flatten")(headModel)
headModel = Dense(256, activation="relu")(headModel)
headModel = Dropout(0.5)(headModel)
headModel = Dense(1, activation="sigmoid")(headModel)
resnet50v2 = Model(inputs=resnet.input, outputs=headModel)


resnet50v2.compile(loss='binary_crossentropy', optimizer=opt, metrics=METRICS)

history = resnet50v2.fit(
        datagen.flow(X_train, y_train, batch_size=32, subset='training'),
        batch_size=batch_size,
        epochs=150,
        steps_per_epoch=steps_per_epoch,    
        validation_data=datagen.flow(X_train, y_train, batch_size=8, subset='validation'))

However, whenever I try to save it with the following command:

resnet50v2.save('Saved_Models/resnet50.h5', save_format='h5')

I get the error


ValueError                                Traceback (most recent call last)
/tmp/ipykernel_3252071/2034094124.py in <module>
----> 1 resnet50v2.save('Saved_Models/resnet50.h5', save_format='h5')

~/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb

~/.local/lib/python3.8/site-packages/h5py/_hl/group.py in create_dataset(self, name, shape, dtype, data, **kwds)
    147                     group = self.require_group(parent_path)
    148 
--> 149             dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
    150             dset = dataset.Dataset(dsid)
    151             return dset

~/.local/lib/python3.8/site-packages/h5py/_hl/dataset.py in make_new_dset(parent, shape, dtype, data, name, chunks, compression, shuffle, fletcher32, maxshape, compression_opts, fillvalue, scaleoffset, track_times, external, track_order, dcpl, allow_unknown_filter)
    140 
    141 
--> 142     dset_id = h5d.create(parent.id, name, tid, sid, dcpl=dcpl)
    143 
    144     if (data is not None) and (not isinstance(data, Empty)):

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5d.pyx in h5py.h5d.create()

ValueError: Unable to create dataset (name already exists)

How can I save my models?

Upvotes: 6

Views: 9126

Answers (1)

sudar
sudar

Reputation: 151

This error occurs when the layer names are duplicates across the namespaces of the pre-trained model and your downstream-task network. It may be useful if you choose to call every layer of your downstream task network with unique names.

resnet = tf.keras.applications.ResNet50V2(input_shape=(224, 224, 3), weights='imagenet', include_top=False)
headModel = tf.keras.layers.AvgPool2D(pool_size=(3,3), name='my_avg_2d_pool')(resnet.output)
headModel = tf.keras.layers.Flatten(name="my_flatten")(headModel)
headModel = tf.keras.layers.Dense(256, activation="relu", name='my_hidden_1')(headModel)
headModel = tf.keras.layers.Dropout(0.5)(headModel)
headModel = tf.keras.layers.Dense(1, activation="sigmoid", name='my_output_1')(headModel)
resnet50v2 = tf.keras.Model(inputs=resnet.input, outputs=headModel)

Upvotes: 1

Related Questions