CODEWITHSUNDEEP
pythontensorflowkerashuggingface-transformers
Minions
Minions

Reputation: 5467

AttributeError: Layer tf_bert_model has no inbound nodes

I have a deep learning model that uses BERT layer from Huggingface library (TF=2.0, Transformers=2.8.0).

The model consists of: BERT embeddings -> Attention -> Softmax. Also I am using tf.keras.callbacks.ModelCheckpoint to save the best model during training.

I'm trying to slice the model to get the attention weights.

My problem is that, if I try to access the output of any layer after loading the saved model using output1 = model.layers[3].output, I get this error:

AttributeError: Layer tf_bert_model has no inbound nodes.

but if I do the same on the original model without saving it (instantly after finishing model.fit()), I have no problem.

This is how I load the saved model:

model = tf.keras.models.load_model(model_dir)

Is there a problem with this way?, giving that it works for prediction.

Upvotes: 1

Views: 302

Answers (1)

user11530462
user11530462

Reputation:

Answering here for the benefit of the community since the user has already found the solution.

Issue was resolved by following steps

  1. Loading the saved model
  2. Creating another new model with the same structure
  3. Copying the weights from the saved model to the newly created one
  4. Then access the needed layer without any problem

Upvotes: 1

Related Questions