CODEWITHSUNDEEP
pythonnlpmodelhuggingface-transformersbert-language-model
youta
youta

Reputation: 323

'BertModel' object has no attribute 'bert' error german bert model

i want to replicate the work in this repo https://github.com/theartificialguy/NLP-with-Deep-Learning/blob/master/BERT/Multi-Class%20classification%20TF-BERT/multi_class.ipynb but with german texts and using other labels i prepared my data and went through the steps doing the necessary modifications instead of tokenizer = BertTokenizer.from_pretrained('bert-base-cased') i used tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-german-cased") from this documentation https://huggingface.co/dbmdz/bert-base-german-cased and instead of model = TFBertModel.from_pretrained('bert-base-cased') i used model = AutoModel.from_pretrained("dbmdz/bert-base-german-cased") then when i came to the point where i had to execute this part

# defining 2 input layers for input_ids and attn_masks
input_ids = tf.keras.layers.Input(shape=(256,), name='input_ids', dtype='int32')
attn_masks = tf.keras.layers.Input(shape=(256,), name='attention_mask', dtype='int32')

bert_embds = model.bert(input_ids, attention_mask=attn_masks)[1] # 0 -> activation layer (3D), 
1 -> pooled output layer (2D)
intermediate_layer = tf.keras.layers.Dense(512, activation='relu', name='intermediate_layer') 
(bert_embds)
output_layer = tf.keras.layers.Dense(5, activation='softmax', name='output_layer') 
(intermediate_layer) # softmax -> calcs probs of classes

sentiment_model = tf.keras.Model(inputs=[input_ids, attn_masks], outputs=output_layer)
sentiment_model.summary()

i had this error

AttributeError                            Traceback (most recent call last)
<ipython-input-42-ed437bbb2d3e> in <module>
  3 attn_masks = tf.keras.layers.Input(shape=(256,), name='attention_mask', dtype='int32')
  4 
 ----> 5 bert_embds = model.bert(input_ids, attention_mask=attn_masks)[1] # 0 -> activation 
 layer (3D), 1 -> pooled output layer (2D)
  6 intermediate_layer = tf.keras.layers.Dense(512, activation='relu', 
 name='intermediate_layer')(bert_embds)
  7 output_layer = tf.keras.layers.Dense(5, activation='softmax', name='output_layer') 
  (intermediate_layer) # softmax -> calcs probs of classes

   /usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py in __getattr__(self, name)
  1263             if name in modules:
  1264                 return modules[name]
  -> 1265         raise AttributeError("'{}' object has no attribute '{}'".format(
  1266             type(self).__name__, name))
  1267 

  AttributeError: 'BertModel' object has no attribute 'bert'

Upvotes: 1

Views: 1343

Answers (1)

Suneburg
Suneburg

Reputation: 41

In bert-base-german-cased, we can see: Currently only PyTorch-Transformers compatible weights are available. If you need access to TensorFlow checkpoints, please raise an issue!

Thus, you either raise an issue, or write a pytorch version.

Upvotes: 2

Related Questions