CODEWITHSUNDEEP
pythontensorflowmultilingualtext-classificationbert-language-model
Sergio
Sergio

Reputation: 109

Using BETO - Spanish version BERT

I'm trying to build a Spanish text classifier based on BERT. So I selected a model called BETO https://github.com/scruz03/beto. I am working on Google Colab and trying to replicate the steps indicated in the example cited at the same page referred before https://colab.research.google.com/drive/1uRwg4UmPgYIqGYY4gW_Nsw9782GFJbPt#scrollTo=HhAqZLs3lwhW. I did download the uncased tensorflow version instead of the pytorch one mentioned in the example. I got the following files in the tensorflow subdirectory:

enter image description here

However, when I ran the following code:

# create the tokenizer and the model
tokenizer = BertTokenizer.from_pretrained("tensorflow/")
model = BertForMaskedLM.from_pretrained("tensorflow/")
model.eval()

I got the following error:

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-37-4b5a17f4238d> in <module>()
      1 # create the tokenizer and the model
      2 tokenizer = BertTokenizer.from_pretrained("tensorflow/")
----> 3 model = BertForMaskedLM.from_pretrained("tensorflow/")
      4 model.eval()

/usr/local/lib/python3.6/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    982                         "Error no file named {} found in directory {} or `from_tf` set to False".format(
    983                             [WEIGHTS_NAME, TF2_WEIGHTS_NAME, TF_WEIGHTS_NAME + ".index"],
--> 984                             pretrained_model_name_or_path,
    985                         )
    986                     )

OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index'] found in directory tensorflow/ or `from_tf` set to False

I would appreciate your help. Thanks in advance.

Upvotes: 1

Views: 2310

Answers (1)

Ashwin Geet D&#39;Sa
Ashwin Geet D&#39;Sa

Reputation: 7379

The reason why you are getting an error is that you are trying to load a tensorflow model to a class that expects pytorch model.

Huggingface API provies a tensorflow equivalent class.

To use it, instead of BertForMaskedLM use TFBertForMaskedLM

Upvotes: 2

Related Questions