CODEWITHSUNDEEP
pythonimporterrorbert-language-modelhuggingface-transformers
Icaru5
Icaru5

Reputation: 93

No module named 'transformers.models' while trying to import BertTokenizer

I am trying to import BertTokenizer from the transformers library as follows:

import transformers
from transformers import BertTokenizer
from transformers.modeling_bert import BertModel, BertForMaskedLM

However, I get the following error:

enter image description here

enter image description here

I am using transformers version 3.5.1 because I had a problem with the updated version which can be found here.

Does anyone know how to fix this? Apart from updating the transformers library to its latest version (that will unfortunately cause more errors).

Any help is appreciated!

Upvotes: 5

Views: 43140

Answers (2)

Brian Formento
Brian Formento

Reputation: 781

How you call the package depends on how you installed the transformers package.

If you've installed it from pip you need to find the directory where the package is installed, for example, if installing with pip with conda as a virtual environment this is where the BertModel package will be stored:

/home/{USER_NAME}/anaconda3/envs/{CONDA_ENV_NAME}/lib/{PYTHON_VERSION}/site-packages/transformers/modeling_bert.py

while the BertTokenizer is automatically called in the init.py, hence can be directly called.

Therefore you should be able to call

from transformers.modeling_bert import BertModel, BertForMaskedLM from transformers import BertTokenizer

otherwise, if you've installed it from source, you need to map to the correct file. For Example, with the following file directory structure:

Code_folder
     transformers #package
         models
             bert
                 modeling_bert.py
     main.py # your file that wants to call the transformer package

Then you can call the following packages in the following way:

from transformers.models.bert.modeling_bert import BertModel,BertForMaskedLM
from transformers import BertTokenizer

Upvotes: 0

lvjiujin
lvjiujin

Reputation: 571

you can change your code from

transformers.modeling_bert import BertModel, BertForMaskedLM

to

from transformers.models.bert.modeling_bert import BertModel,BertForMaskedLM

Upvotes: 8

Related Questions