CODEWITHSUNDEEP
nlpspacy
Mona Jalal
Mona Jalal

Reputation: 38155

SpaCy OSError: Can't find model 'en'

even though I downloaded the model it cannot load it

[jalal@goku entity-sentiment-analysis]$ which python
/scratch/sjn/anaconda/bin/python
[jalal@goku entity-sentiment-analysis]$ sudo python -m spacy download en
[sudo] password for jalal: 
Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz (37.4MB)
    100% |████████████████████████████████| 37.4MB 9.4MB/s 
Installing collected packages: en-core-web-sm
  Running setup.py install for en-core-web-sm ... done
Successfully installed en-core-web-sm-2.0.0

    Linking successful
    /usr/lib/python2.7/site-packages/en_core_web_sm -->
    /usr/lib64/python2.7/site-packages/spacy/data/en

    You can now load the model via spacy.load('en')

import spacy 

nlp = spacy.load('en')
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-2-0fcabaab8c3d> in <module>()
      1 import spacy
      2 
----> 3 nlp = spacy.load('en')

/scratch/sjn/anaconda/lib/python3.6/site-packages/spacy/__init__.py in load(name, **overrides)
     17             "to load. For example:\nnlp = spacy.load('{}')".format(depr_path),
     18             'error')
---> 19     return util.load_model(name, **overrides)
     20 
     21 

/scratch/sjn/anaconda/lib/python3.6/site-packages/spacy/util.py in load_model(name, **overrides)
    118     elif hasattr(name, 'exists'):  # Path or Path-like to model data
    119         return load_model_from_path(name, **overrides)
--> 120     raise IOError("Can't find model '%s'" % name)
    121 
    122 

OSError: Can't find model 'en'

How should I fix this?

If I don't use sudo for downloading the en model, I get:

Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz (37.4MB)
    100% |████████████████████████████████| 37.4MB 9.6MB/s ta 0:00:011   62% |████████████████████            | 23.3MB 8.6MB/s eta 0:00:02
Requirement already satisfied (use --upgrade to upgrade): en-core-web-sm==2.0.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz in /scratch/sjn/anaconda/lib/python3.6/site-packages
You are using pip version 10.0.0, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

    Error: Couldn't link model to 'en'
    Creating a symlink in spacy/data failed. Make sure you have the required
    permissions and try re-running the command as admin, or use a
    virtualenv. You can still import the model as a module and call its
    load() method, or create the symlink manually.

    /scratch/sjn/anaconda/lib/python3.6/site-packages/en_core_web_sm -->
    /scratch/sjn/anaconda/lib/python3.6/site-packages/spacy/data/en


    Download successful but linking failed
    Creating a shortcut link for 'en' didn't work (maybe you don't have
    admin permissions?), but you can still load the model via its full
    package name:

    nlp = spacy.load('en_core_web_sm')

Upvotes: 64

Views: 132113

Answers (14)

lellama
lellama

Reputation: 1

A quick hack to fix: Install an available model (e.g. en_core_web_sm) and then make the symlink yourself. Copy the two paths spacy says that it can't link (probably due to the virtual environment running without admin elevation) and use e.g. mklink on Windows.

E.g. mklink /D C:\Users\USER\PROJECT\venv2\lib\site-packages\spacy\data\en C:\Users\USER\PROJECT\venv2\lib\site-packages\en_core_web_sm

Upvotes: 0

Vetrivel PS
Vetrivel PS

Reputation: 959

FINALLY CLEARED THE ERROR !!!

Best Way to Install now

pip install -U pip setuptools wheel

pip install -U spacy

python -m spacy download en_core_web_sm

Always Open Anaconda Prompt / Command Prompt with Admin Rights to avoid Linking errors!!!

  • Tried multiple options including :

    python -m spacy download en

    conda install -c conda-forge spacy

    python -m spacy download en_core_web_sm

    python -m spacy link en_core_web_sm en

  • None worked since im using my Company's Network. Finally this Command Worked like a Charm :-)

    pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz --no-deps

    • Updated with Latest Link :

    pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz --no-deps

Thanks to the Updated Github Links :-)

Upvotes: 71

Rohan Devaki
Rohan Devaki

Reputation: 3279

  • you need to download en_core_web_sm
  • if you are using anaconda , then run this command
  • conda install -c conda-forge spacy-model-en_core_web_sm
  • and load it as
  • nlp= spacy.load('en_core_web_sm')

Upvotes: 3

Anubhab Panda
Anubhab Panda

Reputation: 1

If you have already downloaded spacy and the language model (E.g., en_core_web_sm or en_core_web_md), then you can follow these steps:

  1. Open Anaconda prompt as admin

  2. Then type : python -m spacy link [package name or path] [shortcut]

    For E.g., python -m spacy link /Users/you/model en

This will create a symlink to the your language model. Now you can load the model using spacy.load("en") in your notebooks or scripts

Upvotes: 0

ListenSoftware Louise Ai Agent
ListenSoftware Louise Ai Agent

Reputation: 4233

  pip install https://github.com/explosion/spacy- 
  models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz

  #fixes an permission error when attempting to create the symlinks on windows 10

  python -m spacy link en_core_web_sm en_core_web_smc

  from Jupyter notes
  import spacy

  nlp = spacy.load('en_core_web_sm')

Upvotes: 2

Himanshu Aggarwal
Himanshu Aggarwal

Reputation: 179

Since you are using python version 3.6, try using -

python3 -m spacy download en

instead of just python -m .....

Upvotes: 0

neelneelpurk
neelneelpurk

Reputation: 81

As you are using Anaconda, open Anaconda Prompt as an Admin and execute the following command

python -m spacy download en

To load Spacy 'en' in Jupyter Notebook use the following command

spacy.load('en')

Upvotes: 3

Palak Jain
Palak Jain

Reputation: 683

Go to https://github.com/explosion/spacy-models

Download the model you want to load in SpaCy

Paste the downloaded file in SpaCy folder present inside the Anaconda folder

open cmd there. Type the following command and hit enter:

pip install en_core_web_md-1.2.0.tar.gz

The above command may vary depending upon the version of the file downloaded.

Voila! Error has gone :)

Upvotes: 2

Thientvse
Thientvse

Reputation: 1791

If you use other python version, you can run :

sudo python3.6 -m spacy download en

With me, my version 3.6 I hope it can help your problem!

Upvotes: 3

dineshydv
dineshydv

Reputation: 107

I am using anaconda jupyter notebook and was getting same error. Ran below commands in anaconda prompt (run as administrator) and it resolved my issue:

(base) C:\WINDOWS\system32>conda install -c conda-forge spacy
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Users\yadav\Anaconda3

  added / updated specs:
    - spacy


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cymem-2.0.3                |   py37h6538335_0          35 KB  conda-forge
    cython-blis-0.4.1          |   py37hfa6e2cd_0         4.3 MB  conda-forge
    murmurhash-1.0.0           |   py37h6538335_0          17 KB  conda-forge
    plac-0.9.6                 |             py_1          18 KB  conda-forge
    preshed-3.0.2              |   py37h6538335_1          89 KB  conda-forge
    spacy-2.2.1                |   py37he980bc4_0         7.4 MB  conda-forge
    srsly-0.2.0                |   py37h6538335_0         189 KB  conda-forge
    thinc-7.1.1                |   py37he980bc4_0         1.4 MB  conda-forge
    wasabi-0.4.0               |             py_0          19 KB  conda-forge
    ------------------------------------------------------------
                                           Total:        13.4 MB

The following NEW packages will be INSTALLED:

  cymem              conda-forge/win-64::cymem-2.0.3-py37h6538335_0
  cython-blis        conda-forge/win-64::cython-blis-0.4.1-py37hfa6e2cd_0
  murmurhash         conda-forge/win-64::murmurhash-1.0.0-py37h6538335_0
  plac               conda-forge/noarch::plac-0.9.6-py_1
  preshed            conda-forge/win-64::preshed-3.0.2-py37h6538335_1
  spacy              conda-forge/win-64::spacy-2.2.1-py37he980bc4_0
  srsly              conda-forge/win-64::srsly-0.2.0-py37h6538335_0
  thinc              conda-forge/win-64::thinc-7.1.1-py37he980bc4_0
  wasabi             conda-forge/noarch::wasabi-0.4.0-py_0


Proceed ([y]/n)? Y


Downloading and Extracting Packages
cython-blis-0.4.1    | 4.3 MB    | ############################################################################ | 100%
cymem-2.0.3          | 35 KB     | ############################################################################ | 100%
srsly-0.2.0          | 189 KB    | ############################################################################ | 100%
thinc-7.1.1          | 1.4 MB    | ############################################################################ | 100%
plac-0.9.6           | 18 KB     | ############################################################################ | 100%
spacy-2.2.1          | 7.4 MB    | ############################################################################ | 100%
preshed-3.0.2        | 89 KB     | ############################################################################ | 100%
wasabi-0.4.0         | 19 KB     | ############################################################################ | 100%
murmurhash-1.0.0     | 17 KB     | ############################################################################ | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done


(base) C:\WINDOWS\system32>python -m spacy download en
Collecting en_core_web_sm==2.2.0
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz (12.0MB)
     |████████████████████████████████| 12.0MB 409kB/s
Requirement already satisfied: spacy>=2.2.0 in c:\users\yadav\anaconda3\lib\site-packages (from en_core_web_sm==2.2.0) (2.2.2)
Requirement already satisfied: numpy>=1.15.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (1.16.2)
Requirement already satisfied: thinc<7.4.0,>=7.3.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (7.3.1)
Requirement already satisfied: wasabi<1.1.0,>=0.3.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (0.4.0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (2.21.0)
Requirement already satisfied: setuptools in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (40.8.0)
Requirement already satisfied: plac<1.2.0,>=0.9.6 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (1.1.3)
Requirement already satisfied: srsly<1.1.0,>=0.1.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (0.2.0)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (2.0.3)
Requirement already satisfied: importlib-metadata>=0.20; python_version < "3.8" in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (0.23)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (1.0.2)
Requirement already satisfied: blis<0.5.0,>=0.4.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (0.4.1)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (3.0.2)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in c:\users\yadav\anaconda3\lib\site-packages (from thinc<7.4.0,>=7.3.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (4.36.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\yadav\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (2019.3.9)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in c:\users\yadav\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (1.24.1)
Requirement already satisfied: idna<2.9,>=2.5 in c:\users\yadav\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\users\yadav\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (3.0.4)
Requirement already satisfied: zipp>=0.5 in c:\users\yadav\anaconda3\lib\site-packages (from importlib-metadata>=0.20; python_version < "3.8"->spacy>=2.2.0->en_core_web_sm==2.2.0) (0.6.0)
Requirement already satisfied: more-itertools in c:\users\yadav\anaconda3\lib\site-packages (from zipp>=0.5->importlib-metadata>=0.20; python_version < "3.8"->spacy>=2.2.0->en_core_web_sm==2.2.0) (6.0.0)
Building wheels for collected packages: en-core-web-sm
  Building wheel for en-core-web-sm (setup.py) ... done
  Created wheel for en-core-web-sm: filename=en_core_web_sm-2.2.0-cp37-none-any.whl size=12019131 sha256=f716e80f029462a80e9fb79ef353c1ac8c0f81d3754778bb6fec520d640fcc87
  Stored in directory: C:\Users\yadav\AppData\Local\Temp\pip-ephem-wheel-cache-bvy0x0eg\wheels\48\5c\1c\15f9d02afc8221a668d2172446dd8467b20cdb9aef80a172a4
Successfully built en-core-web-sm
Installing collected packages: en-core-web-sm
  Found existing installation: en-core-web-sm 2.0.0
    Uninstalling en-core-web-sm-2.0.0:
      Successfully uninstalled en-core-web-sm-2.0.0
Successfully installed en-core-web-sm-2.2.0
✔ Download and installation successful
You can now load the model via spacy.load('en_core_web_sm')
symbolic link created for C:\Users\yadav\Anaconda3\lib\site-packages\spacy\data\en <<===>> C:\Users\yadav\Anaconda3\lib\site-packages\en_core_web_sm
✔ Linking successful
C:\Users\yadav\Anaconda3\lib\site-packages\en_core_web_sm -->
C:\Users\yadav\Anaconda3\lib\site-packages\spacy\data\en
You can now load the model via spacy.load('en')

(base) C:\WINDOWS\system32>

Then in jupyter notebook load it like below:

nlp = spacy.load('en',parse=True,tag=True, entity=True)

Upvotes: 3

Daweed24321
Daweed24321

Reputation: 29

Bc i didnt find my error here (For everyone who uses jupyter Notebook, Alteryx, Company Network and had this error):

i tried to create a macro with python for topic detection but got the Error that there is not a module named "en_core_web_sm"

Install following Packages at the beginning with following code:

from ayx import Package Package.installPackages(['pandas','numpy', 'matplotlib', 'gensim', 'spacy', 'pyLDAvis', 'https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.1.0/en_core_web_sm-2.1.0.tar.gz'])

And if you are referencing to the module use:

import en_core_web_sm

nlp = en_core_web_sm.load()

worked for me perf fine :))

Upvotes: 2

Raghu
Raghu

Reputation: 477

1) Install Spacy

$ python -m spacy download en

2) Install the model en_core_web_sm

$ python -m spacy download en_core_web_sm
>>> import spacy
>>> nlp = spacy.load("en_core_web_sm")

Upvotes: 11

Mona Jalal
Mona Jalal

Reputation: 38155

oh well. turns out even though my which python was showing anaconda python, when I was using python download it was linking it to python2.7 local on my machine. I fixed it using below command:

$ sudo /scratch/sjn/anaconda/bin/python -m spacy download en

Upvotes: 9

Johannes Gontrum
Johannes Gontrum

Reputation: 1161

By using sudo python ... you install the model for a different python interpreter than your local one. In fact, it says in your log that the spaCy model is installed to /usr/lib64/python2.7/site-packages/ instead of /scratch/sjn/anaconda/lib/python3.6/site-packages/.

Try running python -m spacy download en and it should install the model to the correct directory.

Upvotes: 51

Related Questions