CODEWITHSUNDEEP
pythonspacynamed-entity-recognition
antie
antie

Reputation: 11

Named Entity Recognition using Python spaCy

I want to code a Named Entity Recognition system using Python spaCy package. However, I couldn't install my local language inside spaCy package. Is there anyone who can tell me how to install or otherwise use my local language?

I tried:

python -m spacy download xx_ent_wiki_sm

I appreciate the help!

Upvotes: 1

Views: 2060

Answers (1)

mabergerx
mabergerx

Reputation: 1213

spaCy supports a limited amount of languages with standalone models. If you language is one of:

Chinese, Danish, Dutch, English, French, German, Greek, Italian, Japanese, Lithuanian, Norwegian, Bokmål, Polish, Portuguese, Romanian or Spanish

Then you can load the model by first installing it through a similar command that you have posted, for example:

# Lithuanian language
python -m spacy download lt_core_news_sm

# Japanese language
python -m spacy download ja_core_news_sm

You would have to run this command in your command line (terminal). After the model is finished downloading and is linked, you can import it like this:

import spacy

# Loading the Japanese language model.
nlp = spacy.load("ja_core_news_sm")

spaCy also support a multi-language model that you can try to use if your language is not supported with it's own model. For that, you can do (looks like you already tried to install it in your provided command):

# In command line
python -m spacy download xx_ent_wiki_sm

# In Python
import spacy
nlp = spacy.load("xx_ent_wiki_sm")

However, do not expect state-of-the-art results from using the multi-language model as it is not specifically trained on a single language like the other models are.

Upvotes: 3

Related Questions