CODEWITHSUNDEEP
nltkspacynamed-entity-recognition
Meenakshi
Meenakshi

Reputation: 13

How to train own model and test it with spacy

I am using the below code to train an already existing spacy ner model. However, I dont get correct results on tests:

What I am missing?

import spacy
import random
from spacy.gold import GoldParse
from spacy.language import EntityRecognizer

train_data = [
    ('Who is Rocky babu?', [(7, 16, 'PERSON')]),
    ('I like London and Berlin.', [(7, 13, 'LOC'), (18, 24, 'LOC')])
]

nlp = spacy.load('en', entity=False, parser=False)
ner = EntityRecognizer(nlp.vocab, entity_types=['PERSON', 'LOC'])

for itn in range(5):
    random.shuffle(train_data)
    for raw_text, entity_offsets in train_data:
        doc = nlp.make_doc(raw_text)
        gold = GoldParse(doc, entities=entity_offsets)

        nlp.tagger(doc)
        nlp.entity.update([doc], [gold])
Now, When i try to test the above model by using the below code, I don't get the expected output.

text = ['Who is Rocky babu?']

for a in text:
        doc = nlp(a)
        print("Entities", [(ent.text, ent.label_) for ent in doc.ents])
My output is as follows:

Entities []
whereas my expected output is as follows:

Entities [('Rocky babu', 'PERSON')]
Can someone please tell me what I'm missing ?

Upvotes: 1

Views: 1552

Answers (1)

Sofie VL
Sofie VL

Reputation: 3096

Could you retry with

nlp = spacy.load('en_core_web_sm', entity=False, parser=False)

If that gives an error because you don't have that model installed, you can run 

python -m spacy download en_core_web_sm

on the commandline first.

And ofcourse keep in mind that for a proper training of the model, you'll need many more examples for the model to be able to generalize!

Upvotes: 1

Related Questions