CODEWITHSUNDEEP
pythonnlpnltk
chammu
chammu

Reputation: 1305

python NLTK POS tagger not behaving as expected

I ran pos_tag function on below text,it shows output with battery as 'RB'. As battery is noun, it should show as 'NN'.

nltk.pos_tag(nltk.word_tokenize('Camera picture quality was fair but speed was an issue and also battery life was not that good'))

Output:

[('Camera', 'NNP'), ('picture', 'NN'), ('quality', 'NN'), ('was', 'VBD'), ('fair', 'JJ'), ('but', 'CC'), ('speed', 'NN'), ('was', 'VBD'), ('an', 'DT'), ('issue', 'NN'), ('and', 'CC'), ('also', 'RB'), ('battery', 'RB'), ('life', 'NN'), ('was', 'VBD'), ('not', 'RB'), ('that', 'IN'), ('good', 'JJ')]

While if I POS tagged the same statement by this tagger http://cst.dk/online/pos_tagger/uk/ , it shows battery as 'NN' and gives following output:

Camera/NNP picture/NN quality/NN was/VBD fair/JJ but/CC speed/NN was/VBD an/DT issue/NN and/CC also/RB battery/NN life/NN was/VBD not/RB that/IN good/JJ

Edit:

With statement as :

"Camera picture quality was fair but speed was an issue but battery life was not that good"

the NLTK tagger gives following output:

[('Camera', 'NNP'), ('picture', 'NN'), ('quality', 'NN'), ('was', 'VBD'), ('fair', 'JJ'), ('but', 'CC'), ('speed', 'NN'), ('was', 'VBD'), ('an', 'DT'), ('issue', 'NN'), ('but', 'CC'), ('battery', 'NN'), ('life', 'NN'), ('was', 'VBD'), ('not', 'RB'), ('that', 'IN'), ('good', 'JJ')]

Please explain!

Upvotes: 0

Views: 1239

Answers (1)

alvas
alvas

Reputation: 121992

It seems like the only difference is that cst.dk tagged battery as NN and NLTK tagged battery as RB (adverb).

>>> cstdk_output = "Camera/NNP picture/NN quality/NN was/VBD fair/JJ but/CC speed/NN was/VBD an/DT issue/NN and/CC also/RB battery/NN life/NN was/VBD not/RB that/IN good/JJ"
>>> cstdk_postags = [tuple(j for j in i.split('/')) for i in cstdk_output.split()]
>>> from nltk import pos_tag
>>> sent = [i for i,j in cstdk_postags]
>>> nltk_postags = pos_tag(sent)
>>> diff = [(i[0],i[1],j[1]) for i,j in zip(cstdk_postags, nltk_postags) if i[1] != j[1]]
>>> diff
[('battery', 'NN', 'RB')]

There is not much to explain. It's a statistical trained system using Maximum Entropy, see _POS_TAGGER in http://www.nltk.org/_modules/nltk/tag.html#pos_tag, so it is bound to make mistake. See other mistakes it makes, POS tagging - NLTK thinks noun is adjective

Upvotes: 1

Related Questions