CODEWITHSUNDEEP
pythonnltkwordnet
Ron Somerville
Ron Somerville

Reputation: 31

NLTK WordNet error with a word look up using synsets

I am using Python 3.6 with NLTK 3.2.3, and I am getting a "WordNetError" only for the word "escort". I don't get errors with any other words. Here's the transcript showing success with the word "dog" and the error using the word "escort":

Python 3.6.0 |Anaconda custom (64-bit)| (default, Dec 23 2016, 12:22:00) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from nltk.corpus import wordnet
>>> wordnet.synsets('dog')
[Synset('dog.n.01'), Synset('frump.n.01'), Synset('dog.n.03'), Synset('cad.n.01'), Synset('frank.n.02'), Synset('pawl.n.01'), Synset('andiron.n.01'), Synset('chase.v.01')]
>>> wordnet.synsets('escort')
Traceback (most recent call last):
  File "/home/user1/.conda/envs/ca/lib/python3.6/site-packages/nltk/corpus/reader/wordnet.py", line 1403, in _synset_from_pos_and_line
    offset = int(_next_token())
ValueError: invalid literal for int() with base 10: '02026433\x00v'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user1/.conda/envs/ca/lib/python3.6/site-packages/nltk/corpus/reader/wordnet.py", line 1491, in synsets
    for p in pos
  File "/home/user1/.conda/envs/ca/lib/python3.6/site-packages/nltk/corpus/reader/wordnet.py", line 1493, in <listcomp>
    for offset in index[form].get(p, [])]
  File "/home/user1/.conda/envs/ca/lib/python3.6/site-packages/nltk/corpus/reader/wordnet.py", line 1335, in synset_from_pos_and_offset
    synset = self._synset_from_pos_and_line(pos, data_file_line)
  File "/home/user1/.conda/envs/ca/lib/python3.6/site-packages/nltk/corpus/reader/wordnet.py", line 1448, in _synset_from_pos_and_line
    raise WordNetError('line %r: %s' % (data_file_line, e))
nltk.corpus.reader.wordnet.WordNetError: line '02025829 38 v 01 escort 0 006 @ 02025550 v 0000 + 09992538 n 0102 ~ 02026203 v 0000 ~ 02026327 v 0000 ~ 02026433\x00v 0000 ~ 02026712 v 0000 04 + 08 00 + 09 00 + 20 00 + 21 00 | accompany as an escort; "She asked her older brother to escort her to the ball"  \n': invalid literal for int() with base 10: '02026433\x00v'

However, when I use the online WordNet search tool at http://wordnetweb.princeton.edu/perl/webwn, it performs the lookup as expected. The latest WordNet corpus was downloaded using nltk.download().

The error seems to reference a hex value in the WordNet definition for the word when it is expecting to find an integer value.

Any ideas? Please advise if you've run into something like this.

Upvotes: 1

Views: 1145

Answers (1)

Ron Somerville
Ron Somerville

Reputation: 31

After closer inspection of the "verb" dictionary file, I discovered it was in fact corrupted. Here's what it looks like in an editor like "vi":

02025829 38 v 01 escort 0 006 @ 02025550 v 0000 + 09992538 n 0102 ~ 02026203 v 0000 ~ 02026327 v 0000 ~ 02026433^@v 0000 ~ 02026712 v 0000 04 +

I replaced the "^@" with a space, and the problem went away. I guess the bigger question is how it got corrupted in the first place.

Problem solved!

Upvotes: 1

Related Questions