CODEWITHSUNDEEP
pythonword2vec
Ganchimeg L
Ganchimeg L

Reputation: 83

word2vec getting encode error

I am getting the following error when i execute my code

Traceback (most recent call last):
  File "test.py", line 21, in <module>
    print model.most_similar(positive=['男人'])
  File "/usr/local/lib/python2.7/dist-packages/gensim/models/word2vec.py", line 660, in most_similar
    raise KeyError("word '%s' not in vocabulary" % word)
KeyError: "word '\xe7\x94\xb7\xe4\xba\xba' not in vocabulary"

My code here

 # -*- coding: utf8 -*    
    from gensim.models import word2vec
    import logging

logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)
        sentences = word2vec.Text8Corpus('/tmp/text8')
        model = word2vec.
    Word2Vec(sentences, size=200)
        model.most_similar(['男人'])

Upvotes: 1

Views: 1251

Answers (1)

KameeCoding
KameeCoding

Reputation: 723

"it works by the following changes. model.most_similar([u'男人'])"

that means you are working with probably utf-8 encoded strings rather than unicode strings, a good practice is to decode on input work with unicode then encode on output.

.decode('utf-8') your string

Upvotes: 1

Related Questions