CODEWITHSUNDEEP
apache-sparkpysparkword2vec
verkter
verkter

Reputation: 778

PySpark - Word2Vec load model, can't use findSynonyms to get words

I have trained a Word2Vec model with PySpark and saved it. When loading the model .findSynonyms method does not work. 

model = word2vec.fit(text)
model.save(sc, 'w2v_model')
new_model = Word2VecModel.load(sc, 'w2v_model')
new_model.findSynonyms('word', 4)

Getting the following error:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/spark/python/pyspark/mllib/feature.py", line 487, in findSynonyms
words, similarity = self.call("findSynonyms", word, num)
ValueError: too many values to unpack

I found the following, but not sure how the issue was fixed: https://issues.apache.org/jira/browse/SPARK-12016

Please let me know if there are any work arounds!

Many thanks.

Upvotes: 2

Views: 2290

Answers (1)

JUNPA
JUNPA

Reputation: 240

Looks like it's fixed on 1.6.1 but not on 1.5.2.

The error is not about findSynonyms but about Word2VecModel.load. I checked it works on 1.6.1.; no error while loading the model and calling findSynonyms method.

I guess v. 1.5.2 is not fixed yet.

Upvotes: 1

Related Questions