CODEWITHSUNDEEP
pythonpython-3.xnltk
Mike
Mike

Reputation: 73

sentiment_analyser error: 'bytes' object has no attribute 'encode' using

Working on a project to sentiment analyse stocks using nltk. I've searched through GH and found nothing similar for sentimaent_analyser or popularity_scores calls.

I also looked at Python 3.4 - 'bytes' object has no attribute 'encode' and it is not a duplicate as I'm not calling bcrypt.gensalt().encode('utf-8'). Though it does hint a the issue of something being the wrong type.

Can anyone help in resolving this error?

I get the error:

/lib/python3.5/site-packages/nltk/sentiment/vader.py in init(self, text) 154 def init(self, text): 155 if not isinstance(text, str): --> 156 text = str(text.encode('utf-8')) 157 self.text = text 158 self.words_and_emoticons = self._words_and_emoticons()

AttributeError: 'bytes' object has no attribute 'encode'

The dataframe df_stocks.head(5) is :

            prices  articles
2007-01-01  12469   What Sticks from '06. Somalia Orders Islamist...
2007-01-02  12472   Heart Health: Vitamin Does Not Prevent Death ...
2007-01-03  12474   Google Answer to Filling Jobs Is an Algorithm...
2007-01-04  12480   Helping Make the Shift From Combat to Commerc...
2007-01-05  12398   Rise in Ethanol Raises Concerns About Corn as...

The code is below with the error occuring on the last line:

import numpy as np
import pandas as pd
from nltk.classify import NaiveBayesClassifier
from nltk.corpus import subjectivity
from nltk.sentiment import SentimentAnalyzer
from nltk.sentiment.util import *from nltk.sentiment.vader import     SentimentIntensityAnalyzer
import unicodedata
for date, row in df_stocks.T.iteritems():
    sentence = unicodedata.normalize('NFKD', df_stocks.loc[date, 'articles']).encode('ascii','ignore')
    ss = sid.polarity_scores(sentence)

Thanks

Upvotes: 1

Views: 2726

Answers (1)

aircraft
aircraft

Reputation: 26896

From the unicodedata.normalize() docs, the method is convert a UNICODE string into a common format string.

import unicodedata

print(unicodedata.normalize('NFKD', u'abcdあäasc').encode('ascii', 'ignore'))

It will get :

b'abcdaasc'

So, the issue is here: df_stocks.loc[date, 'articles'] is not a UNICODE string.

Upvotes: 1

Related Questions