CODEWITHSUNDEEP
pythonpython-2.7twitterunicodetweepy
Hana
Hana

Reputation: 157

Unicode decode error when retrieving Twitter data using Python

When retrieving Twitter data for a specific Arabic keyword as follows:

#imports
from tweepy import Stream
from tweepy import OAuthHandler 
from tweepy.streaming import StreamListener

#setting up the keys
consumer_key = '………….' 
consumer_secret = '…………….'
access_token = '…………..'
access_secret = '……...'

class TweetListener(StreamListener):
    # A listener handles tweets are the received from the stream.
    #This is a basic listener that just prints received tweets to standard output

    def on_data(self, data):
        print (data)
        return True

    def on_error(self, status):
        print (status)

    #printing all the tweets to the standard output
    auth = OAuthHandler(consumer_key, consumer_secret)
    auth.set_access_token(access_token, access_secret)

    stream = Stream(auth, TweetListener())
    stream.filter(track=['سوريا'])

I got this error message :

Traceback (most recent call last):
File "/Users/Mona/Desktop/twitter.py", line 29, in <module>
stream.filter(track=['سوريا'])
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site- packages/tweepy/streaming.py", line 303, in filter
encoded_track = [s.encode(encoding) for s in track]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd8 in position 0: ordinal not in range(128)

Please any Help!!

Upvotes: 3

Views: 2187

Answers (1)

YardGlassOfCode
YardGlassOfCode

Reputation: 366

I looked at the source code for tweepy and found the line in the source for Stream that seems to cause the problem. The line is from the filter method. When you call stream.filter(track=['سوريا']) in your code, Stream calls s.encode('utf-8') where s = 'سوريا' (looking at the source code for filter, you'll utf-8 is the default encoding). It is at this point the code throws an exception.

To fix this, we need to use a Unicode string. 

 t = u"سوريا"
 stream.filter(track=[t])

(I just put your string into the variable t for clarity).

Upvotes: 6

Related Questions