CODEWITHSUNDEEP
pythonencodingcomputer-sciencetweepy
yasin
yasin

Reputation: 141

How can I fix Encoding in tweepy?

I'm working on tweety module and I tried this code for taking tweet by written Turkish(in turkish language,some charecter doesn't support ascii such as ğ,ş,ö,ç,İ,ı) but I need clearly whole data.

from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
import time

ckey    = "****"
csecret  = "****"
atoken  = "****"
asecret = "****"


class listener(StreamListener):

    def on_data(self,data):
        tweet    = data.split(',"text":"')[1].split('","source')[0]
        print tweet
        saveThis = str(time.time()) + '::' + tweet
        saveData = open("archive.csv","a")
        saveData.write(saveThis)
        saveData.write("\n")
        saveData.close()
        return True

    def on_error(self,status):
        print status

auth = OAuthHandler(ckey,csecret)
auth.set_access_token(atoken,asecret)

twitterStream = Stream(auth,listener() )
twitterStream.filter(track = ["galatasaray"])

it gives me 

1445282560.38::RT @EndlesGALA: T\u00fcrkiyedir #Galatasaray \nKim daha b\u00fcy\u00fck tart\u0131smayal\u0131m isterseniz http:\/\/t.co\/0K6jLC0CHd
1445282563.02::RT @Gkhanutkan1907: Galatasaray'\u0131 Her Sene Yenmek Bizim \u0130\u00e7in Ba\u015far\u0131 De\u011fil Genlerimizde Olan Bir GELENEKT\u0130R!
1445282563.26::22:22 GALATASARAY
1445282564.84::RT @mthnzncrkrn: Karanl\u0131k Elbet Kavu\u015fur Ayd\u0131nl\u0131\u011fa. Allah yard\u0131mc\u0131n olsun Kadir BABA .\nGALATASARAY taraftar\u0131 hep seninle! #KadirAkta\u015fSu\u00e7suzd\u2026
1445282569.29::RT @EndlesGALA: T\u00fcrkiyedir #Galatasaray \nKim daha b\u00fcy\u00fck tart\u0131smayal\u0131m isterseniz http:\/\/t.co\/0K6jLC0CHd
1445282570.29::Fenerbah\u00e7e - Galatasaray derbisinin biletlerine yo\u011fun ilgi:  https:\/\/t.co\/VZ2whsiZNo
1445282571.2::RT @EndlesGALA: T\u00fcrkiyedir #Galatasaray \nKim daha b\u00fcy\u00fck tart\u0131smayal\u0131m isterseniz http:\/\/t.co\/0K6jLC0CHd
1445282571.95::Kom\u015fularla s\u0131f\u0131r sorundan s\u0131f\u0131r kom\u015fuya!! #galatasaray #NTV #yakinda #Bug\u00fcnTV #Ak\u015fam #Takvim #D\u00fcnyaKahveG\u00fcn\u00fc #menzil https:\/\/t.co\/bcHNoG4UMN

How can I fix?

Upvotes: 1

Views: 1250

Answers (1)

Stephane Martin
Stephane Martin

Reputation: 1642

tweepy gives you JSON-encoded data like almost every web-service in the world.

tweet    = data.split(',"text":"')[1].split('","source')[0]

that's not the right way to deal with json. do instead:

import json (at top of script)

then:

class listener(StreamListener):
    def on_data(self,data):
        tweet = json.loads(data)
        print tweet

tweet will be a classical python object. here its a dict.

Upvotes: 2

Related Questions