CODEWITHSUNDEEP
pythonpython-2.7twittertweepy
KubiK888
KubiK888

Reputation: 4723

Python "while true" loop does NOT end (using Python Tweepy)

The following code seems to be mostly "working". Meaning that it scrapes all the tweets from Twitter API for a specified day. Though it seems the while True loop never breaks and I don't see the expected "Finished!!" string even through the csv file is complete.

import tweepy
import time
import csv

ckey = "xxx"
csecret = "xxx"
atoken = "xxx-xxx"
asecret = "xxx"

OAUTH_KEYS = {'consumer_key':ckey, 'consumer_secret':csecret,
    'access_token_key':atoken, 'access_token_secret':asecret}
auth = tweepy.OAuthHandler(OAUTH_KEYS['consumer_key'], OAUTH_KEYS['consumer_secret'])
api = tweepy.API(auth)

startSince = '2014-10-03'
endUntil = '2014-10-04'
suffix = '_03OCT2014.csv'

searchTerms = 'xyz'

tweets = tweepy.Cursor(api.search, q=searchTerms,
    since=startSince, until=endUntil).items()

while True:
    try: 
        for tweet in tweets:

            placeHolder = []
            placeHolder.append(tweet.author.name.encode('utf8'))
            placeHolder.append(tweet.author.screen_name.encode('utf8'))
            placeHolder.append(tweet.created_at)

            prefix = 'TweetData_xyz'
            wholeFileName = prefix + suffix     
            with open(wholeFileName, "ab") as f:
                writeFile = csv.writer(f)
                writeFile.writerow(placeHolder)

    except tweepy.TweepError:
        time.sleep(60*15)
        continue

    except IOError:
        time.sleep(60*5)
        continue

    except StopIteration:
        break

print "Finished!!!"

Upvotes: 0

Views: 1267

Answers (2)

Martijn Pieters
Martijn Pieters

Reputation: 1121924

StopIteration is never raised in your code. The for statement would catch it if it were raised by tweepy.Cursor().items(), it would not propagate further.

Just break out if the for loop ends:

while True:
    try:
        for tweet in tweets:
            # do stuff

        # completed iterating successfully
        break

and remove the except StopIteration: handler altogether.

Upvotes: 2

Beri
Beri

Reputation: 11610

You code has no exit condition. It seems you don;t want to exit that loop if you got an error thrown. So I assume that when you reach end ob your while body you would like to exit, yes?

def process_tweet(tweet):

    placeHolder = []
    placeHolder.append(tweet.author.name.encode('utf8'))
    placeHolder.append(tweet.author.screen_name.encode('utf8'))
    placeHolder.append(tweet.created_at)

    prefix = 'TweetData_xyz'
    wholeFileName = prefix + suffix     
    with open(wholeFileName, "ab") as f:
       writeFile = csv.writer(f)
       writeFile.writerow(placeHolder)

while True:
    try: 
        for tweet in tweets:
            process_tweet(tweet)
        break
    except tweepy.TweepError:
        time.sleep(60*15)
        continue

    except IOError:
        time.sleep(60*5)
        continue

    except StopIteration:
        break

print "Finished!!!"

Upvotes: 1

Related Questions