CODEWITHSUNDEEP
pythonapitwittertweepyrate-limiting
user2047228
user2047228

Reputation: 73

Tweepy implementation to avoid Twitter rate limiter (python)

I am trying to execute the following code and receive '88' error, also copied below. I've stripped out the twitter account handles, but I had included 25 account handles, each of which has around 3k-80k followers (probably averaging around 20k).

import time
import tweepy
import csv
from itertools import zip_longest

auth = tweepy.OAuthHandler('', '')
auth.set_access_token('', '')

api = tweepy.API(auth)

accounts = [25 twitter account handles here as strings]

numberOfAccts = len(accounts)
d = [[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],[''],['']]

with open('FollowerIDLists.csv', 'a', newline="") as f:
    #writer = csv.writer(f)
    for i in range(0, numberOfAccts):
        print(accounts[i])
        ids = []
        for page in tweepy.Cursor(api.followers_ids, screen_name=accounts[i]).pages():
            ids.extend(page)
            time.sleep(60)
        d[i] = ids
    export_data = zip_longest(*d, fillvalue = '')
    wr = csv.writer(f)
    wr.writerow(accounts)
    wr.writerows(export_data)
f.close()

And here's the error:

Traceback (most recent call last):
  File "C:\Users\USER\Desktop\composeFollowerIDLists.py", line 30, in <module>
    for page in tweepy.Cursor(api.followers_ids, screen_name=accounts[i]).pages():
  File "C:\Users\USER\AppData\Local\Programs\Python\Python36-32\lib\site-packages\tweepy\cursor.py", line 49, in __next__
    return self.next()
  File "C:\Users\USER\AppData\Local\Programs\Python\Python36-32\lib\site-packages\tweepy\cursor.py", line 75, in next
    **self.kargs)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python36-32\lib\site-packages\tweepy\binder.py", line 245, in _call
    return method.execute()
  File "C:\Users\USER\AppData\Local\Programs\Python\Python36-32\lib\site-packages\tweepy\binder.py", line 227, in execute
    raise RateLimitError(error_msg, resp)
tweepy.error.RateLimitError: [{'message': 'Rate limit exceeded', 'code': 88}]

I've heard the right way to avoid the rate limiter is to use some combination of 'wait_on_rate_limit' and 'wait_on_rate_limit_notify', but I don't know how to implement these in the above. As you can see, I tried unsuccessfully with time.sleep(60) command. Anyone who might be able to lend me a hand?

Upvotes: 0

Views: 901

Answers (1)

Syed Muzamil
Syed Muzamil

Reputation: 85

Check out my tweet crawler i have used 'wait_on_rate_limit' and 'wait_on_rate_limit_notify : https://github.com/Sy-Muzammil/Tweet-Crawlers/blob/master/geo_country.py

Upvotes: 1

Related Questions