CODEWITHSUNDEEP
pythontweepy
Jose Ramon
Jose Ramon

Reputation: 5444

Remaining limits error

I am trying to find a way, to create a remain limits in tweepy, in order to handle remaining limits error that twitter API sends. I am using the following code:

limits = myapi.rate_limit_status()
remain_search_limits = limits['resources']['search']['/search/tweets']['remaining']
print remain_search_limits
print limits

Actually the above code prints:

179
{u'rate_limit_context': {u'access_token': u'179927437-vtHojLnksgUD3FfnueTWVag62Kx5mvAA3uhX7qd7'}, u'resources': {u'account': {u'/account/verify_credentials': {u'reset': 1399886205, u'limit': 15, u'remaining': 0}, u'/account/settings': {u'reset': 1399886857, u'limit': 15, u'remaining': 15}}, u'blocks': {u'/blocks/list': {u'reset': 1399886857, u'limit': 15, u'remaining': 15}, u'/blocks/ids': {u'reset': 1399886857, u'limit': 15, u'remaining': 15}}, u'users': {u'/users/contributors': {u'reset': 1399886857, u'limit': 15, u'remaining': 15}, u'/users/lookup': {u'reset':

The weird issue, is that every time that I print remain_search_limits I am getting 179 as answer. What should I have to use in order to waint until I ll be able again to fetched data from twitter?

My final code is the following:

limits = api.rate_limit_status()
remain_follower_limits = limits['resources']['followers']['/followers/ids']  ['remaining']

with open("ids.txt") as f:
    content = f.readlines()

ids = []
for page in tweepy.Cursor(api.followers_ids, user_id="dankanter").pages():
while remain_follower_limits >4:
    limits = api.rate_limit_status()
    remain_follower_limits = limits['resources']['followers']['/followers/ids']['remaining']
    ids.extend(page)
else:
    time.sleep(60)

print ids
print len(ids)

However sometimes it stucks in the line limits = api.rate_limit_status() inside while loop. In case that a user has millions of users it returns only a subset which actually is very small.

Upvotes: 3

Views: 184

Answers (1)

gumption
gumption

Reputation: 1128

[Summarizing, clarifying and elaborating on the comment thread in the initial post]

I get the same value, too ... unless/until I make a call to myapi.search, after which a lower value is returned.

limits['resources'] contains a list of dictionaries for different resource families (types of API calls):

['account', 'blocks', 'users', 'friends', 'help', 'saved_searches', 'lists', 'search', 'application', 'trends', 'followers', 'favorites', 'friendships', 'geo', 'direct_messages', 'statuses', 'mutes'].

Each of the dictionaries associated with a resource family includes one or more keys representing the specific methods associated with that family. The values of each of those keys, in turn, are dictionaries with keys for 'limit', 'remaining' and 'reset'.

The documentation for application/rate_limit_status method describes these as follows:

This method responds with a map of methods belonging to the families specified by the resources parameter, the current remaining uses for each of those resources within the current rate limiting window, and its expiration time in epoch time. It also includes a rate_limit_context field that indicates the current access token or application-only authentication context.

Since limits['resources']['search']['/search/tweets']['remaining'] represents the remaining search calls you can make, you can continue making additional search calls as long as this number is greater than zero; when it reaches zero, you will need to wait to make additional calls until the reset time (in Epoch format) is reached.

With respect to a followup comment, the limits['resources']['followers'] resource family contains keys for the methods '/followers/ids' and '/followers/list', and the 'remaining' calls can be accessed using the same pattern as for search.

You may find some additional useful information in this thread: Return the number of remaining hits tweepy

Upvotes: 2

Related Questions