CODEWITHSUNDEEP
pythontwitterpython-twitter
mou55
mou55

Reputation: 730

Get the top followed followers of a user in twitter using python-twitter

I want to get the top followed followers of a user in twitter using python-twitter. And that without getting the 'Rate limit exceeded' error message.

I can get followers of a user then get the number of folowers of each one, but the problem is when that user is big (thousands).

I use the following function to get the followers ids of a particular user:

def GetFollowerIDs(self, userid=None, cursor=-1):
   url = 'http://twitter.com/followers/ids.json'
   parameters = {}
   parameters['cursor'] = cursor
   if userid:
       parameters['user_id'] = userid
   json = self._FetchUrl(url, parameters=parameters)
   data = simplejson.loads(json)
   self._CheckForTwitterError(data)
   return data

and my code is:

import twitter
api = twitter.Api(consumer_key='XXXX',
                  consumer_secret='XXXXX',
                  access_token_key='XXXXX',
                  access_token_secret='XXXXXX')
user=api.GetUser(screen_name="XXXXXX")
users=api.GetFollowerIDs(user)

#then i make a request per follower in users so that I can sort them according to the number of followers.

the problem is that when the user has a lot of followers i get the 'Rate limit exceeded' error message.

Upvotes: 0

Views: 2507

Answers (1)

iambatman
iambatman

Reputation: 107

I think you need to get the results in chunks as explained in this link.

This is the work around currently shown on the github page. But if you would want an unlimited stream, you should upgrade the subscription for your twitter application.

def GetFollowerIDs(self, userid=None, cursor=-1, count = 10):
   url = 'http://twitter.com/followers/ids.json'
   parameters = {}
   parameters['cursor'] = cursor
   if userid:
       parameters['user_id'] = userid
   remaining = count
   while remaining > 1:
       remaining -= 1
       json = self._FetchUrl(url, parameters=parameters)
       try:
           data = simplejson.loads(json)
           self._CheckForTwitterError(data)
       except twitterError:
           break
   return data

def main():
    api = twitter.Api(consumer_key='XXXX',
                      consumer_secret='XXXXX',
                      access_token_key='XXXXX',
                      access_token_secret='XXXXXX')
    user=api.GetUser(screen_name="XXXXXX")
    count = 100 # you can find optimum value by trial & error
    while(#users not empty):
        users=api.GetFollowerIDs(user,count)

Or another possibility might be to try running Cron jobs in intervals as explained here. http://knightlab.northwestern.edu/2014/03/15/a-beginners-guide-to-collecting-twitter-data-and-a-bit-of-web-scraping/

Construct your scripts in a way that cycles through your API keys to stay within the rate limit.

Cronjobs — A time based job scheduler that lets you run scripts at designated times or intervals (e.g. always at 12:01 a.m. or every 15 minutes).

Upvotes: 1

Related Questions