How to grab at least 10,000 tweets given the Twitter API Rate Lmit?

Question

As per title, how can I grab at least 10,000 tweets given that Twitter has their own limit ?

Twitter allows 180 queries per 15 minutes. So I was planning to use Timer and TimerTask to set an interval and keep running the code until I get 10,000 tweets like this:

timer.schedule((TimerTask) getTweets("$FB up"),0,900000);

The problem is that, every 15 minutes it will get back the same data as the first 15 minutes. How do I make it a continuation from where it stops in the previous 15 minutes ?

Below is the function for the getTweets(String term)

int wantedTweets = 10000;
        long lastSearchID = Long.MAX_VALUE;
        int remainingTweets = wantedTweets;
        Query query = new Query(term);
         try{ 

          while(remainingTweets > 0)
          {
            remainingTweets = wantedTweets - tweets.size();
            if(remainingTweets > 100)
            {
              query.count(100);
            }
            else
            {
             query.count(remainingTweets); 
            }
            QueryResult result = twitter.search(query);
            tweets.addAll(result.getTweets());
            Status s = tweets.get(tweets.size()-1);
            long firstQueryID = s.getId();
            query.setMaxId(firstQueryID);
            remainingTweets = wantedTweets - tweets.size();
          }

          for (int i=0 ; i < tweets.size() ; i++) {
               b = tweets.get(i);  
              //System.out.println(s);
              //System.out.println("@" + b.getUser().getScreenName() +  " - " + b.getText());
          }
        }
        catch(TwitterException te)
        {
          System.out.println("Failed to search tweets: " + te.getMessage());
        }
         return b;

Answer 1

Use the twitter streaming API instead