CODEWITHSUNDEEP
apitwittertweets
apscience
apscience

Reputation: 7253

How to retrieve all tweets from a user and not just the first 3,200 as Twitter limits it’s timeline and API to

With https://dev.twitter.com/docs/api/1/get/statuses/user_timeline I can get 3,200 most recent tweets. However, certain sites like http://www.mytweet16.com/ seems to bypass the limit, and my browse through the API documentation could not find anything.

How do they do it, or is there another API that doesn't have the limit?

Upvotes: 48

Views: 136997

Answers (5)

sevenkul
sevenkul

Reputation: 966

You can use twitter search page to bypass 3,200 limit. However you have to scroll down many times in the search results page. For example, I searched tweets from @beyinsiz_adam. This is the link of search results:

https://twitter.com/search?q=from%3Abeyinsiz_adam&src=typd&f=realtime

Now in order to scroll down many times, you can use the following javascript code.

var myVar=setInterval(function(){myTimer()},1000);
function myTimer() {
    window.scrollTo(0,document.body.scrollHeight);
}

Just run it in the FireBug console. And wait some time to load all tweets.

Upvotes: 29

Zilvinas
Zilvinas

Reputation: 11

You can use a Python library snscrape to do it. Or you can use ExportData tool to get all tweets for the user, which returns already preprocessed CSV and spreadsheet files. The first option is free, but has less information and requires more manual work.

Upvotes: 0

Paul Knopf
Paul Knopf

Reputation: 9776

You can use a tool I wrote that bypasses the limit.

It saves the Tweets in a JSON format.

https://github.com/pauldotknopf/twitter-dump

Upvotes: 2

Arjun Jain
Arjun Jain

Reputation: 409

I've been in this (Twitter) industry for a long time and witnessed lots of changes in Twitter API and documentation. I would like to clarify one thing to you. There is no way to surpass 3200 tweets limit. Twitter doesn't provide this data even in its new premium API.

The only way someone can surpass this limit is by saving the tweets of an individual Twitter user.

There are tools available which claim to have a wide database and provide more than 3200 tweets. Few of them are followersanalysis.com, keyhole.co which I know of.

Upvotes: 2

meetar
meetar

Reputation: 7611

The only way to see more is to start saving them before the user's tweet count hits 3200. Services which show more than 3200 tweets have saved them in their own dbs. There's currently no way to get more than that through any Twitter API.

http://www.quora.com/Is-there-a-way-to-get-more-than-3200-tweets-from-a-twitter-user-using-Twitters-API-or-scraping

https://dev.twitter.com/discussions/276

Note from that second link: "…the 3,200 limit is for browsing the timeline only. Tweets can always be requested by their ID using the GET statuses/show/:id method."

Upvotes: 9

Related Questions