CODEWITHSUNDEEP
javascripttwitter
SimonBarker
SimonBarker

Reputation: 1424

Get Twitter Feed as JSON without authentication

I wrote a small JavaScript a couple of years ago that grabbed a users (mine) most recent tweet and then parsed it out for display including links, date etc.

It used this json call to retrieve the tweets and it no longer works.

http://twitter.com/statuses/user_timeline/radfan.json

It now returns the error:

{"errors":[{"message":"Sorry, that page does not exist","code":34}]}

I have looked at using the api version (code below) but this requires authentication which I would rather avoid having to do as it is just to display my latest tweet on my website which is public anyway on my profile page:

http://api.twitter.com/1/statuses/radfan.json

I haven't kept up with Twitter's API changes as I no longer really work with it, is there a way round this problem or is it no longer possible?

Upvotes: 27

Views: 94116

Answers (8)

RafaSashi
RafaSashi

Reputation: 17205

You can access and scrape Twitter via advanced search without being logged in:

GET request

When performing a basic search request you get:

https://twitter.com/search?q=Babylon%205&src=typd
  • q (our query encoded)
  • src (assumed to be the source of the query, i.e. typed)

by default, Twitter returns top 25 results, but if you click on all you can get the realtime tweets:

https://twitter.com/search?f=realtime&q=Babylon%205&src=typd

JSON contents

More Tweets are loaded on the page via AJAX:

https://twitter.com/i/search/timeline?f=realtime&q=Babylon%205&src=typd&include_available_features=1&include_entities=1&last_note_ts=85&max_position=TWEET-553069642609344512-553159310448918528-BD1UO2FFu9QAAAAAAAAETAAAAAcAAAASAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Use max_position to request the next tweets

The following json array returns all you need to scrape the contents:

https://twitter.com/i/search/timeline?f=realtime&q=Babylon%205&src=typd
  • has_more_items (bool)
  • items_html (html)
  • max_position (key)
  • refresh_cursor (key)

DOM elements

Here comes a list of DOM elements you can use to extract

The authors twitter handle

div.original-tweet[data-tweet-id]

The name of the author

div.original-tweet[data-name]

The user ID of the author

div.original-tweet[data-user-id]

Timestamp of the post

span._timestamp[data-time]

Timestamp of the post in ms

span._timestamp[data-time-ms]

Text of Tweet

p.tweet-text

Number of Retweets

span.ProfileTweet-action–retweet > span.ProfileTweet-actionCount[data-tweet-stat-count]

Number of Favo

span.ProfileTweet-action–favorite > span.ProfileTweet-actionCount[data-tweet-stat-count]

Resources

Upvotes: 14

Fiach Reid
Fiach Reid

Reputation: 7059

You can use a Twitter API wrapper, such as TweetJS.com which offers a limited set of the Twitter API's functionality, but does not require authentication. It's called like this;

TweetJs.ListTweetsOnUserTimeline("PetrucciMusic",
function (data) {
    console.log(data);
});

Upvotes: 0

MarcoK
MarcoK

Reputation: 6110

As you can see in the documentation, using the REST API you'll need OAuth Tokens in order to do this. Luckily, we can use the Search (which doesn't use OAuth) and use the from:[USERNAME] operator

Example: http://search.twitter.com/search.json?q=from:marcofolio
Will give you a JSON object with tweets from that user, where

object.results[0]

will give you the last tweet.

Upvotes: 0

Jon Olick
Jon Olick

Reputation: 21

Here is a quick hack (really a hack, should be used with caution as its not future proof) which uses http://anyorigin.com to scrape twitter site for the latest tweets.

http://codepen.io/JonOlick/pen/XJaXBd

It works by using anyorigin (you have to pay to use it) to grab the HTML. It then parses the HTML using jquery to extract out the relevant tweets.

Tweets on the mobile site use a div with the class .tweet-text, so this is pretty painless.

The relevant code looks like this:

$.getJSON('http://anyorigin.com/get?url=mobile.twitter.com/JonOlick&callback=?', function(data){

  // Remap ... utf8 encoding to ascii. 
  var bar = data.contents;
  bar = bar.replace(/…/g, '...');

  var el = $( '<div></div>' );
  el.html(bar);

  // Change all links to point back at twitter
  $('.twitter-atreply', el).each(function(i){
    $(this).attr('href', "https://twitter.com" + $(this).attr('href'))
  });

  // For all tweets
  $('.tweet-text', el).each(function(i){
    // We only care about the first 4 tweets
    if(i < 4) {
      var foo = $(this).html();
      $('#test').html($('#test').html() + "<div class=ProfileTweet><div class=ProfileTweet-contents>" + foo + "</div></div><br>");
    }
  });

});

Upvotes: 0

Dustin
Dustin

Reputation: 376

Previously the Search API was the only Twitter API that didn't require some form of OAuth. Now it does require auth.

Twitter's Search API is acquired from a third party acquisition - they rarely support it and are seemingly unenthused that it even exists. On top of that, there are many limitations to the payload, including but not limited to a severely reduced set of key:value pairs in the JSON or XML file you get back.

When I heard this, I was shocked. I spent a LONG time figuring out how to use the least amount of code to do a simple GET request (like displaying a timeline).

I decided to go the OAuth route to be able to ensure a relevant payload. You need a server-side language to do this. JavaScript is visible to end users, and thus it's a bad idea to include the necessary keys and secrets in a .js file.

I didn't want to use a big library so the answer for me was PHP and help from @Rivers' answer here. The answer below it by @lackovic10 describes how to include queries in your authentication.

I hope this helps others save time thinking about how to go about using Twitter's API with the new OAuth requirement.

Upvotes: 17

cosmocatalano
cosmocatalano

Reputation: 87

If you're still looking for unauthenticated tweets in JSON, this should work: https://github.com/cosmocatalano/tweet-2-json

Upvotes: 6

mcbjam
mcbjam

Reputation: 7454

The method "GET statuses/user_timeline" need a user Authentification like you can see on the official documentation :

You can use the search method "GET search" wich not require authentification.

You have a code for starting here : http://jsfiddle.net/73L4c/6/

function searchTwitter(query) {
    $.ajax({
        url: 'http://search.twitter.com/search.json?' + jQuery.param(query),
        dataType: 'jsonp',
        success: function(data) {
            var tweets = $('#tweets');
            tweets.html('');
            for (res in data['results']) {
                tweets.append('<div>' + data['results'][res]['from_user'] + ' wrote: <p>' + data['results'][res]['text'] + '</p></div><br />');
            }
        }
    });
}

$(document).ready(function() {
    $('#submit').click(function() {
        var params = {
            q: $('#query').val(),
            rpp: 5
        };
        // alert(jQuery.param(params));
        searchTwitter(params);
    });
})

Upvotes: -4

Dorukcan Kişin
Dorukcan Kişin

Reputation: 1131

You can use the twitter api v1 to take the tweets without using OAuth. For example: this link turns @jack's last 100 tweets.

The timeline documentation is here.

Upvotes: -1

Related Questions