CODEWITHSUNDEEP
javatwitter
Dexter
Dexter

Reputation: 11831

Issue using Twitter REST API

I have a list of screen names on Twitter and I wish to get meta data about their twitter profile. I am using Twitter's REST API for the same. The users/show method is apt for my task. The API documentation clearly states that it requires no authentication. Here's the code I wrote for my task:

package Twitter;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;

public class TwitterAPI {

    private static String url = "http://api.twitter.com/1/users/show/";

    /*
     * Sends a HTTP GET request to a URL     
     * @return - The response from the end point
     */
    public static String sendGetRequest(String endpoint, String screen_name) {
        String result = null;


        if (endpoint.startsWith("http://")){
            //Send HTTP request to the servlet
            try {
                //Construct data
                StringBuffer data = new StringBuffer();

                //Send data
                String urlStr = endpoint ;  

                if(screen_name!=null && screen_name.length() > 0){
                    urlStr += screen_name + ".json";
                }
                System.out.println(screen_name.length());
                System.out.println("The URL call is: " + urlStr);
                URL url = new URL(urlStr);
                URLConnection conn = url.openConnection ();

                //Get the response
                BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
                StringBuffer sb = new StringBuffer();
                String line;

                while((line = rd.readLine())!=null){
                    sb.append(line);
                }

                rd.close();
                result = sb.toString();

            } catch (MalformedURLException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            } catch (IOException e) {

                //If API issue, collect screen names to write to API issue file             
                System.out.println("Twitter API issue :" + screen_name);

                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }


        return result;
    }
    /**
     * @param args
     */
    public static void main(String[] args) {
        // TODO Auto-generated method stub

        String result = sendGetRequest(url, "denzil_correa");
        System.out.println(result);
    }

}

However, on running the same I receive the following exception :

13
The URL call is: http://api.twitter.com/1/users/show/denzil_correa.json
Twitter API issue :denzil_correa
java.net.ConnectException: Connection timed out: connect
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:365)
    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:227)
null
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:214)
    at java.net.Socket.connect(Socket.java:531)
    at java.net.Socket.connect(Socket.java:481)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
    at sun.net.www.http.HttpClient.New(HttpClient.java:306)
    at sun.net.www.http.HttpClient.New(HttpClient.java:323)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:783)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:724)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:649)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:972)
    at Twitter.TwitterAPI.sendGetRequest(TwitterAPI.java:43)
    at Twitter.TwitterAPI.main(TwitterAPI.java:76)

The URL is correct as when I try the URL : http://api.twitter.com/1/users/show/denzil_correa.json in my browser I receive the following:

{"time_zone":"Mumbai","description":"","lang":"en","profile_link_color":"1F98C7","status":{"coordinates":null,"contributors":null,"in_reply_to_screen_name":"shailaja","truncated":false,"in_reply_to_user_id":14089830,"in_reply_to_status_id":16789217674,"source":"web","created_at":"Tue Jun 22 19:43:46 +0000 2010","place":null,"geo":null,"favorited":false,"id":16793898396,"text":"@shailaja Harsh !"},"profile_background_image_url":"http://s.twimg.com/a/1276711174/images/themes/theme2/bg.gif","profile_sidebar_fill_color":"DAECF4","following":false,"profile_background_tile":false,"created_at":"Sun Jun 29 20:23:29 +0000 2008","statuses_count":1157,"profile_sidebar_border_color":"C6E2EE","profile_use_background_image":true,"followers_count":169,"contributors_enabled":false,"notifications":false,"friends_count":246,"protected":false,"url":"http://https://sites.google.com/a/iiitd.ac.in/denzilc/","profile_image_url":"http://a3.twimg.com/profile_images/643636081/Cofee_Mug_normal.jpg","geo_enabled":true,"profile_background_color":"C6E2EE","name":"Denzil Correa","favourites_count":3,"location":"India","screen_name":"denzil_correa","id":15273105,"verified":false,"utc_offset":19800,"profile_text_color":"663B12"}

which is in the JSON format I want.

Kindly let me know if I am doing anything stupid here.

Regards, --Denzil

Hank/Splix as told I tried using the HTTP Components Client. Here's my modified code : 

 
package Twitter;



import java.io.IOException;



import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;



import org.apache.http.impl.client.DefaultHttpClient;



public class TwitterAPI {



private static String url = "http://api.twitter.com/1/users/show/denzil_correa.json";


/**
 * @param args
 */
public static void main(String[] args) {
    // TODO Auto-generated method stub

    HttpClient httpclient = new DefaultHttpClient();
    HttpGet httpget = new HttpGet(url);

    try {
        HttpResponse response = httpclient.execute(httpget);
        System.out.println(response.toString());

    } catch (ClientProtocolException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}




}
Here's the error I receive: 
 
org.apache.http.conn.HttpHostConnectException: Connection to http://api.twitter.com refused
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:127)
    at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:147)
    at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:108)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
    at Twitter.TwitterAPI.main(TwitterAPI.java:30)
Caused by: java.net.ConnectException: Connection timed out: connect
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:365)
    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:227)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:214)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:378)
    at java.net.Socket.connect(Socket.java:531)
    at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:123)
    ... 7 more
Surprisingly this also gives a similar exception to the code written for handling HTTP responses manually. I understand that manually handling HTTP responses may be sub-optimal but currently I am not looking at writing optimal code. I would like to get my task done even if it means to be quick & dirty.

Just to let you know, I can successfully call the Facebook Graph API using the first code I posted. I am receiving the same response I would receive if I paste the URL in my browser.

I will also try using the Twitter4J API once again and check if I can get my task done. Will keep you updated.

So, here's the code using Twitter4J : 


package Twitter;



import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
import twitter4j.User;



public class TwitterAPI {



/**
 * @param args
 */
public static void main(String[] args) {
    Twitter unauthenticatedTwitter = new TwitterFactory().getInstance();
    try {
        User user = unauthenticatedTwitter.showUser("denzil_correa");

        System.out.println(user.getLocation());
    } catch (TwitterException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}




}
Pretty straightforward as expected using the API. However, here's the error I receive: 

Jun 23, 2010 7:12:10 PM twitter4j.internal.logging.CommonsLoggingLogger info
INFO: Using class twitter4j.internal.logging.CommonsLoggingLoggerFactory as logging factory.
Jun 23, 2010 7:12:11 PM twitter4j.internal.logging.CommonsLoggingLogger info
INFO: Use twitter4j.internal.http.HttpClientImpl as HttpClient implementation.
TwitterException{statusCode=-1, retryAfter=0, rateLimitStatus=null}
    at twitter4j.internal.http.HttpClientImpl.request(HttpClientImpl.java:316)
    at twitter4j.internal.http.HttpClientWrapper.request(HttpClientWrapper.java:68)
    at twitter4j.internal.http.HttpClientWrapper.get(HttpClientWrapper.java:90)
    at twitter4j.Twitter.showUser(Twitter.java:538)
    at Twitter.TwitterAPI.main(TwitterAPI.java:17)
Caused by: java.net.ConnectException: Connection refused: connect
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:365)
    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:227)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:214)
    at java.net.Socket.connect(Socket.java:531)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:152)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
    at com.ibm.net.ssl.www2.protocol.https.c.(c.java:166)
    at com.ibm.net.ssl.www2.protocol.https.c.a(c.java:9)
    at com.ibm.net.ssl.www2.protocol.https.d.getNewHttpClient(d.java:55)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:724)
    at com.ibm.net.ssl.www2.protocol.https.d.connect(d.java:20)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:972)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:385)
    at com.ibm.net.ssl.www2.protocol.https.b.getResponseCode(b.java:52)
    at twitter4j.internal.http.HttpResponseImpl.(HttpResponseImpl.java:42)
    at twitter4j.internal.http.HttpClientImpl.request(HttpClientImpl.java:279)
    ... 4 more

Again, I see that the error is essentially the same. So, all options tried! I'm sure there's something I am missing here. It would be great if you could point out the same.

Hank, Unfortunately the same doesn't work in Python too :-(


Traceback (most recent call last):
  File "", line 1, in 
    urllib.urlopen("http://api.twitter.com/1/users/show/denzil_correa.json").read()
  File "C:\Python26\lib\urllib.py", line 86, in urlopen
    return opener.open(url)
  File "C:\Python26\lib\urllib.py", line 205, in open
    return getattr(self, name)(url)
  File "C:\Python26\lib\urllib.py", line 347, in open_http
    errcode, errmsg, headers = h.getreply()
  File "C:\Python26\lib\httplib.py", line 1060, in getreply
    response = self._conn.getresponse()
  File "C:\Python26\lib\httplib.py", line 986, in getresponse
    response.begin()
  File "C:\Python26\lib\httplib.py", line 391, in begin
    version, status, reason = self._read_status()
  File "C:\Python26\lib\httplib.py", line 349, in _read_status
    line = self.fp.readline()
  File "C:\Python26\lib\socket.py", line 397, in readline
    data = recv(1)
IOError: [Errno socket error] [Errno 10054] An existing connection was forcibly closed by the remote host

Upvotes: 0

Views: 2379

Answers (1)

Hank Gay
Hank Gay

Reputation: 71939

As @splix mentioned in the comments, doing this using just java.net is… suboptimal. I've never yet encountered a situation where HttpClient wasn't a better option. Event better is his suggestion of twitter4j; unless you're trying to create an alternative, it's almost always better to use an API wrapper like that vs. handling the raw HTTP interactions yourself.

UPDATE:

@Denzil it's odd that you're getting this same error even with twitter4j (I can't test the code until I get some free time to grab the lib, etc.) so I begin to suspect a problem on Twitter's end. If you have Python installed, try the following:

>>> import urllib
>>> urllib.urlopen("http://api.twitter.com/1/users/show/denzil_correa.json").read()

This worked for me.

UPDATE 2:

This definitely sounds like Twitter is intentionally refusing your requests. Possible reasons could include: your IP is on their blacklist for some reason, proxy voodoo, or things I haven't thought of. To elaborate on the proxy voodoo: I don't know what exactly it's doing to your requests, but it's possible it's adding a header or something that the Twitter API doesn't like. I'd recommend contacting Twitter support (if there is such a thing for API problems) or posting to the mailing list.

BTW, here's a thread from the mailing list that mentions ways to see if you're blacklisted.

Upvotes: 1

Related Questions