CODEWITHSUNDEEP
javahttpclient
arsenal
arsenal

Reputation: 24154

Avoid Circular Redirect using HttpClient 4.1.1

How can I avoid circular redirect using HttpClient 4.1.1. As I am getting the error like this:-

executing requestGET http://home.somehost.com/Mynet/pages/cHome.xhtml HTTP/1.1
org.apache.http.client.ClientProtocolException
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:822)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
    at edu.uci.ics.crawler4j.url.WebURL.setURL(WebURL.java:122)
    at edu.uci.ics.crawler4j.crawler.CrawlController.addSeed(CrawlController.java:207)
    at edu.uci.ics.crawler4j.example.advanced.Controller.main(Controller.java:31)
Caused by: org.apache.http.client.CircularRedirectException: Circular redirect to 'http://home.somehost.com/Mynet/pages/Home.xhtml'
    at org.apache.http.impl.client.DefaultRedirectStrategy.getLocationURI(DefaultRedirectStrategy.java:168)
    at org.apache.http.impl.client.DefaultRedirectStrategy.getRedirect(DefaultRedirectStrategy.java:193)
    at org.apache.http.impl.client.DefaultRequestDirector.handleResponse(DefaultRequestDirector.java:1021)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:482)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)

This is my code...

DefaultHttpClient client = null;

        try
        {
            // Set url
            //URI uri = new URI(url.toString());

            client = new DefaultHttpClient();

            client.getCredentialsProvider().setCredentials(
                    new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM),
                    new UsernamePasswordCredentials("test", "test"));


            URL url1 = new URL (url);
            HttpURLConnection connection = (HttpURLConnection) url1.openConnection();
            connection.setFollowRedirects(false);

            HttpGet request = new HttpGet(url);
            final HttpParams params = new BasicHttpParams();
            HttpClientParams.setRedirecting(params, false);
            HttpContext context = new BasicHttpContext();

            System.out.println("----------------------------------------");
            System.out.println("executing request" + request.getRequestLine());
            HttpResponse response = client.execute(request, context);
            HttpEntity entity = response.getEntity();


            System.out.println(response.getStatusLine());
                    InputStream content = entity.getContent();
                    BufferedReader in   = 
                        new BufferedReader (new InputStreamReader (content));
                    String line;
                    while ((line = in.readLine()) != null) {
                       // System.out.println(line);
                    }
                } catch(Exception e) {
                    e.printStackTrace();
                }

Upvotes: 19

Views: 34384

Answers (6)

vanmathi
vanmathi

Reputation: 1

I faced this issue while spring version upgrade, the context is not initialized properly in my case.

In org.apache.http.impl.client.DefaultRedirectStrategy:

RedirectLocations redirectLocations = (RedirectLocations) clientContext.getAttribute(
                HttpClientContext.REDIRECT_LOCATIONS);

The value of the clientContext should be basicHttpContext, but Spring Web (4.3.x.RELEASE) is initializing the context in:

org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal();

The value of clientContext is changing, which results in circular redirect error. The Spring Web (3.2.x.RELEASE) don't initialize the context and value will be null.

Upvotes: 0

Provat
Provat

Reputation: 41

You may try:

RequestConfig requestConfig = RequestConfig.custom()
                              .setCircularRedirectsAllowed(true)
                              .build();

HttpClient httpClient = HttpClients.custom()
                        .setDefaultRequestConfig(requestConfig)
                        .setRedirectStrategy(new LaxRedirectStrategy())
                        .build();

HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);

Upvotes: 4

Chris
Chris

Reputation: 4593

Check that your request isnt sent to a proxy before being sent to the url you requested.

Upvotes: 0

nos
nos

Reputation: 229088

You can set the ClientPNames.ALLOW_CIRCULAR_REDIRECTS to true, this will allow redirects to the same location. 

  client.getParams().setParameter(ClientPNames.ALLOW_CIRCULAR_REDIRECTS, true);

See more info here

Upvotes: 36

lex chou
lex chou

Reputation: 95

There is a bug that will cause circular redirect in Apache HttpClient since 4.0, it wasn't fixed even in the latest version.

In DefaultRequestDirector.java, it creates a HttpRedirect to perform redirection, and it will reuse all headers in your original HttpGet, the problem here is it will also reuse Host header, which mean the server will still get the original host after it's attempt to redirect to new URI.

I fixed this by reimplemented the DefaultRequestDirector:

public class RedirectRequestDirector extends DefaultRequestDirector
{
    RedirectRequestDirector(
            final HttpRequestExecutor requestExec,
            final ClientConnectionManager conman,
            final ConnectionReuseStrategy reustrat,
            final ConnectionKeepAliveStrategy kastrat,
            final HttpRoutePlanner rouplan,
            final HttpProcessor httpProcessor,
            final HttpRequestRetryHandler retryHandler,
            final RedirectHandler redirectHandler,
            final AuthenticationHandler targetAuthHandler,
            final AuthenticationHandler proxyAuthHandler,
            final UserTokenHandler userTokenHandler,
            final HttpParams params) 
    {
        super(requestExec, conman, reustrat, kastrat, rouplan, httpProcessor, retryHandler, redirectHandler, targetAuthHandler, proxyAuthHandler, userTokenHandler, params);

    }
    @Override
    protected RoutedRequest handleResponse(RoutedRequest roureq,
            HttpResponse response,
            HttpContext context)
                    throws HttpException, IOException
    {
        RoutedRequest req = super.handleResponse(roureq, response, context);
        if(req != null)
        {
            String redirectTarget = req.getRoute().getTargetHost().getHostName();
            req.getRequest().getOriginal().setHeader("Host", redirectTarget);
        }
        return req;
    }

}

and DefaultHttpClient:

public class RedirectHttpClient extends DefaultHttpClient
{
    @Override
    protected RequestDirector createClientRequestDirector(
            final HttpRequestExecutor requestExec,
            final ClientConnectionManager conman,
            final ConnectionReuseStrategy reustrat,
            final ConnectionKeepAliveStrategy kastrat,
            final HttpRoutePlanner rouplan,
            final HttpProcessor httpProcessor,
            final HttpRequestRetryHandler retryHandler,
            final RedirectHandler redirectHandler,
            final AuthenticationHandler targetAuthHandler,
            final AuthenticationHandler proxyAuthHandler,
            final UserTokenHandler stateHandler,
            final HttpParams params) {
        return new RedirectRequestDirector(
                requestExec,
                conman,
                reustrat,
                kastrat,
                rouplan,
                httpProcessor,
                retryHandler,
                redirectHandler,
                targetAuthHandler,
                proxyAuthHandler,
                stateHandler,
                params);
    }
}

Now I won't complain about the Circular Redirect.

Upvotes: 2

Adam Batkin
Adam Batkin

Reputation: 52994

You just avoided it. HttpClient detected the circular redirect and threw an exception. Had it not been "avoided", it would continue redirecting forever (until you decided to kill the process). There aren't a whole lot of other options, if that's what the server responds with.

The only way to truly avoid a circular redirect loop is to fix the server.

If you are wondering what is going on (like why it seems to work find in a browser but not from your program), try turning on some of the extra HttpClient logging. In particular, make sure you can see all of the HTTP headers being sent back and forth. You can then look at the conversation taking place when you make the same request in your browser, noting the differences. It could be a missing cookie, crazy browser detection, etc...

There are a number of ways of tracing your browser's communications. Here are a few ways that I often use, in order from easiest to hardest (IMHO):

  • Firefox + HttpFox (or LiveHttpHeaders, Firebug, etc...)
  • Fiddler (Windows only)
  • Wireshark/tcpdump

For low-level testing, try using telnet (unless you use Windows, in which case you may be better off with something like PuTTY/plink) and ruling in/out what changes cause the circular redirects.

Upvotes: 3

Related Questions