Apache HttpClient 4.5.5 - wrong headers in response after a GET request

Question

When I'm using Apache HttpClient and loading a webpage via GET request after the page is loaded in the response I have the headers that are different from ones I have when loading the same page in browser. Here is the example of the page: http://empoweredfoundation.org/wp-login.php?action=register

In browser I have the following headers:

Status code: 302
Content-Type: text/html; charset=UTF-8
X-Port: port_10210
X-Cacheable: YES:Forced
Location: http://empoweredfoundation.org/register/
Content-Encoding: gzip
Transfer-Encoding: chunked
Date: Thu, 22 Feb 2018 04:04:35 GMT
Age: 0
Vary: User-Agent
X-Cache: uncached
X-Cache-Hit: MISS
X-Backend: all_requests

When I use HttpClient in my application I have these headers in response:

Status code: 200
Content-Type: text/html; charset=UTF-8
X-Port: port_10210
X-Cacheable: YES:Forced
Transfer-Encoding: chunked
Date: Thu, 22 Feb 2018 04:44:58 GMT
Age: 28224
Vary: Accept-Encoding, User-Agent
X-Cache: cached
X-Cache-Hit: HIT
X-Backend: all_requests
Server: nginx/1.12.1
Date: Thu, 22 Feb 2018 04:45:24 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.45

So, I should have a 302 status code but I have 200. And also I see other headers are different than the ones from the browser. I can't figure out why and what should I do to fix this.

Here is the code:

HttpClient httpclient = null;
HttpClientBuilder builder = HttpClients.custom();
Builder requestConfigBuilder = RequestConfig.custom();

// here goes the cookie store creation, ssl configuration etc

builder.setDefaultRequestConfig(requestConfigBuilder.build());
httpclient = builder.build();

HttpResponse response = null;
HttpGet httpget = null;

Escaper escaper = UrlEscapers.urlFragmentEscaper();
httpget = new HttpGet(escaper.escape(url));
httpget.getParams().setParameter("http.socket.timeout", new Integer(socketTimeout));
httpget.getParams().setParameter("http.connection.timeout", new Integer(connectTimeout));
httpget.addHeader("Accept", "text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1");
httpget.addHeader("Accept-Language", "en-US,en;q=0.9");
httpget.addHeader("Accept-Encoding", "identity, *;q=0");

response = httpclient.execute(httpget);

I also tried CloseableHttpClient, had the same result.

Answer 1

I resolved this issue, this solution works: https://memorynotfound.com/apache-httpclient-redirect-handling-example/

I still have 200 status code, not 302. But now I can handle 302 redirects (even when response.getStatusLine() shows 200).

Here is the code from the article:

package com.memorynotfound.httpclient;

import org.apache.http.HttpHost;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.client.utils.URIUtils;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.client.LaxRedirectStrategy;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.List;

/**
 * This example demonstrates the use of {@link HttpGet} request method.
 * and handling redirect strategy with {@link LaxRedirectStrategy}
 */
public class HttpClientRedirectHandlingExample {

    public static void main(String... args) throws IOException, URISyntaxException {

        CloseableHttpClient httpclient = HttpClients.custom()
                .setRedirectStrategy(new LaxRedirectStrategy())
                .build();

        try {
            HttpClientContext context = HttpClientContext.create();
            HttpGet httpGet = new HttpGet("http://httpbin.org/redirect/3");
            System.out.println("Executing request " + httpGet.getRequestLine());
            System.out.println("----------------------------------------");

            httpclient.execute(httpGet, context);
            HttpHost target = context.getTargetHost();
            List<URI> redirectLocations = context.getRedirectLocations();
            URI location = URIUtils.resolve(httpGet.getURI(), target, redirectLocations);
            System.out.println("Final HTTP location: " + location.toASCIIString());

        } finally {
            httpclient.close();
        }
    }
}

And also I added builder.setRedirectStrategy(new LaxRedirectStrategy()); when created the HttpClient class object.

If you know any solution to get the correct status code (which should be 302), please tell me.