CODEWITHSUNDEEP
javaapache-httpclient-4.x
Tom Henricksen
Tom Henricksen

Reputation: 161

How to handle 419 http response in Java

I am trying to parse through a particular website and am getting an HTTP response code of 419 when my java code calls it. I need to parse through the response to find content and I am stuck on the response code.

I have tried putting together a Java program using apache http client(version 4.5.6) to call a website that I need to parse. The http response code I get back is 419.

try (CloseableHttpClient httpclient = HttpClients.createDefault()) {
    HttpGet httpGet = new HttpGet("http://www.website.com");
    try (CloseableHttpResponse response1 = httpclient.execute(httpGet)) {
        System.out.println(response1.getStatusLine());
        HttpEntity entity1 = response1.getEntity();
        EntityUtils.consume(entity1);
    }
}

The result that it prints out is this:

HTTP/1.1 419 status code 419

I am expecting a 200

HTTP/1.1 200 OK

I get that when I change the website to google or other sites.

Upvotes: 5

Views: 1411

Answers (2)

Md.Habibur Rahman
Md.Habibur Rahman

Reputation: 351

If you hit the website URL from any browser then you can easily get the content, but when you hit the url from postman or from your code, then it says the status code 419, which actually means that it expects a csrf token. But how does the browser get the content then? it is because the site you are trying to hit is configured in such a way that when it gets request from browser Only then it will render the result. When the browser hits the http request, it sends a param called "user-agent" where it sends some information. for example-

user-agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"

  • "Mozilla/5.0" - Indicates that the browser is compatible with Mozilla, which is a common web browser engine.

  • "(Windows NT 10.0; Win64; x64)" - Indicates that the operating system is Windows 10, and that it is running on a 64-bit processor.

  • "AppleWebKit/537.36 (KHTML, like Gecko)" - Indicates that the browser is using the WebKit rendering engine, which is also used by other popular browsers like Safari and Opera. "KHTML" refers to the fact that WebKit is based on the KHTML layout engine.

  • "Chrome/58.0.3029.110" - Indicates that the browser is Google Chrome version 58.0.3029.110.

  • "Safari/537.36" - Indicates that the browser is also using some components from the Safari browser.

So, if we can fake the http call from our java code(below) as if sending from a browser then it no longer will give the 419 status.

            String url = "your-site-url";
            HttpClient client = HttpClientBuilder.create().build();
            request = new HttpGet(url);

            String userAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36";
            request.addHeader("User-Agent", userAgent);
            HttpResponse response = client.execute(request);

            HttpEntity entity = response.getEntity();
            String content = EntityUtils.toString(entity);
            System.out.println(content);

p.s- apache httpclient is used here, make sure to add the dependency to the pom.xml

Upvotes: 0

MD Ruhul Amin
MD Ruhul Amin

Reputation: 4502

I was making a get request through HttpClient library as well as from POSTMAN and facing same 419 error. To solve this 419 error we need to add csrf token while making form submission.

However, In-case if you are still wondering how to find csrf token even when you are making a GET request and facing status 419. In my case I solved the problem by adding the user-agent: xxxx token in header.

Example:

user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36

HttpClient Code:

  connectionManager = new PoolingHttpClientConnectionManager();
  ...
  ...
  ...

  httpClient = HttpClients.custom()
                .setConnectionManager(connectionManager)
                .setRedirectStrategy(new LaxRedirectStrategy())
                .setUserAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36")
                .build();

Upvotes: 3

Related Questions