CODEWITHSUNDEEP
javahttpresponsehttpclient
swei
swei

Reputation: 71

HttpClient doesn't return the completed html source code

I need to login to a website, click a few links to a final screen to download some data, here is the steps:

here is my problem: I am using Httpclient. It passed the login page, and it can get to the third page, but it only return the static part on the page, the part dynamically generated data based on input 'account number' is not returned.

Here is the code:

HttpClient client = new HttpClient();
client.getHostConfiguration().setHost(loginUrl);

PostMethod postMethod = new PostMethod(serverUrl);
 // Prepare login parameters
NameValuePair[] data = {
    new NameValuePair("passUID",account),
    new NameValuePair("passUCD",password)
};

postMethod.setRequestBody(data);
// I can print out the html code of the login page here

//request the third page with URL: serverUrl4
postMethod = new PostMethod(serverUrl4);
   NameValuePair[] data2 = {
               new NameValuePair("passUID",account),
               new NameValuePair("passUCD",""),
               new NameValuePair("page", "view"),
               new NameValuePair("procacct", "0"),
               new NameValuePair("AcctNo", "xxxxxxxxx")
   };

postMethod.setRequestBody(data2);
client.executeMethod(postMethod);
byte[] responseBody = postMethod.getResponseBody();

If I paste the URL with above namevaluepairs in the URL to the browser, the account data is displayed correctly. But the responsebody doesn't return the dynamically generated account data, anything else is returned but the section of the 'account data'.

Does anybody know why? any help is highly appreciated.

Upvotes: 2

Views: 1190

Answers (2)

user435657
user435657

Reputation: 643

Usually there will be a redirect request(HTTP/1.1 302) from the server after you POST, check the status code of server responses. Also you should supply cookies which used by the server to identify logged in users.

Edit:

Wish this code snippet helps:

if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY ||  statusCode == HttpStatus.SC_MOVED_TEMPORARILY) {
    Header locationHeader = postMethod.getResponseHeader("location");
    if (locationHeader != null) {
        String location = locationHeader.getValue();
        System.out.println("The page was redirected to:" + location);
        /* **strong text**
        here is code to handle redirect to 
        "location" got from response headers
        */
    } else {
            System.err.println("Location field value is null.");
    }
}

Upvotes: 0

Andrew White
Andrew White

Reputation: 53496

Does the page in question use JavaScript to generate this data? If so, HTTPClient isn't going to be enough to get what you want.

Upvotes: 2

Related Questions