How can I programmatically fill out a web form with JAVA?

Question

I just need help filling out a web form programmatically with java. I use Apache HttpClient 4.0.1. The form looks like this:

HTML code of it looks like this:

DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd" <ol>Some tags</ol> <ol> 
Do not show the ticket (pre)view when the user first comes to the "New Ticket" page.

Wait until they hit preview. Ticket Box (ticket fields along with description)</ol> <ol>form action="/tracenvir/newticket" method="post" id="propertyform"--div--input type="hidden" name="__FORM_TOKEN" value="dff95a43ddec5a653627d2c0"</ol>

<ol>input type="text" id="field-summary" name="field_summary" size="70"</ol> <ol>textarea id="field-description" name="field_description" class="wikitext" rows="10" cols="68"</ol> <ol>input type="hidden" name="field_status" value="new" </ol> <ol>

      input type="submit" name="preview" value="Preview" </ol> <ol>

      input type="submit" name="submit" value="Create ticket"</ol>

And there are many other tags. Here's my java code:

DefaultHttpClient client = new DefaultHttpClient();

    client.getParams().setParameter(ClientPNames.COOKIE_POLICY, CookiePolicy.BEST_MATCH);

    client.setCookieStore(new BasicCookieStore());


    //**LOG IN**//


    //System.setProperty("javax.net.ssl.trustStore", "/home/rauch/NetBeansProjects/jssecacerts");

    HttpGet login = new HttpGet("http://localhost:8000/tracenvir/login");

    client.getCredentialsProvider().setCredentials(AuthScope.ANY,

            new UsernamePasswordCredentials("rauch", "qwerty"));

And then correctly Login... I get 200 OK and everything is well.

    //**POST NewTicket**

    HttpPost post = new HttpPost("http://localhost:8000/tracenvir/newticket");

    List<NameValuePair> formparams = new ArrayList<NameValuePair>();

        formparams.add(new BasicNameValuePair("__FORM_TOKEN", cookies.get(1).getValue()));

        formparams.add(new BasicNameValuePair("field_summary", "Someerror"));

        formparams.add(new BasicNameValuePair("field_descryption","AnyDescryption"));

        formparams.add(new BasicNameValuePair("field_type", "defect"));

        formparams.add(new BasicNameValuePair("field_priority", "major"));

        formparams.add(new BasicNameValuePair("field_milestone", "milestone3"));

        formparams.add(new BasicNameValuePair("field_component", "comp2"));

        formparams.add(new BasicNameValuePair("field_version", "1.0"));

        formparams.add(new BasicNameValuePair("field_keywords", ""));

        formparams.add(new BasicNameValuePair("field_cc", ""));

        formparams.add(new BasicNameValuePair("field_owner", "java server"));

        formparams.add(new BasicNameValuePair("field_status", "new"));

        formparams.add(new BasicNameValuePair("submit", "Create ticket"));

    try {

        UrlEncodedFormEntity entity;

        entity = new UrlEncodedFormEntity(formparams, "UTF-8");

        post.setEntity(entity);

        post.addHeader("Referer","http://localhost:8000/tracenvir/newticket");


        HttpResponse response = client.execute(post);


        System.out.println("Create ticket: "+response.getStatusLine());


        client.getConnectionManager().shutdown();


    } catch(UnsupportedEncodingException ex) {

        ex.printStackTrace();

    } catch(IOException ex) {

        ex.printStackTrace();

    }

And Server responses HTTP/1.0 200 OK . But this "New Ticket" doesn't appear at ViewTickets web page. If I do the same with a normal web browser, fill out fields and push Button "Create ticket" everything is OK and I can see this NewTicket at ViewTickets web page. This is what the browser generates request:

__FORM_TOKEN=0856803edd721d8b9592231d&field_summary=fuckingStatusField&field_description=mmm+status&field_type=defect&field_priority=major&field_milestone=milestone1&field_component=component1&field_version=2.0&field_keywords=&field_cc=&field_owner=ubuntu-server&field_status=new&submit=Create+ticket)</ol>

Why does it not work? By default I shouldn't use this:

formparams.add(new BasicNameValuePair("__FORM_TOKEN", cookies.get(1).getValue()));

DefaultHttpClient must do this, but it doesn't. If I comment this statement, Server responses HTTP/1.0 400 Bad Request

What should I do to correctly fill out this form?

I tryed to imitate Browser: Firstly GET /newticket page, And then generate POST request with Headers like Browser generates.....But programatically I have 200 OK from Server, but this NewTicket doesn't appear at list of Tickets.

Answer 1

Apache Http Components (or the old HttpClient), Selenium, HtmlUnit - depends on your exact case

Answer 2

1. Use a packet capture utility such as Wireshark to monitor the http requests.

2. Compare what the browser is sending to what your code is sending.

3. Modify your code accordingly.