CODEWITHSUNDEEP
javahtmlpostscreen-scrapingjsoup
Guilherme Coelho Carvalho
Guilherme Coelho Carvalho

Reputation: 53

Unable to POST using jSoup

Trying to get some data out of a router website but can't figure out the POST Request Headers.

Here's the Request Headers:

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Content-Length:342
Content-Type:multipart/form-data; boundary=----WebKitFormBoundary6C8rEu8jENa10v7s
Cookie:AIROS_SESSIONID=48f1f0f41859ba467e3a2bf1de1f6dd0; ui_language=en_US
Host:10.0.0.2
Origin:http://10.0.0.2
Referer:http://10.0.0.2/login.cgi?uri=/index.cgi
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML,      like Gecko) Chrome/30.0.1599.101 Safari/537.36

And here's the Request Payload:

------WebKitFormBoundary6C8rEu8jENa10v7s
Content-Disposition: form-data; name="uri"

/index.cgi
------WebKitFormBoundary6C8rEu8jENa10v7s
Content-Disposition: form-data; name="username"

ubnt
------WebKitFormBoundary6C8rEu8jENa10v7s
Content-Disposition: form-data; name="password"

ubnt
------WebKitFormBoundary6C8rEu8jENa10v7s--

Parsing the response into a document and printing it, it shows my credentials as invalid. Could it be that I got my Content-Type wrong?

Here's my code for the connection:

res1 = Jsoup.connect("http://10.0.0.2/login.cgi")
.userAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:25.0) Gecko/20100101     Firefox/25.0")
.header("Content-Type", "multipart/form-data")
.followRedirects(false)
.referrer("http://10.0.0.2/login.cgi?uri=/index.cgi")
.data("uri", "/index.cgi")
.data("username", "ubnt").data("password", "ubnt")
.method(Method.POST).execute();

Any insight would be appreciated!

Thanks in advance!

EDIT: Parsing the response, the server returns the following error message which I believe furthers my suspicion that it is something to do with how the website handles the Content-Type.

<body>
  <b>File Upload Error: No MIME boundary found</b>
  <br /> 
  <b>There should have been a &quot;boundary=something&quot; in the Content-Type string</b>
  <br /> 
  <b>The Content-Type string was: &quot;multipart/form-data&quot;</b>
  <br /> 
  <b><i>l10n.inc:</i> Oops, SetCookie called after header has been sent on line 98</b>
  <br /> 
  <tt> setcookie(&quot;ui_language&quot;, $active_language, 2147483647<b>
    <blink>
     ); /* last valid 32 bit time_t */ 
    </blink></b></tt>

Upvotes: 2

Views: 1786

Answers (1)

Guilherme Coelho Carvalho
Guilherme Coelho Carvalho

Reputation: 53

Alright, just worked it out!

Apparently the server was not expecting a Data Field with "username","ubnt","password","ubnt" but it was expecting raw Post Data. Changing the content-type to:

post.setHeader("Content-Type","multipart/form-data;  boundary=---------------------------21240622191493050652355892969");

And changing the Entity to 

StringEntity postParamsString = new StringEntity("-----------------------------21240622191493050652355892969\r\nContent-Disposition: form-data; name=\"uri\"\r\n\r\n/stalist.cgi\r\n-----------------------------21240622191493050652355892969\r\nContent-Disposition: form-data; name=\"username\"\r\n\r\nubnt\r\n-----------------------------21240622191493050652355892969\r\nContent-Disposition: form-data; name=\"password\"\r\n\r\nubnt\r\n-----------------------------21240622191493050652355892969--\r\n");

I managed to get a successful login. Also changed to Apache Commons because, as Peter pointed out, jSoup doesn't handle multipart/form-data very well but I think it would have worked with jSoup as well.

Upvotes: 2

Related Questions