CODEWITHSUNDEEP
javahttpurlconnection
Yevgeny Simkin
Yevgeny Simkin

Reputation: 28349

need help with bizarre java.net.HttpURLConnection behavior

I am attempting to download a jpg using HttpURLConnection and am encountering a very peculiar bug.

Here's the url: http://www.vh1.com/sitewide/promoimages/shows/m/my_antonio/video/supertrailer/seg_1/320x240.jpg

if you open it in a browser you will see the image.

However, when I use HttpURLConnection I don't get the image... What I get is a 301 which, quite strangely, redirects to http://wap.vh1.com

so 

    con.setInstanceFollowRedirects(true);
//additional stream code here to go and get the stuff found in con

proceeds to go ahead and download the text from wap.vh1.com, rather than the jpg that you see in the browser.

I'm guessing that there is some header wackiness that's causing this, but I haven't the faintest idea what the host is expecting to see in order to redirect me to the same place as where it's redirecting the browser (and curl and wget and everything else I can think to point at it).

I'm just about ready to shoot myself, so, if you help me you will be preventing my 6 year old daughter from going fatherless.

Upvotes: 2

Views: 1963

Answers (5)

jutky
jutky

Reputation: 3965

The java.net package doesn't support lot of the needed features out of the box (like automatically saving and sending cookies). Use Apache's httpClient instead

Upvotes: 0

nos
nos

Reputation: 229088

While I can't help you with your specific problem, here's what I would do:

  • Download wireshark, sniff the HTTP request sent by your java application

  • Copy/Paste the request, and run it with telnet (or a tool such as WFetch)

  • Fiddle with the request headers and see if behavior changes.

(I'd suspect the site screens the request based on the User agent header or something similar)

Upvotes: 0

EJB
EJB

Reputation: 2413

It seems like the server interprets your request as coming from a mobile device, possibly based on the User-Agent header. That's why your redirected to the mobile site. Try setting the User-Agent explicitly.

Upvotes: 1

miku
miku

Reputation: 188014

For more flexibility you can utilize the http commons libraries, which have great debugging support for the wire through log4j ...

Also, user agents and more request parameters can be set easily.

For more information, see their tutorial.

Upvotes: 0

ZZ Coder
ZZ Coder

Reputation: 75456

The site redirects you based on user-agent. Add this before you open the connection,

conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.15) Gecko/2009101600 Firefox/3.0.15");

Upvotes: 4

Related Questions