CODEWITHSUNDEEP
javascalahttpurlconnection
Dan Wiebe
Dan Wiebe

Reputation: 549

URL.openConnection() returns already-open connection the second time

I'm working on a website-scraping application in Scala. The site I'm scraping is heavily session-oriented, so I have to hit the site once to get a session ID before I can do anything else.

I get the connection for retrieving the session ID like this:

url.openConnection().asInstanceOf[HttpURLConnection]

It works fine. The .connected field of the returned HttpURLConnection is false, and it flips to true when I call .connect() on it. No problem.

The first hint of trouble occurs when I finish with the connection and call .disconnect() on it. The .connected field stays true. Hm.

So now I've got my session ID, and I go to get the page that has the form I want on it. I call

url.openConnection().asInstanceOf[HttpURLConnection]

again, just like last time--same code, in fact--except this time the HttpURLConnection it gives me has the .connected field set to true! I thought at first that somehow it was giving me the same object it gave me before, but no, the memory ID is different.

So of course now when I call .setRequestProperty() on the connection, it blows up with an IllegalStateException: Already connected.

Am I misunderstanding how to use HttpURLConnection?

Notes: Scala 2.9.2, Java 6.0. Also, the URL objects on which I call .openConnection() are different objects, not the same.

Thanks...

Upvotes: 4

Views: 3661

Answers (3)

MadProgrammer
MadProgrammer

Reputation: 347334

The URL class is not well suited for session based work (especially cookie based sessions) in my experience.

If you want to take advantage of this, I'd suggest using something like Apache HTTPClient

IMHO

Upvotes: 1

user207421
user207421

Reputation: 311039

It's called connection pooling, in quest of HTTP Keep-alive. It's good. You want it. If you really don't, call the disconnect() method.

Upvotes: 1

matt b
matt b

Reputation: 140041

Sounds like HttpUrlConnection is keeping the connection alive for you under the covers.

Take a look at this article for some tips to force it to close the connection and not be overly helpful.

Although in your case it sounds like you might want to use keep-alive, as it could speed up your calls to the website by avoiding unnecessary connection handshakes.

Upvotes: 0

Related Questions