CODEWITHSUNDEEP
javajsoup
Peck3277
Peck3277

Reputation: 1423

Jsoup malformed url

I'm having trouble with connecting to a url with JSoup.

The url I am trying to test is www.xbox.com/en-US/security which is a 302(I think) redirect to http://www.xbox.com/en-US/Live/Account-Security. I have set up jsoup to follow redirect and get the new url using .headers("location"). The url returned is /en-US/Live/Account-Security. I'm not sure how to handle it, my code is below:

while (i < retries){
        try {
            response = Jsoup.connect(checkUrl)
                    .userAgent("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.21 (KHTML, like Gecko) Chrome/19.0.1042.0 Safari/535.21")
                    .followRedirects(false)
                    .timeout(10000)
                    .execute();
            success = true;
            break;
        } catch (SocketTimeoutException ex){
            timeout = true;
        }  catch (MalformedURLException ep){
            malformedUrl = true;
        }catch (IOException e) {
            statusCode = 404;
        }
    }

private void getStatus(){
    if (success){
        statusCode = response.statusCode();
        success = false;
    }
    if (statusCode >= 300 && statusCode <= 399){
        //System.out.println("redirect: " +statusCode + " " +checkUrl);
        checkUrl = response.header("location");
        //System.out.println(checkUrl);
        connect();
        getStatus();
    }
}

Has anyone got suggestions on how to handle this? Or should I do a check on my checkUrl = response.header("location"); to see if it is a valid url and if not don't test it?

Upvotes: 3

Views: 7592

Answers (1)

Igor Brusamolin Lobo Santos
Igor Brusamolin Lobo Santos

Reputation: 1205

First things first: If you try to access "www.xbox.com/en-US/security", it'll throw you a MalformedException and thus not redirect you to where you want.

Than there's the issue that I'd use only the boolean variable success, and set it as false if any exception is caught. Then again I don't know if you're using timeout, or malformed variables for anything.

After that I'd say that the line right after IOException is never useful. I again couldn't tell, since I can't see the full code.

Now... To your question: The returned string is a domain within the first URL you provided. It'll go simply like this:

//Assuming you won't ever change it, make it a final
//variable for less memory usage.
final String URL = "http://www.xbox.com/en-US/security";

//Whatever piece of processing here

//Some tests just to make sure you'll get what you're
//fetching:
String newUrl = ""
if (checkUrl.startsWith("/"))
    newUrl = URL + checkUrl;

if (checkUrl.startsWith("http://"))
    newUrl = checkUrl;

if (checkUrl.startsWith("www"))
    newUrl = "http://" + checkUrl;

This piece of code will basically make sure you can navigate through urls, without getting some MalformedUrlException. I'd suggest putting a manageUrl() method somewhere and test if the fetched URL is within the domain you're searching, or ele you might end up in e-commerces or publicuty websites.

Hope it helps =)

Upvotes: 6

Related Questions