CODEWITHSUNDEEP
javaconnectionjsoup
Peck3277
Peck3277

Reputation: 1423

Jsoup connections and retries

I'm using JSoup to connect to a website. I sometimes find that JSoupwill have a connection timeout, when this happens I want JSoup to retry the connection and when it fails on the 3rd time it shall add a string to an array list.

My code at the moment is:

try {
        Document doc = Jsoup.connect(sitemapPath)
                .userAgent("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.21 (KHTML, like Gecko) Chrome/19.0.1042.0 Safari/535.21")
                .timeout(10000)
                .get();

        Elements element = doc.select("loc");
        return element;
    } catch (IOException e) {
        return null;
    }

I was thinking of doing something with while loops but I have to return the element so I'm unsure how to do this.

Upvotes: 2

Views: 6091

Answers (2)

talawahtech
talawahtech

Reputation: 630

This is a slightly more concise version based on Rodri_gore's answer. It uses a for loop instead of a while and takes advantage of the fact that "doc" will only be null if the loop is exited normally after three timeouts.

This is the case because I am only catching the SocketTimeoutException. If I attempted to catch other exceptions in the same try/catch block then the logic would no longer hold. "doc" could be null because a different exception occurred.

By the way, you can use http://www.google.com:81/ to test timeout errors.

  Document doc = null;

  for (int i = 1; i <= 3; i++) {
      try{
          doc = Jsoup.connect(url).get();
          break; // Break immediately if successful
      }
      catch (SocketTimeoutException e){
          // Swallow exception and try again
          logger.warn("jsoup Timeout occurred " + i + " time(s)");
      }                 
  }

  if (doc == null){
      // Timed out 3 times. Do whatever you want here.
  }
  else{
      // Do something with doc
  }

I usually create a utility method with this code and then call this method passing either the url or a connection object. I then wrap that method call in a try catch to handle all the other exceptions that jsoup might throw.

Upvotes: 2

Rodrigo Gauzmanf
Rodrigo Gauzmanf

Reputation: 2527

    ArrayList<String> l = new ArrayList();
    Document doc = null;
    int i = 0;
    boolean success = false;

    while( i < 3){
        try {
            doc = Jsoup.connect(sitemapPath).get();
            success = true;
            break;
        } catch (SocketTimeoutException ex){
            l.add("text...");               
        }
        catch (IOException e) {
        }           
        i++;
    }

    if(success){
        // Selector code ...
        Elements element = doc.select("loc");
    }

Upvotes: 7

Related Questions