CODEWITHSUNDEEP
javajsoup
Gil Peretz
Gil Peretz

Reputation: 2419

Jsoup extract group of selectors until reached designated selector

I have the following page:

<div>
 <h3>...</h3> 
 <span>...</span>
 <p>...</p>
 <span...</span>
 <span...</span>
 <span...</span>
 <p>...</p>
 <span...</span>
 <span...</span>
 <hr />

 <h3>...</h3> 
 <span>...</span>
 <p>...</p>
 <p>...</p>
 <hr />

 <h3>...</h3> 
 <span>...</span>
 <span>...</span>
 <p>...</p>
 <p>...</p>
 <hr />
</div>

As you can see most of the selectors are within the same level. I'm trying to figure how can I scrap one block at a time using Jsoup. Block means all selectors that starts with <h3> and ends with <hr> (in the example above there are 3 blocks). The selectors between are in-consist and the amount can be vary.

I read the official API documentation, but couldn't figure a proper way to do so.

Upvotes: 0

Views: 301

Answers (1)

Alkis Kalogeris
Alkis Kalogeris

Reputation: 17755

package stack;

import java.io.File;
import java.util.ArrayList;
import java.util.List;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;


public class Stack {

    public static void main(String args[]) throws Exception {
        File input = new File("test.html");
        Document doc = Jsoup.parse(input, "UTF-8");

        List<Elements> blocks = new ArrayList<>();

        Elements listofh3 = doc.getElementsByTag("h3");
        for(Element h3 : listofh3) {
            Elements block = new Elements();
            block.add(h3);
            Element cursor = h3;
            while(!cursor.tagName().equals("hr")) {
                cursor = cursor.nextElementSibling();
                block.add(cursor);
            }
            blocks.add(block);
        }

        for(Elements block : blocks) {
            System.out.println(block);
            System.out.println("----------------------------");
        }
    }
}

Another solution could be this

package stack;

import java.io.File;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;


public class Stack {

    public static void main(String args[]) throws Exception {
        File input = new File("test.html");
        Document doc = Jsoup.parse(input, "UTF-8");

        Elements listofh3 = doc.getElementsByTag("h3");
        for(Element h3 : listofh3) {
            Element span = doc.createElement("span");
            span.addClass("block");

            Element cursor = h3;
            while(!cursor.tagName().equals("hr")) {
                Element next = cursor.nextElementSibling();
                span.appendChild(cursor);
                cursor = next;
            }
            cursor.remove(); //remove hr
            doc.body().appendChild(span);
        }

        System.out.println(doc);
    }
}

Test input

<div>
 <h3>header 1</h3> 
 <span>span 1</span>
 <p>p 1</p>
 <span>span 11</span>
 <span>span 111</span>
 <span>span 1111</span>
 <p>p 11</p>
 <span>span 11111</span>
 <span>span 111111</span>
 <hr />

 <h3>header 2</h3> 
 <span>span 2</span>
 <p>p 2</p>
 <p>p 22</p>
 <hr />

 <h3>header 3</h3> 
 <span>span 3</span>
 <span>span 33</span>
 <p>p 3</p>
 <p>p 33</p>
 <hr />
</div>

Upvotes: 2

Related Questions