CODEWITHSUNDEEP
javaweb-scrapingjsoup
Abdullah Zaki
Abdullah Zaki

Reputation: 28

jsoup not scraping all elements?

jsoup

When scraping data from website that has 200 elements , the Output is only the first 49 or 50 elements of the 200 elements , why ? - how can i solve this prob. to get all 200 elements data ?``

Document d = Jsoup.connect("https://www.ebay.com/sch/i.html?_from=R40&_trksid=p2047675.m570.l1313.TR0.TRC0.H0.Xsilver.TRS0&_nkw=silver&_sacat=0").timeout(6000).get();
Elements ele = d.select("div#ResultSetItems");

for (Element element : ele.select("li.shic")) {
    String img_url = element.select("li").attr("listingid");
    System.out.println(img_url);
}

Upvotes: 0

Views: 374

Answers (1)

Kayaman
Kayaman

Reputation: 73548

The other elements are fetched on demand by Javascript as is common these days, so they're invisible to JSoup. There is no way to have JSoup perform those fetches, so you're going to have to come up with a better way than scraping to get that data. I suggest you look at API options that EBay offers.

Upvotes: 1

Related Questions