CODEWITHSUNDEEP
seleniumselenium-webdriverselenium-chromedriver
double_wizz
double_wizz

Reputation: 79

How to load all elements of a page using selenium?

I know this has been answered before, like here. But none of this seems to work in my case. I am trying to load all the elements of a particular class using chromedriver in selenium. The driver only gets 60 for each page and moves on to the next one in the loop I have added to the script. The url bellow doesn't send to an infinite loading page (like social networks) but I think it dynamically loads more elements as I scroll down. If I open developer tools and look for the targeted XPATH, all the elements get found. Is there a way to replicate that?

Targeted url

Upvotes: 0

Views: 1031

Answers (1)

Max Daroshchanka
Max Daroshchanka

Reputation: 2978

For your case, you should scroll not window, but the scrollable element.

driver.execute_script("var el = document.getElementsByClassName('infinite-scroller')[0]; el.scroll(0, el.scrollHeight)")

And you should put this in a loop and wait for more elements loaded, e. g. until total target elements count will stop increasing.

Upvotes: 0

Related Questions