CODEWITHSUNDEEP
pythonseleniumbeautifulsoupphantomjs
ballade4op52
ballade4op52

Reputation: 2267

How can Selenium (or BeautifulSoup) be used to access these hidden elements?

Here is an example page with pagination controlling dynamically loaded results.

http://www.rehabs.com/local/jacksonville-fl/

All that I presently know to try is:

curButton = 1
driver.find_element_by_css_selector('ul[class="pagination"]').find_elements_by_tag_name('li')[curButton].click()

Nothing seems to happen (also when trying to access and click the a tag or driver.get() the href of the a element).

Is there another way to access the hidden elements? For instance, when reading the html of the entire page, the elements of different pagination are shown, but are apparently inaccessible with BeautifulSoup.

Upvotes: 0

Views: 310

Answers (2)

JRazor
JRazor

Reputation: 2817

Pagination was added for humans. Maybe you used the wrong xpath or css. Check it.

Use this xpath: 

//div[@id="listing-basic"]/article/div[@class="h3"]/a/@href

Upvotes: 1

Anurag
Anurag

Reputation: 59

You can click on the pagination button using:

driver.find_elements_by_css_selector('.pagination li a')[1].click()

Upvotes: 1

Related Questions