CODEWITHSUNDEEP
pythonselenium
kevin
kevin

Reputation: 2014

Python Selenium keep clicking next button

I am trying to crawl multiple pages using Python Selenium from https://www.walmart.com/ip/Clorox-Disinfecting-Wipes-On-The-Go-Citrus-Blend-Scent-34-Wipes/29701960?page=seeAllReviews.

There are buttons at the bottom of the web page. HTML looks like below:

<ul class="paginator-list">
<li><button aria-label="Page 1 of 6 selected" class="active">1</button></li>
<li><button aria-label="Page 2 of 6 " class="">2</button></li>
<li><button aria-label="Page 3 of 6 " class="">3</button></li>
<li><button aria-label="Page 4 of 6 " class="">4</button></li>
<li><button aria-label="Page 5 of 6 " class="">5</button></li>
<li class="paginator-list-gap"></li>
<li><button aria-label="Page 3141 of 6 " class="">3141</button></li>
</ul>

How do I click on second button (Page 2 of 6) using Selenium? How do I keep clicking on next button as page changes. Any suggestions?

Upvotes: 0

Views: 569

Answers (2)

undetected Selenium
undetected Selenium

Reputation: 193108

As per the HTML you have shared and your comment to click on 2nd button (Page 2 of 6) you have to scroll the pagination webelement within the Viewport and then invoke click() as follows :

from selenium import webdriver

driver=webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("https://www.walmart.com/ip/Clorox-Disinfecting-Wipes-On-The-Go-Citrus-Blend-Scent-34-Wipes/29701960?page=seeAllReviews")
pagination_element = driver.find_element_by_xpath("//div[@class='ReviewsFooter-pagination arrange arrange-spaced']")
driver.execute_script("return arguments[0].scrollIntoView(true);", pagination_element)
driver.find_element_by_xpath("//div[@class='ReviewsFooter-pagination arrange arrange-spaced']//ul[@class='paginator-list']/li/button[@aria-label='Page 2 of 6 ']").click()
print("Clicked on Page 2")
driver.quit()

Console Output :

Clicked on Page 2

Upvotes: 1

Ankur Singh
Ankur Singh

Reputation: 1289

You can use this xpath to click on nxt button link every time ://button[@class="active"]/ancestor::li/following-sibling::li[1]

Upvotes: 1

Related Questions