CODEWITHSUNDEEP
pythonseleniumpaginationwebdriver
Alfie Grace
Alfie Grace

Reputation: 252

How do I click the next button in pagination, where the next button doesn't have a class?

I'm having some trouble with scraping all of the content off of a website using selenium web driver. The problem I'm having, is that there is no class for the next button on the pagination, here is the html for the pagination section:

<div>
     <ul class="pagination">
         <li class="disabled>
              <a href="#">
                  <span>« </span>
                  First
              </a>
         </li>
         <li class="disabled">
              <a href="#">
                  <span>‹ </span>
                  Previous
              </a>
         </li>
         <li class="disabled">
              <a href="#">
                  1
              </a>
         </li>
         <li class="">
              <a href="#" rel="next">
                  2
              </a>
         </li>
         <li class="">
             <a href="#">
                 3
             </a>
         </li>
         #........ same again for pages 4 and 5
         <li class="">
             <a href="#" rel="next">
                 Next
                 <span> ›</span>
             </a>
         </li>
         <li class="">
             <a href="#">
                 Last
                 <span> »</span>
             </a>
         </li>

Here's the code I've been using to click an element with the 'pagination' class:

while True:

# get data from page here

# click next page
    try:
        element = wait.until(EC.element_to_be_clickable((By.CLASS_NAME, 'pagination')))
        element.click()
    except TimeoutException:
        # no pages left
        break

The problem I've been having with this is that it only identifies the first clickable button in the pagination, meaning that it alternates between clicking page 2 (which enables the 'First' button), then clicking the first button and switching back to page 1. Is there a way I can narrow down the clickable elements further so that it only clicks the next button?

Upvotes: 2

Views: 952

Answers (1)

vitaliis
vitaliis

Reputation: 4212

Try to wait for next button:

element = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.pagination a[rel=next]'))

Upvotes: 2

Related Questions