CODEWITHSUNDEEP
pythonselenium
abhanan93
abhanan93

Reputation: 249

Selenium: How to click on all DIVs with same class name

I've been working on scraping a site which has usually 10 to 12 DIVs with the same class name. When clicked on any one of these DIVs, it goes to a new page. I want to scrape data from that page and then navigate back to previous page and then click on the next div and so on.

But I can't figure out that when I come back to the previous page how can I click on the next div?

Any help would be appreciated.

matches = browser.find_elements_by_class_name('ipo-CompetitionBase ')
index = 0
while index <= len(matches):
    matches[index].click()
    browser.back()
    index += 1

Upvotes: 2

Views: 1720

Answers (1)

alecxe
alecxe

Reputation: 473903

The problem with that is that you cannot simply find all the divs and click them one by one because when you get back to the previous page the elements you've previously found already became "stale" and you need to "re-find" them.

I'd maintain an index of a current div element I'm clicking and, every time I'd get back to the main page, I would click the div element at the index + 1 position. Continue that until you've exhausted all the div elements on the main page.

More like a pseudo-code:

index = 0
while True:
   divs = browser.find_elements_by_class_name('className')

   try: 
       divs[index].click()
   except IndexError:
       break  # no more elements, exit the loop

   # do smth
   # ...

   browser.back()

   index += 1

Upvotes: 1

Related Questions