CODEWITHSUNDEEP
pythonseleniumweb-scrapingautomation
karanrp
karanrp

Reputation: 230

Click all links of table using Selenium Python

In my project, I am downloading all the reports by clicking each link written as a "Date". Below is the image of the table. Table to be scraped

I have to extract a report of each date mentioned in the table column "Payment Date". Each date is a link for a report. So, I am clicking all the dates one-by-one to get the report downloaded.

for dt in driver.find_elements_by_xpath('//*[@id="tr-undefined"]/td[1]/span'):
    dt.click()
    time.sleep(random.randint(5, 10))

So, the process here going is when I click one date it will download a report of that date. Then, I will click next date to get a report of that date. So, I made a for loop to loop through all the links and get a report of all the dates.

But it is giving me Stale element exception. After clicking 1st date it is not able to click the next date. I am getting error and code stops.

How can I solve this?

Upvotes: 0

Views: 2007

Answers (2)

Yaakov Bressler
Yaakov Bressler

Reputation: 12018

You're getting a stale element exception because the DOM is updating elements in your selection on each click.

An example: on-click, a tag "clicked" is appended to an element's class. Since the list you've selected contains elements which have changed (1st element has a new class), it throws an error.

A quick and dirty solution is to re-perform your query after each iteration. This is especially helpful if the list of values grows or shrinks with clicks.

# Create an anonymous function to re-use
# This function can contain any selector
get_elements = lambda: driver.find_elements_by_xpath('//*[@id="tr-undefined"]/td[1]/span')

i = 0
while True:
    elements = get_elements()

    # Exit if you're finished iterating
    if not elements or i>len(elements):
        break
    
    # This should always work
    element[i].click()

    # sleep
    time.sleep(random.randint(5, 10))

    # Update your counter
    i+=1

Upvotes: 1

Prophet
Prophet

Reputation: 33351

The simplest way to solve it is to get a specific link each time before clicking on it.

links = driver.find_elements_by_xpath('//*[@id="tr-undefined"]/td[1]/span')
for i in range(len(links)):
    element = driver.find_elements_by_xpath('(//*[@id="tr-undefined"]/td[1]/span)[i+1]')
    element.click()
    time.sleep(random.randint(5, 10))

Upvotes: 0

Related Questions