CODEWITHSUNDEEP
pythonseleniumweb-scraping
Cave Man
Cave Man

Reputation: 76

Looping through rows of a table while clicking links in selenium (python)

The sample page source looks like this

<div class='div1'>
  <table class="foot-market">
    <tbody data-live="false">
      <td class='today-name'/>
    </tbody>
    <tbody data-live="false">
      <td class='today-name'/>
    </tbody>
    <tbody data-live="false">
      <td class='today-name'/>
    </tbody>
  </table

  <table class="foot-market">
    <tbody data-live="false">
      <td class='today-name'/>
    </tbody>
    <tbody data-live="false">
      <td class='today-name'/>
    </tbody>
    <tbody data-live="false">
      <td class='today-name'/>
    </tbody>
  </table
</div>

Explanation Hi, So let's get to it. As shown above the code snippet I'm interacting with is housed in a <div class='div1'>. The <td class='today-name'> is clickable and once clicked it renders a page(the page of interest)

so I would like to loop through getting each <tbody data-live="false"> and click it.

From my research I haven't found anything similar but I have found interesting stuff but nothing to help. I appreciate all help given.

Thanks.

my code

import time
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.firefox.options import Options
from BeautifulSoup import BeautifulSoup

#some of the imports are for headless firefox so don't mind them
#for.testing.purposes.only
driver = webdriver.Firefox()

url = 'here://code.above.com/'
driver.get(url)

#looping rows in selenium
table = driver.find_elements_by_xpath("//table[@class='foot-market']")

for row in table.find_element_by_xpath("//tbody[@data-live='false']"):
    row.find_element_by_xpath("//td[@class='today-name']").click()

    #try for one row first
    # driver.back()
    break

Error

The error returned is in line:

for row in table.find_element_by_xpath("//td[@data-live='false']"):

Exception:

AttributeError: 'list' object has no attribute 'find_element_by_xpath'

which makes sense and all, but how do I achieve this...

Upvotes: 0

Views: 4362

Answers (1)

undetected Selenium
undetected Selenium

Reputation: 193348

The error you are seeing says it all :

for row in table.find_element_by_xpath("//td[@data-live='false']"): AttributeError: 'list' object has no attribute 'find_element_by_xpath'

The following line is error prone :

for row in table.find_element_by_xpath("//td[@data-live='false']"):

As per your previous line :

table = driver.find_elements_by_xpath("//table[@class='foot-market']")

table is a List and you can't invoke find_element_by_xpath() on a List. find_element_by_xpath() can be invoked either on webdriver instance or on a webelement.

Your working code will be as follows :

all_tds = driver.find_elements_by_xpath("//table[@class='foot-market']//td[@class='today-name']")
for tds in all_tds :
    tds.click()
    #try for one row first and break out
    break

Note : This is a basic solution to overcome your error AttributeError: 'list' object has no attribute 'find_element_by_xpath' and to try for one row first

Update

If you intend to loop through all the td elements of the table, then you have to capture the base url so we can revert back to the main page to find and click the subsequent elements as follows :

base_url = driver.current_url
count_all_tds = len(driver.find_elements_by_xpath("//table[@class='foot-market']//td[@class='today-name']"))
for td in range(count_all_tds):
    driver.find_elements_by_xpath("//table[@class='foot-market']//td[@class='today-name']")[td].click()
    # code block for other required actions and finally redirect to base_url
    driver.get(base_url)

Upvotes: 1

Related Questions