CODEWITHSUNDEEP
pythonseleniumweb-scraping
Merix
Merix

Reputation: 63

python scrape ajax content

I'd like to parse website with rates but i couldn't take out data from <td> elements.

I wrote short code to test which gets 1st line with data tabel:

from selenium import webdriver

driver = webdriver.Firefox()
driver.get('https://www.gpw.pl/wskazniki_spolek_full')
table = driver.find_elements_by_xpath("//table[@class='tab03']/tbody/tr")[4].text
print table

driver.quit()

and i'm getting results:

2 PLNFI0800016 141 08OCTAVA 42 786 848 44,07 63,86 2016-12-31 H 0,69 27,80 ---

but i'd like go through all <td> elements in <tr> tag in loop by all tables which has class='tab03'

table = driver.find_elements_by_xpath("//table[@class='tab03']/tbody/tr")

for el in table:
    col_id =  el.find_element_by_tag_name('td')[1].text
    col_kod = el.find_element_by_tag_name('td')[2].text

    print("{}".format(col_id, col_kod))

driver.quit()

but i'm getting error: selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: td

Upvotes: 1

Views: 162

Answers (1)

alecxe
alecxe

Reputation: 473873

There are some header rows that don't have td elements inside them, skip them:

rows = driver.find_elements_by_xpath("//table[@class='tab03']/tbody/tr")

for row in rows[3:]:
    cells = row.find_elements_by_tag_name('td')
    col_id = cells[0].text
    col_kod = cells[1].text

    print("{}".format(col_id, col_kod))

Also note that, to get to the td cells, use the find_elements_by_tag_name() and get the desired elements by index (0-based).

Upvotes: 1

Related Questions