CODEWITHSUNDEEP
pythonseleniumfor-loopselenium-webdriverweb-scraping
Mathematics physics Biology
Mathematics physics Biology

Reputation: 1

TypeError: 'WebElement' object is not iterable

from selenium.webdriver.common.keys import Keys
import pandas as pd
from selenium.webdriver.common.by import By
from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.investing.com/crypto/currencies")
elem = driver.find_element(By.TAG_NAME,'table')

head = elem.find_element(By.TAG_NAME,'thead')
body = elem.find_element(By.TAG_NAME,'tbody')

list_rows = []

for items in body.find_element(By.TAG_NAME,'tr'):
    list_cells = []
    for item in items.find_element(By.TAG_NAME,'td'):
        list_cells.append(item.text)
    list_rows.append(list_cells)
driver.close()

Output for items in body.find_element(By.TAG_NAME,'tr'): TypeError: 'WebElement' object is not iterable

I want to scrape a table from website by selenium and pandas. But I have get some error in my for loop. please any expert solve this problem. please give me a write code that I can use to scrape data from table in any webpage.

MY error is down blow for items in body.find_element(By.TAG_NAME,'tr'): TypeError: 'WebElement' object is not iterable

Upvotes: 0

Views: 2033

Answers (1)

Prophet
Prophet

Reputation: 33361

find_element returns a single web element object while you need to get a list of web element objects. find_elements does this.
So, you need to change form find_element to find_elements so it will be:

for items in body.find_elements(By.TAG_NAME,'tr'):
    list_cells = []
    for item in items.find_elements(By.TAG_NAME,'td'):
        list_cells.append(item.text)
    list_rows.append(list_cells)

Upvotes: 3

Related Questions