CODEWITHSUNDEEP
pythonseleniumweb-scraping
Omega
Omega

Reputation: 881

Selenium Python for loop only gets first item

My for loop only seems to only find the first item from https://public.tableau.com/en-gb/gallery/?tab=viz-of-the-day&type=viz-of-the-day instead of looping through all dates and titles. If I print(viz) I can see different elements, but that doesn't seem to be carried through.

driver.get("https://public.tableau.com/en-gb/gallery/?tab=viz-of-the-day&type=viz-of-the-day")
wait = WebDriverWait(driver, 10)

time.sleep(10)

vizzes = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".gallery-items-list div.gallery-list-item-container")))
for viz in vizzes:
    print(viz)

    #publish date
    date_id = driver.find_element_by_css_selector('[data-test-id="published-date"]').text
    print(date_id)

    #name of the viz
    viz_name = driver.find_element_by_xpath("//a[contains(@href, '/en-gb/gallery/')]").text
    print(viz_name)

For xpath I tried using

viz_name = driver.find_element_by_xpath(".//a[contains(@href, '/en-gb/gallery/')]").text

and

viz_name = driver.find_element_by_xpath("//*[contains(@href, '/en-gb/gallery/')]").text

which yielded the same result.

Upvotes: 1

Views: 626

Answers (2)

SIM
SIM

Reputation: 22440

Although it appears that you have already defined explicit wait, you used hardcoded delay which you can get rid of to make the script robust. Here is another way of doing the same using css selector:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://public.tableau.com/en-gb/gallery/?tab=viz-of-the-day&type=viz-of-the-day"

with webdriver.Chrome() as driver:
    wait = WebDriverWait(driver, 10)
    driver.get(link)
    for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "[class$='item-container']"))):
        viz_name = item.find_element_by_css_selector("[class$='item-title-left'] > a").text
        date_id = item.find_element_by_css_selector("[data-test-id='published-date']").text
        print(viz_name,date_id)

Upvotes: 1

Ioanna
Ioanna

Reputation: 376

You need to correct with viz.find_element.. instead of driver.find_element..:

#publish date
date_id = viz.find_element_by_css_selector('[data-test-id="published-date"]').text
print(date_id)

#name of the viz
viz_name = viz.find_element_by_xpath("//a[contains(@href, '/en-gb/gallery/')]").text
print(viz_name)

Upvotes: 3

Related Questions