CODEWITHSUNDEEP
pythonseleniumselenium-webdriverweb-scrapingwebdriverwait
Jarod Jacobs
Jarod Jacobs

Reputation: 21

Selenium Python Unable to locate element

I'm trying to gather price information for each product variation from this web page: https://www.safetysign.com/products/7337/ez-pipe-marker

I'm using Selenium and FireFox with Python 3 and Windows 10.

Here is my current code:

driver = webdriver.Firefox()
driver.get('https://www.safetysign.com/products/7337/ez-pipe-marker')
#frame = driver.find_element_by_class_name('product-dual-holder')
# driver.switch_to.frame('skuer5c866ddb91611')
# driver.implicitly_wait(5)
driver.find_element_by_id('skuer5c866ddb91611-size-label-324').click()   
price = driver.find_element_by_class_name("product-pricingnodecontent product-price-content").text.replace('$', '')
products.at[counter, 'safetysign.com Price'] = price
print(price)
print(products['safetysign.com URL'].count()-counter)

So, I'm trying to start by just selecting the first product variation by id (I've also tried class name). But, I get an Unable to locate element error. As suggested in numerous SO posts, I tried to change frames (even though I can't find a frame tag in the html that contains this element). I tried switching to different frames using index, class name, and id of different div elements that I thought might be a frame, but none of this worked. I also tried using waits, but that return the same error.

Any idea what I am missing or doing wrong?

Upvotes: 1

Views: 658

Answers (2)

QHarr
QHarr

Reputation: 84455

They may well be dynamic. Select by label type selector instead and index to click on required item e.g. 0 for the item you mention (first in the list). Also, add a wait condition for labels to be present.

If you want to limit to just those 5 size choices then use the following css selector instead of label :

.sku-contents ul:nth-child(3) label

i.e.

sizes = WebDriverWait(driver,10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".sku-contents ul:nth-child(3) label")))
sizes[0].click()

After selecting size you can grab the price from the price node depending on whether you want the price for a given sample size e.g. 0-99.

To get final price use: 

.product-under-sku-total-label

Code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = 'https://www.safetysign.com/products/7337/ez-pipe-marker'
driver = webdriver.Chrome()
driver.get(url)
labels = WebDriverWait(driver,10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "label")))
labels[0].click()
price0to99 = driver.find_element_by_css_selector('.product-pricingnodecontent').text
priceTotal = driver.find_element_by_css_selector('.product-under-sku-total-label').text
print(priceTotal, price0To99)
# driver.quit()

Upvotes: 0

undetected Selenium
undetected Selenium

Reputation: 193058

To locate the elements you have to induce WebDriverWait for the visibility_of_all_elements_located() and you can create a List and iterate over it to click() each item and you can use the following solution:

  • Code Block:

    from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


driver=webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("https://www.safetysign.com/products/7337/ez-pipe-marker")
for product in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//form[@class='product-page-form']//div[@class='sku-contents']//following::ul[1]/li//label[starts-with(@for, 'skuer') and contains(., 'Pipe')]"))):
    WebDriverWait(driver, 20).until(EC.visibility_of(product)).click()
driver.quit()

Upvotes: 1

Related Questions