CODEWITHSUNDEEP
pythonseleniumweb-scraping
Matthew Doig
Matthew Doig

Reputation: 25

Why can't I locate this element?

I am planning on web scraping GPU stock data from NVIDIA however when trying to load the full page by clicking on the load more button it says "selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".buy-link load-more-btn"}"

HTML enter image description here The code

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import  expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException
import time

PATH = Service('C:\Program Files (x86)\chromedriver.exe')
driver = webdriver.Chrome(service=PATH)

driver.get("https://store.nvidia.com/en-gb/geforce/store/gpu/?page=1&limit=9&locale=en-gb&category=GPU&gpu=RTX%203090,RTX%203080%20Ti,RTX%203080,RTX%203070%20Ti,RTX%203070,RTX%203060%20Ti,RTX%203060,RTX%203050%20Ti,RTX%203050&gpu_filter=RTX%203090~30,RTX%203080%20Ti~19,RTX%203080~24,RTX%203070%20Ti~18,RTX%203070~22,RTX%203060%20Ti~19,RTX%203060~28,RTX%203050%20Ti~0,RTX%203050~0,RTX%202080%20Ti~1,RTX%202080%20SUPER~0,RTX%202080~0,RTX%202070%20SUPER~0,RTX%202070~0,RTX%202060%20SUPER~1,RTX%202060~8,GTX%201660%20Ti~3,GTX%201660%20SUPER~5,GTX%201660~5,GTX%201650%20SUPER~2,GTX%201650~19")

load = driver.find_element(By.CLASS_NAME, "buy-link load-more-btn")

load.click()

Upvotes: 0

Views: 28

Answers (1)

Guy Nachshon
Guy Nachshon

Reputation: 2655

The call to find the element is executed before it is present on the page. Take a look at the wait timeout documentation. Here is an example from the docs

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
try:
    element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "myDynamicElement"))
    )
finally:
    driver.quit()

you could also use implicitly_wait, but with this solution, you have to guess how long it will take to load

driver.implicitly_wait(20) # gives an implicit wait for 20 seconds

Upvotes: 1

Related Questions