CODEWITHSUNDEEP
pythonpython-3.xseleniumselenium-webdriverweb-scraping
Pratik Gokhale
Pratik Gokhale

Reputation: 35

Unable to scroll webpage using .scrollTo() and .scrollIntoView() in selenium (python) any other way to do it?

I am trying to scroll down a webpage "https://www.novitecgroup.com/en/brands/ferrari/roma/" using selenium for python. So far I have tried using following but none seem to work. Please suggest me a better way make it work.

My End goal is to scroll down the webpage and get the browser to automatically click one of those car parts under Suspension and also Aerodynamics.

{NOTE: x in following code is a place holder for a specific part I am trying to scroll to and then click}

using scroll To function

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
driver.execute_script("arguments[0].click();", x)

Using arguments.scroll into view function

driver.execute_script("arguments[0].scrollIntoView();", x)
driver.execute_script("arguments[0].click();", x)

Thank you for you help in advance

Upvotes: 1

Views: 1052

Answers (1)

undetected Selenium
undetected Selenium

Reputation: 193098

To scroll down the webpage and get the browser to automatically click() on Suspension or Aerodynamics you need to induce WebDriverWait for the element_to_be_clickable() which automatically scrolls the element into the view and you can use either of the following Locator Strategies:

  • Clicking on Suspension using XPATH:

    driver.get("https://www.novitecgroup.com/en/brands/ferrari/roma/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='cookie-confirm__submit']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[text()='Suspension']"))).click()

  • Clicking on Suspension using CSS_SELECTOR:

    driver.get("https://www.novitecgroup.com/en/brands/ferrari/roma/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#cookie-confirm__submit"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href$='aerodynamic']"))).click()

  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Upvotes: 1

Related Questions