CODEWITHSUNDEEP
pythonseleniumselenium-webdriverscrollselenium-chromedriver
Noob Gamer
Noob Gamer

Reputation: 55

How to “scroll down” some part using selenium in python?

Hope you are good I m trying to make an simple script but I got stuck on there I am trying to scroll the list to get more but i am unable to scroll down. Anybody have an idea how is that done.

here is my code:

import selenium
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from shutil import which
import time
import pandas as pd
import json
# from fake_useragent import UserAgent
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
chrome_path = which('chromedriver')
driver = webdriver.Chrome(executable_path=chrome_path)
driver.maximize_window()

driver.get('http://mapaescolar.murciaeduca.es/mapaescolar/#')

driver.find_element_by_xpath('//ul[@class="nav property-back-nav floating-box pull-right"]//button').click()
time.sleep(3)
driver.find_element_by_xpath('//button[@ng-click="openCloseFilters()"]').click()
time.sleep(3)
driver.find_element_by_xpath('//select[@title="Enseñanza"]/option[1]').click()



element = driver.find_element_by_xpath('//div[@id="container1"]')
driver.execute_script("return arguments[0].scrollIntoView(true);", element)

And the list I want to scroll down :

This is list i want to scroll

Upvotes: 3

Views: 18330

Answers (7)

Carlost
Carlost

Reputation: 837

Try this one:

driver.execute_script("window.scrollTo(0, 300);")

Upvotes: 0

Oliviero
Oliviero

Reputation: 9

element.location_once_scrolled_into_view

I used this when trying to access an element that wasnt't because needed to be scrolled down.

Upvotes: 0

Mori
Mori

Reputation: 4641

In your case that there is a list of items, so you can follow this method :

for c in range(1, 12): # check the list start from 0 or 1 
    time.sleep(5)  # Give time to loading the information
    element = driver.find_element_by_xpath(f'//*[@id="grid-search-results"]/ul/li[{c}]') # variable c refer to next item
    driver.execute_script("arguments[0].scrollIntoView();", element)

Upvotes: 1

Trinh Phat
Trinh Phat

Reputation: 564

Because the scroll is actually inside an element, all the javascript command with window. will not work. Also the element is not interactable so Key down is not suitable too. I suggest using scrollTo javascript executor and set up a variable which will increase through your loop:

element = driver.find_element_by_xpath('//div[@id="container1"]')
time.sleep(10)

verical_ordinate = 100
for i in range(0, 50):
   print(verical_ordinate)
   driver.execute_script("arguments[0].scrollTop = arguments[1]", element, verical_ordinate)
   verical_ordinate += 100
   time.sleep(1)

I have tested with chrome so it should work.

Reference

https://developer.mozilla.org/en-US/docs/Web/API/Element/scrollTop

Upvotes: 6

BrunoT
BrunoT

Reputation: 442

Simply change the script to arguments[0].scrollTop = arguments[0].scrollHeight. You can call that script in some loop with a timeout to continuously fetch more data (that is continuously scrolling the div to bottom)

Example:

while True:
    time.sleep(1)
    driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", element)
    if no_new_data_available():
        break

Upvotes: 1

Architrixs
Architrixs

Reputation: 38

# Use this line in a loop, accordingly how much screen to be scrolled down
# this just scrolls down to the height of browser screen, hence loop.

driver.execute_script("window.scrollTo(0,document.body.scrollHeight)")

# Now if the website needs time to load its data after scroll, add this in the loop..

time.sleep(5)

Upvotes: 0

Adil kasbaoui
Adil kasbaoui

Reputation: 663

Try this :

    SCROLL_PAUSE_TIME = 0.5

# Get scroll height
last_height = driver.execute_script("return document.body.scrollHeight")

while True:
    # Scroll down to bottom
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

    # Wait to load page
    time.sleep(SCROLL_PAUSE_TIME)

    # Calculate new scroll height and compare with last scroll height
    new_height = driver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
        break
    last_height = new_height

Or simply select your element in tis case the div containing the scroll view and type this :

label.sendKeys(Keys.PAGE_DOWN);

Upvotes: 1

Related Questions