CODEWITHSUNDEEP
pythonselenium
Mori shah
Mori shah

Reputation: 3

Scrolling to an element in Selenium

I am using selenium to scrape a dynamic website. My problem is that I can't scroll down to the items I am looking for. First, I made a list of those items I want to get information from (python_button1). I have to click on each item in this list to see a new window, in which I want to scrape them. Then I made a loop to click on those items. After that, I close the window I opened. Unfortunately, I can't scroll down to each item to click them. I used the following code but it scrolls down to the last item and skips other items. I would really appreciate if you can help me. Thanks! 

python_button1 = driver.find_elements_by_class_name('simboloEvento')

for x in python_button1:
    x.click()
    time.sleep(2)
    driver.find_element_by_class_name('cerrarBoton').click() 
    driver.execute_script("coordinates=arguments[0]. 
    getBoundingClientRect();scrollTo( coordinates.x,coordinates.y);", x)
    time.sleep(2)


innerHTML = driver.execute_script("return document.body.innerHTML")
print(innerHTML)

the scroll down list is a JavaScript Object. I have to click on each of those objects to activate the javascript function ( a new window). I can activate those objects via my code, but the problem is when at least one of those items are not in the current screen. That's why I should scroll down to that object ( here x) first to be able to click on it. But I don't know how to scroll down to that object. I have used the code I used in this post as well as this one : driver.execute_script("arguments[0].scrollIntoView();", x) but none of them worked for me!

Upvotes: 0

Views: 563

Answers (2)

TomH
TomH

Reputation: 2729

Have you tried using the selenium actions class?

builder = ActionChains(driver)
builder.move_to_element(x).perform()

Upvotes: 0

dmb
dmb

Reputation: 298

For this you need two things. In first place know where the element is, and you can do it with python-js like this:

script = 'return arguments[0].getBoundingClientRect().top + document.documentElement.scrollTop;'
posY = driver.execute_script(script, element).split('.')[0]

The above combo will return a string that represent the relative position of a element to the top of the document(website)

Then you can scroll to the position you want

driver.execute_script('window.scrollTo(0, '+ posY +');')

This one will move the screen where the element is, some element needs to be waited for, so I would suggest some waiter function of this nature:

def waiter(token):
    # token is whatever you spect to be in the element like str("/") in a date field
    max_time = 60 #segs
    driver.switch_to.default_content()

    while token not in driver.find_element_by_xpath('//*[@Class="important"]'):
        time.sleep(1)
        max_time += -1
        if max_time == 0:
            raise Exception('Waited too long')

Made myself a small modules with js navigation, because even though selenium is powerfull sometimes it doesn't want to do it's thing on some websites.

EDIT: For completeness this snippet makes the element visible:

driver.execute_script(“arguments[0].style.visibility = 'visible';”, element)

This works because arguments[0] represents the element argument in execute_script

Upvotes: 1

Related Questions