Selenium Python: Can't click the button

Question

Hope you are all safe!

My issue is pretty common, and I've seen tons of similar questions, though things like "wait/explicitly wait/wait until clickable etc" didn't work for me. I think I would like to ask for tailored solution for this problem :)

I try to scrape some data from here http://kuap.ru/banks/8012/balances/en (I use Russian language version of the website, but I think source must be pretty the same)

To do this, I have to choose appropriate data from drop-down list and then press button "Compare Data". However, I can't click that button.

The button

My simple code:

driver.find_element_by_css_selector('select[name = CurDate]').click()
select = Select(driver.find_element_by_css_selector('select[name = CurDate]'))
select.select_by_visible_text('01.01.2019')
driver.find_element_by_xpath('//*[@id="BalanceTable"]/thead/tr[1]/th[3]/input').click()

The last row doesn't actually click. I've seen that many times the problem is in the path, and I tried several CSS paths - nothing worked. It looks like the same problem is here: when I try to get .text of the button it returns ' ', whereas for other buttons it returns their actual text.

My main hypothesis is that the problem may be in this mysterious "Shadow content" (See a snapshot) I can't view it in Safari. I tried to access it via "...input//" (though it is a bit stupid, I know), but didn't manage to see it. I've read something about DOM etc... But it didn't really help (or I read badly)

So, any useful hints will be of a very great value! Already spent several hours on this very basic stuff :)

P.S. To make full disclosure - I'm super new to Python, trying to learn about data collection from the beginning.

Answer 1

Try below xpath :

 wait = WebDriverWait(driver, 30)
 wait.until(EC.element_to_be_clickable((By.XPATH, "//input[@class='btn btn-default']"))).click()

Note : please add below imports to your solution

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

Working code:

driver.get('http://kuap.ru/banks/8012/balances/en')
wait = WebDriverWait(driver, 10)
select = Select(driver.find_element_by_xpath("//select[@name='CurDate']"))
select.select_by_visible_text('01.01.2019')
wait.until(EC.element_to_be_clickable((By.XPATH, "//input[@class='btn btn-default']"))).click()

Another solution using java script ::

element=wait.until(EC.element_to_be_clickable((By.XPATH, "//input[@class='btn btn-default']")))
driver.execute_script("arguments[0].click();", element)