select_by_visible_text python selenium ElementNotVisibleException

Question

I am trying to programmatically select a currency in the site: http://www.asos.com/asos/asos-skinny-chinos-in-dark-khaki/prd/5542109

However it's doing some problems to me, I am using this following piece of code:

def set_currency(text):
    one = WebDriverWait(driver1, 10).until(EC.visibility_of_element_located((By.CLASS_NAME, "selected-currency")))
    one.click()
    select_element = WebDriverWait(driver1, 10).until(EC.visibility_of_element_located((By.ID, "currencyList")))
    select = Select(select_element)
    select.select_by_visible_text(text)

Sometimes it gives me this error: selenium.common.exceptions.ElementNotVisibleException: in this line select.select_by_visible_text(text) but sometimes it works just fine. I am using WebDriverWait until it is visible so I can't understand why it s doing this errors.

Full error if needed:

Traceback (most recent call last):
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 35, in <module>
    set_currency(currency)
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 18, in set_currency
    select.select_by_visible_text(text)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\support\select.py", line 120, in select_by_visible_text
    self._setSelected(opt)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\support\select.py", line 212, in _setSelected
    option.click()
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\webelement.py", line 77, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\webelement.py", line 494, in _execute
    return self._parent.execute(command, params)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:17758","User-Agent":"Python-urllib/3.5"},"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1479563250968\", \"sessionId\": \"b2aa4180-ae5e-11e6-b8b3-e1a4ad040bb7\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/b2aa4180-ae5e-11e6-b8b3-e1a4ad040bb7/element/:wdc:1479563250968/click"}}
Screenshot: available via screen

Edit:

CURRENT CODE:

    from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from selenium.common.exceptions import ElementNotVisibleException
import traceback
from selenium.webdriver.support.select import Select

driver1 = webdriver.PhantomJS(r'C:\Users\dodob\Desktop\Apps Workspace\phantomjs-2.1.1-windows\bin\phantomjs.exe')

def set_currency(label):
    is_change_currency_displayed = driver1.find_element_by_id("currencyList").is_displayed()
    if not is_change_currency_displayed:
        print("dropdown is not displayed.")
        one = WebDriverWait(driver1, 10).until(EC.element_to_be_clickable((By.CLASS_NAME, "selected-currency")))
        one.click()
    select_element = WebDriverWait(driver1, 10).until(EC.visibility_of_element_located((By.ID, "currencyList")))
    select = Select(select_element)
    select.select_by_visible_text(label)


def get_all_currencies():
    one = WebDriverWait(driver1, 10).until(EC.visibility_of_element_located((By.CLASS_NAME, "selected-currency")))
    one.click()
    el = WebDriverWait(driver1, 10).until(EC.visibility_of_element_located((By.ID, "currencyList")))
    currency2 = []
    options = el.find_elements_by_tag_name('option')
    for option in options:
        currency2.append(option.text)
    return currency2


def main(url):
    print(url)
    driver1.get(url)
    to_return_string = ''
    list_of_currencies = get_all_currencies()
    print(list_of_currencies)
    for currency in list_of_currencies:
        try:
            set_currency(currency)
            current_price = WebDriverWait(driver1, 10).until(
                EC.visibility_of_element_located((By.CSS_SELECTOR, ".current-price")))
            to_return_string += ("In " + currency + " : " + current_price.text + ' \n')
            print("In", currency, ":", current_price.text)
        except TimeoutException:
            print(traceback.print_exc())
            print("In", currency, ":", "Timed out waiting for page to load")
            to_return_string += ("In " + currency + " : " + " Timed out waiting for page to load" + ' \n')
        except ElementNotVisibleException:
            print(traceback.print_exc())
    return to_return_string


main('http://www.asos.com/it/asos/asos-jeans-skinny-alla-caviglia-kaki/prd/6759361')

CURRENT OUTPUT:

C:\Python\Python35\python.exe C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py
http://www.asos.com/it/asos/asos-jeans-skinny-alla-caviglia-kaki/prd/6759361
['£ GBP', '$ USD', 'C$ CAD', 'kr SEK', 'kr NOK', 'kr DKK', '₣ CHF', ' € EUR', '$ AUD', '¥ RMB', '$ HKD', '$ NZD', '$ SGD', 'NT$ TWD', 'руб.  RUB']
In £ GBP : € 33,99
In $ USD : € 33,99
None
Traceback (most recent call last):
In C$ CAD : Timed out waiting for page to load
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 43, in main
    set_currency(currency)
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 19, in set_currency
    select_element = WebDriverWait(driver1, 10).until(EC.visibility_of_element_located((By.ID, "currencyList")))
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: 

dropdown is not displayed.
In kr SEK : € 33,99
In kr NOK : € 33,99
None
Traceback (most recent call last):
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 43, in main
    set_currency(currency)
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 21, in set_currency
    select.select_by_visible_text(label)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\support\select.py", line 120, in select_by_visible_text
    self._setSelected(opt)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\support\select.py", line 212, in _setSelected
    option.click()
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\webelement.py", line 77, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\webelement.py", line 494, in _execute
    return self._parent.execute(command, params)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:60873","User-Agent":"Python-urllib/3.5"},"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1479841552848\", \"sessionId\": \"aae52750-b0e6-11e6-a0c1-5193111f996c\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/aae52750-b0e6-11e6-a0c1-5193111f996c/element/:wdc:1479841552848/click"}}
Screenshot: available via screen

dropdown is not displayed.
In ₣ CHF : € 33,99
In  € EUR : € 33,99
Traceback (most recent call last):
None
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 43, in main
    set_currency(currency)
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 21, in set_currency
    select.select_by_visible_text(label)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\support\select.py", line 120, in select_by_visible_text
    self._setSelected(opt)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\support\select.py", line 212, in _setSelected
    option.click()
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\webelement.py", line 77, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\webelement.py", line 494, in _execute
    return self._parent.execute(command, params)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:60873","User-Agent":"Python-urllib/3.5"},"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1479841552851\", \"sessionId\": \"aae52750-b0e6-11e6-a0c1-5193111f996c\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/aae52750-b0e6-11e6-a0c1-5193111f996c/element/:wdc:1479841552851/click"}}
Screenshot: available via screen

dropdown is not displayed.
In ¥ RMB : € 33,99
In $ HKD : € 33,99
None
In $ NZD : Timed out waiting for page to load
Traceback (most recent call last):
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 43, in main
    set_currency(currency)
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 19, in set_currency
    select_element = WebDriverWait(driver1, 10).until(EC.visibility_of_element_located((By.ID, "currencyList")))
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: 

dropdown is not displayed.
In $ SGD : € 33,99
In NT$ TWD : € 33,99
None
Traceback (most recent call last):
In руб.  RUB : Timed out waiting for page to load
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 43, in main
    set_currency(currency)
  File "C:/Users/dodob/PycharmProjects/AsosPriceCheckerWindows/currency.py", line 19, in set_currency
    select_element = WebDriverWait(driver1, 10).until(EC.visibility_of_element_located((By.ID, "currencyList")))
  File "C:\Python\Python35\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: 


Process finished with exit code 0

Answer 1

Issue:

Code seems to be fine. But it is the order in which the calls are made are causing the issue.

When the following line is called in main method:

list_of_currencies = get_all_currencies()

you are clicking on selected-currency in the following line of get_all_currencies() method:

one.click() // opens the "Change-Currency" dropdown menu as shown in below image

and no further action taken in the method.

then we call set_currency(currency) in for loop of main method.

in the set_currency(currency) method again we are clicking on selected-currency, which is actually closing the opened dropdown of Change-Currency, which results in Element Not Visible Exception.

Solution:

So, change the flow to make sure that Change-Currency dropdown is opened before you call set_by_visible_text method.

Check whether the Change-Currency dropdown is already opened using isDisplayed method.

Following is the code: (added 2 new lines in the beginning)

def set_currency(label):
    isChangeCurrencyDisplayed  = driver.find_element_by_id("currencyList").is_displayed()
    if not isChangeCurrencyDisplayed:
        print "dropdown is not displayed."
        one = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CLASS_NAME, "selected-currency")))
        one.click()
    select_element = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID, "currencyList")))
    select = Select(select_element)
    select.select_by_visible_text(text) # here text is "$ USD"

Answer 2

You can wait for the wanted select element (currency element) to appear:

select_visible_element = WebDriverWait(driver1, 10).until(EC.visibility_of_element_located((By.XPATH, "//a[contains(text(),'%s')]" %text)))

Alternatively you can click the dropdown, wait for the select_visible_element and click() it instead of select.