Selenium and Geckodriver issue with creating a webdriver in Python

Question

I have a piece of code in a python crawler that used to work. I installed it on a new system, and am trying to get the right dependencies now. When using geckodriver 0.13.0 and executing the following code:

        def login(self):
            print self.colors.OKBLUE + "Logging into my site as User: " + self.config.email + self.colors.ENDC
            username = self.driver.find_element_by_css_selector('.my_user_field')
            for c in self.config.email:
                    print "Sending key: " + c
                    username.send_keys(c + "")

I get the following error:

Sending key: b
Traceback (most recent call last):
  File "main.py", line 20, in <module>
    crawler.start()
  File "/home/tyrick/dev/pycrawlers/sc/src/main/python/new.py", line 39, in start
    self.login()
  File "/home/tyrick/dev/pycrawlers/sc/src/main/python/new.py", line 147, in login
username.send_keys(c)
   File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 349, in send_keys
    'value': keys_to_typing(value)})
   File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 493, in _execute
    return self._parent.execute(command, params)
   File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Expected [object Undefined] undefined to be a string

I read in a few places that geckodriver has a bug with this, and I should use 0.16.0. So I've tried that as well as 0.17.0, but am now getting the following error:

Traceback (most recent call last):
  File "main.py", line 18, in <module>
    crawler = New()
  File "/home/tyrick/dev/pycrawlers/sc/src/main/python/new.py", line 28, in __init__
    self.driver = webdriver.Firefox()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 152, in __init__
keep_alive=True)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 98, in __init__
    self.start_session(desired_capabilities, browser_profile)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 188, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: elementScrollBehavior was not a the name of a known capability or a valid extension capability

It's as if I now can't even initialize the driver. I am using Selenium 3.4.3, which from what I read is fine.

If anyone can guide me towards a solution, I’d appreciate it much! Thanks

Answer 1

I don't have enough reputation to add a comment, but in reference to @iamdanchiv's answer, what does "lack of safety when it comes to finding the username field. You should use an explicit wait!" Do you mean that there is no safety in terms of bringing up an error?

Answer 2

Here is the Answer to your Question:

• As per the Question heading I understand the issue is Starting Mozilla Firefox through GeckoDriver.
• As you have mentioned your Selenium version as 3.4.3, you can consider using any of the following either geckodriver v0.16.0 or v0.16.1 or v0.17.0
• As per your latest error stack trace the main issue is configuring the WebDriver instance.

• It is to be noted that the current Selenium-Python binding is unstable with geckodriver and looks to be Architecture specific. You can find the github discussion and merge here. So you may additionally need to pass the absolute path of the firefox binary as firefox_binary argument while initializing the webdriver

• Here is the working set of code block using Selenium version 3.4.3, geckodriver v0.17.0 & Mozilla Firefox 53.0 through Python 3.6.1 which opens the URL http://www.python.org:

  from selenium import webdriver
  from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
  
  if __name__ == '__main__':
  
      binary = FirefoxBinary('C:\\Program Files\\Mozilla Firefox\\firefox.exe')
      driver = webdriver.Firefox(firefox_binary=binary,executable_path="C:\\Utility\\BrowserDrivers\\geckodriver.exe")
      driver.get("http://www.python.org")
  

Let me know if this Answers your Question.

Answer 3

This actually has nothing to do with your code, this is a bug with the latest release of Firefox check this issue thread on the Geckodriver repository, you will have to :

• Downgrade to Firefox 52.0 .
• Wait for the next release of Geckodriver.
• Download and compile Geckodriver manually.
• Use another driver like Chrome driver

The best option i believe is the first one, you will also have to disable automatic updates once you downgrade to prevent Firefox from going back to the latest version (Settings->Advanced->Update->Never check for updates)

Answer 4

You are right, you have two different issues.

Issue with geckodriver 0.13.0:

This is most likely because your c is undefined.

You have to verify/assert that self.config.email actually returns a valid string (email). So do a check that c contains your expected email before issuing the .send_keys() command.

Another enhancement worth noting here is your lack of safety when it comes to finding the username field. You should use an explicit wait !

# Library imports
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# (...)

def login(self):
    print self.colors.OKBLUE + "Logging into my site as User: " + 
    self.config.email + self.colors.ENDC

    # Polls the DOM for 3 seconds trying to find '.my_user_field'
    username = WebDriverWait(self.driver, 3).until(EC.presence_of_element_located((By.CSS_SELECTOR, '.my_user_field')))

    for c in self.config.email:

        # Validate 'c' is of type string
        if (str(type(c)).find('str') != -1):  
            print "Sending key: " + c
            username.send_keys(c + "")
        else:
            print "'c' is not what it used to be!"

Lastly, add the full snippet because it looks like you're cycling through a list of emails and sending them in the previously found username field.

Issue with geckodriver 0.16.0:

This is failing because you have an issue with your driver instantiation: self.driver = webdriver.Firefox().

Please update the question with the complete snippet of your driver declaration (including capabilities & profiles, if any). Else, it's really hard to figure out the cause of the error.