In Ruby, with selenium, difficulty clicking Google Search

Question

I get this far, opening the firefox browser, navigating to google, and finding the google search element

irb(main):001:0> require 'selenium-webdriver'
=> true

irb(main):002:0> driver = Selenium::WebDriver.for:firefox
=> #<Selenium::WebDriver::Firefox::Marionette::Driver:0x..fb3c81796cc82b708 browser=:firefox>

irb(main):003:0> driver.navigate().to("http://www.google.com")
=> nil

irb(main):188:0> driver.find_element(:name, "q").send_keys "fff"
=> nil    

irb(main):112:0> driver.find_element(:name, "btnK");
=> #<Selenium::WebDriver::Element:0x5fb450f4379c50ce id="d767311c-27a2-3544-8f11-e4edc9736588">

irb(main):113:0> driver.find_element(:name, "btnK").attribute('value');
=> "Google Search"

But I can't manage to click it!

irb(main):114:0> driver.find_element(:name, "btnK").click
Traceback (most recent call last):
       16: from /usr/local/lib/ruby/gems/2.5.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/w3c/bridge.rb:552:in `execute'
       15: from /usr/local/lib/ruby/gems/2.5.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/bridge.rb:166:in `execute'
       14: from /usr/local/lib/ruby/gems/2.5.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/http/common.rb:62:in `call'
       13: from /usr/local/lib/ruby/gems/2.5.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/http/default.rb:104:in `request'
       12: from /usr/local/lib/ruby/gems/2.5.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/http/common.rb:84:in `create_response'
       11: from /usr/local/lib/ruby/gems/2.5.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/http/common.rb:84:in `new'
       10: from /usr/local/lib/ruby/gems/2.5.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/response.rb:32:in `initialize'
        9: from /usr/local/lib/ruby/gems/2.5.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/response.rb:69:in `assert_ok'
        8: from clickElement@chrome://marionette/content/listener.js:1209:5
        7: from navigate@chrome://marionette/content/listener.js:409:13
        6: from navigate/<@chrome://marionette/content/listener.js:410:13
        5: from clickElement/<@chrome://marionette/content/listener.js:1210:14
        4: from interaction.clickElement@chrome://marionette/content/interaction.js:130:11
        3: from webdriverClickElement@chrome://marionette/content/interaction.js:159:11
        2: from ElementNotInteractableError@chrome://marionette/content/error.js:286:5
        1: from WebDriverError@chrome://marionette/content/error.js:178:5
Selenium::WebDriver::Error::ElementNotInteractableError (Element <input name="btnK" type="submit"> could not be scrolled into view)
irb(main):115:0> 

It says that the element "could not be scrolled into view" !

I can do driver.find_element(:tag_name, "body").send_keys :page_down; which would page down. Which is mentioned Looking at ruby selenium documentation for send_keys https://www.rubydoc.info/gems/selenium-webdriver/Selenium%2FWebDriver%2FElement%3Asend_keys . I see a list of key codes https://www.rubydoc.info/gems/selenium-webdriver/Selenium/WebDriver/Keys#KEYS-constant . And also listed https://github.com/SeleniumHQ/selenium/blob/master/rb/lib/selenium/webdriver/common/keys.rb . Or I can page down manually. But scrolling (at least in the normal sense of the term) doesn't seem to be the issue, I still get that error. Maybe it's in a frame I can switch to and I don't know which.

The button is of course visible. It's just a regular google search page.

I can find the button fine but I can't see how to click it, as .click isn't working for me.

added

in reply to a comment asking me if the google suggestions box is obfuscating the button. I can see in the browser that the button isn't obfuscated 'cos sometimes I have done escape manually, or clicked the background window(to get rid of that. I am using IRB so running each statement manually so I have time to do that). But I also did escape with the code, and escape works to get rid of the suggestion box, but still same error when trying to click the button

Answer 1

But you can do the click with javascript (sending javascript to the browser).

b=driver.find_element(:name, "btnK")

driver.execute_script("arguments[0].click();",b)

Jeff explains why the javascript method(selenium with javascript), works on either button but the non-javascript method(selenium without javascript) works only on one button, he wrote "Both of the buttons would work, the JS click will click on any element no matter where it is... or if it's visible, etc. Selenium was designed to interact with the page as a user would so it was unable to click on the element due to the error."

Answer 2

While discussing this problem in the comments, I wrote some C# code to demonstrate how we wait for clickable in C#

new WebDriverWait(Driver, TimeSpan.FromSeconds(5)).Until(ExpectedConditions.ElementToBeClickable(By.Name("btnK"))).Click();

In the process of writing the C# code I posted, I think I see what the issue is. When you start typing in the search box, a dropdown appears that contains the suggested searches. This covers the "Google Search" button that you are trying to click... BUT another "Google Search" button appears at the bottom of the dropdown itself. You should print a count of the (:name, "btnK") element(s)... I'm assuming it will print 2 (if a proper wait is added). From there, you just need to click the one that is on top.

Another option would be to send a \r\n at the end of your search string and avoid this whole issue... or you could just navigate to the final search URL and save even more headaches and time.

Added note from barlop

To clarify, this has nothing to do with the suggestions popup showing(and having to get rid of the popup).. and nothing to do with having a wait (Since I was working with IRB, I was naturally waiting). And I'd already clicked outside or hit escape to get rid of the popup. The issue was, and Jeff's answer alerted me to this - Even after the popup is gone, there are still two buttons with that btnK name. So, using find_elements rather than find_element, and running .click on the second one works! (Also, using find_elements(plural) for the name attribute makes sense, since as noted in an answer herehttps://stackoverflow.com/questions/5518458/does-a-name-attribute-have-to-be-unique-in-a-html-document. the name attribute is not unique / not a unique identifier.

Answer 3

This is what I told you to use WATIR, If you had used WATIR, this problem wouldn't have arrived. The problem in your code, it's not waiting for visibility. Recently Chrome Driver has added implicit wait for click as well, so it would wait for visibility if you set the implicit wait. Otherwise move to WATIR which is a good wrapper which doesn't wait via driver, In WATIR waiting for element status happens from local language binding, so use this selenium code

require 'selenium-webdriver'
driver = Selenium::WebDriver.for :chrome
driver.manage.timeouts.implicit_wait = 10
driver.navigate().to("http://www.google.com")
driver.find_element(:name, "q").send_keys "fff"
driver.find_element(:name, "btnK").click

Remember this selenium code wouldn't work for firefox because implicit wait for click is not added yet for firefox.

Since WATIR is handling the timing from local language binding, it doesn't matter whether you use Chrome or Firefox, it would perfectly work

WATIR Code (Default is Chrome)

require 'watir'
b=Watir::Browser.new
b.goto 'www.google.com'
b.text_field(name: 'q').set 'fff'
b.button(name: 'btnK').click

If you want to drive Firefox

b=Watir::Browser.new :firefox