How to run Cucumber headless tests on Jenkins Server (Linux)

Question

How to run Cucumber headless tests on Jenkins Server (Linux)?

What's the proper way to use the 'headless' gem executing Cucumber/Watir-Webdriver tests on Jenkins?

First attempt: I have this in my env.rb:

require 'rubygems'
require 'cucumber'
require 'rest_client'
require 'json'
require 'watir-webdriver'
require 'page-object'
require 'page-object/page_factory'
require 'allure-cucumber'
require 'rspec'
require 'data_magic'
require 'fig_newton'
require 'yaml'
require 'headless'
require 'phantomjs'

ENVT = FigNewton.load('staging.yml')
DATA = DataMagic.load('data.yml')
KBA = YAML.load_file('config/data/kba.yml')

#Actions performed before each scenario

headless = Headless.new
headless.start
browser = Watir::Browser.start

Before do
  @browser = browser
end

at_exit do
  browser.close
  headless.destroy
end

World(PageObject::PageFactory)

And this is the output I get:

wrong number of arguments (0 for 1+) (ArgumentError)
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/watir-webdriver-0.6.11/lib/watir-webdriver/browser.rb:28:in `start'
/var/lib/jenkins/jobs/idme_revelator/workspace/features/support/env.rb:24:in `<top (required)>'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/rb_support/rb_language.rb:95:in `load'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/rb_support/rb_language.rb:95:in `load_code_file'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime/support_code.rb:180:in `load_file'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime/support_code.rb:83:in `block in load_files!'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime/support_code.rb:82:in `each'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime/support_code.rb:82:in `load_files!'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime.rb:184:in `load_step_definitions'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime.rb:42:in `run!'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/cli/main.rb:47:in `execute!'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/bin/cucumber:13:in `<top (required)>'
/var/lib/jenkins/.rbenv/versions/2.1.0/bin/cucumber:23:in `load'
/var/lib/jenkins/.rbenv/versions/2.1.0/bin/cucumber:23:in `<main>'
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Second Attempt: When I make ONE change in the env file on this line:

browser = Watir::Browser.start

to

browser = Watir::Browser.new

Then I get this output, but still error:

Could not find Firefox binary (os=linux). Make sure Firefox is installed or set the path manually with Selenium::WebDriver::Firefox::Binary.path= (Selenium::WebDriver::Error::WebDriverError)
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/firefox/binary.rb:127:in `path'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/firefox/binary.rb:60:in `execute'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/firefox/binary.rb:34:in `start_with'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/firefox/launcher.rb:70:in `start_silent_and_wait'

Third Attempt:

If I change my env.rb file to:

require 'rubygems'
require 'cucumber'
require 'rest_client'
require 'json'
require 'watir-webdriver'
require 'page-object'
require 'page-object/page_factory'
require 'allure-cucumber'
require 'rspec'
require 'data_magic'
require 'fig_newton'
require 'yaml'
require 'headless'
require 'phantomjs'

ENVT = FigNewton.load('staging.yml')
DATA = DataMagic.load('data.yml')
KBA = YAML.load_file('config/data/kba.yml')

#Actions performed before each scenario

headless = Headless.new
headless.start
# browser = Watir::Browser.start

Before do
  @browser = headless
end

at_exit do
  #browser.close
  headless.destroy
end

World(PageObject::PageFactory)

It looks a lot better but still fails:

WARN: Unresolved specs during Gem::Specification.reset:
      mini_portile (~> 0.6.0)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
Using the default profile...
@regression @hlp @hlp_admin
Feature: Test the HLP DocumentPage functionality

  Background:                                    # features/HLP_features/hlp_DocumentPage.feature:4
    * I visit the Hosted Landing Page Admin tool # features/step_definitions/hlp/hlp_admin/hlp_admin_step_defs.rb:1
      Unable to pick a platform for the provided browser. (RuntimeError)
      ./features/step_definitions/hlp/hlp_admin/hlp_admin_step_defs.rb:2:in `/^I visit the Hosted Landing Page Admin tool$/'
      features/HLP_features/hlp_DocumentPage.feature:5:in `* I visit the Hosted Landing Page Admin tool'
    * I login to the Hosted Landing Page Admin   # features/step_definitions/hlp/hlp_admin/hlp_admin_step_defs.rb:5
    * I visit the Add DocumentPage page          # features/step_definitions/hlp/hlp_admin/hlp_admin_document_step_defs.rb:1
      undefined method `window' for #<Headless:0x007f47cb1854b0> (NoMethodError)
      /var/lib/jenkins/jobs/revelator/workspace/features/support/hooks.rb:23:in `After'
      undefined method `driver' for #<Headless:0x007f47cb1854b0> (NoMethodError)
      /var/lib/jenkins/jobs/revelator/workspace/features/support/hooks.rb:16:in `After'

  Scenario: Add a new DocumentPage                 # features/HLP_features/hlp_DocumentPage.feature:9
    * I create a new DocumentPage                  # features/step_definitions/hlp/hlp_admin/hlp_admin_document_step_defs.rb:5
    * I should see DocumentPage edit page elements # features/step_definitions/hlp/hlp_admin/hlp_admin_document_step_defs.rb:10
    * I delete the "DocumentPage"                  # features/step_definitions/hlp/hlp_admin/hlp_admin_edit_shared_step_defs.rb:1

So what's the proper way to use the 'headless' gem executing Cucumber/Watir-Webdriver tests on Jenkins?

UPDATE: How to use Xvfb and Watir?

my env.rb:

require 'rubygems'
require 'cucumber'
require 'json'
require 'watir-webdriver'
require 'page-object'
require 'page-object/page_factory'
require 'allure-cucumber'
require 'rspec'
require 'data_magic'
require 'fig_newton'
require 'yaml'
require 'phantomjs'
require 'headless'

ENVT = FigNewton.load('staging.yml')
DATA = DataMagic.load('data.yml')
KBA = YAML.load_file('config/data/kba.yml')

headless = Headless.new
headless.start
browser = Watir::Browser.new

Before do
  @browser = browser
end

at_exit do
  @browser.close
end

World(PageObject::PageFactory)

Answer 1

The last place I worked we did not use the headless gem, we just started up XVFB in the Jenkins script that ran the tests, before it issued the cucumber command that started the tests. And used Firefox as the browser. Thaat also allowed us to take screenshots on failing tests to aid with debugging the failures.

If I still worked there I would try using phantomjs also and see if it worked, then use whichever was more reliable, and/or faster.

Nothing was different in env.rb, same file we used for local runs we used on jenkins also.

EDIT: Update for late 2017.. with recent changes the current recommended best solution for headless browsing is using chrome along with chromedriver and chrome's new headless option.

Answer 2

The third error is because Headless is not a browser, just a virtual frame buffer. @browser = headless will not work, because it should be a Watir::Browser object. The virtual frame buffer is where your browser, like Firefox or Chrome draws objects.

Based on your list of requires, it looks like you're trying to run with PhantomJS. Watir::Browser.start requires a URL to navigate to as an argument, and will try to start Firefox by default, if you do not pass in :phantomjs, in addition to a url as an argument.

@base_url = "https://my-test-domain.com"
@browser = Watir::Browser.start @base_url, :phantomjs

Watir::Browser.new runs the browser as :firefox by default. If you want to use PhantomJS, pass :phantomjs as an argument.

browser = Watir::Browser.new :phantomjs

If you intend to use PhantomJS as your browser, you likely don't need the Headless gem, and can remove that from your env.rb.