Selenium tests fail when run by Jenkins, but succeed when run by command line

Question

I have two virtual machines running Windows Server 2012. One is running Jenkins, and another that is running one of Jenkins' build jobs. The job checks out test code from subversion and then runs mvn test -Dtest=com.package.name.TestName in a windows batch script. The test compiles and runs, but fails with an ElementNotVisibleException. These are TestNG tests with Selenium and a Firefox driver.

If I take the exact same command and run it from the Windows command line, the test succeeds. Another thing I notice is that when run from the command line, the browser window appears and I can watch it run through the test, but when run by Jenkins no window appears.

I found this question that suggests this is because our Jenkins is running as a service, and the tests are written to require a browser window in the foreground. Altering the tests to use a headless driver isn't an option for us, as that will result in inaccurate test results. Is there a way to get these tests working with Jenkins?

Answer 1

I'd recommend using the Selenium Grid. Its explicit function is to serve several different tests at the same time.

Your jenkins would fire off tests against the grid which would point all the tests towards your nodes.

Answer 2

We faced a similar type of problem while running our selenium test using jenkins on firefox. Jenkins slave was installed as service and firefox was giving troubles. Our solution was to use service by machine administrator creadentials. You can do it by opening services.msc and right click on jenkins service -> properties -> log on. select this account and set administrator credentials. Restart the service and you are good to go.

Answer 3

The best option will be to stop running Jenkins as a windows service. Open a command window and start Jenkins manually. In my case it worked.