CODEWITHSUNDEEP
jenkinsrobotframework
Brandon Olson
Brandon Olson

Reputation: 639

Locally-run tests pass, but Jenkins tests fail; why, and how can I fix this?

I'm running a fairly large suite of python-based tests with a much larger number of steps on an Ubuntu Linux VM. When I run them via any number of methods manually (via the console) they all run and pass just fine.

After I ported them to a Jenkins server, four out of the thirty fail. I tried the usually recommended fix - increasing the wait time for keywords to work to 1s before every single click - so I'm fairly certain it isn't a timing issue. The site loads a lot faster than that on Windows, which I know is slower than Jenkins on Linux.

After Googling around a little for an answer, I found that apparently no one has come up with an accepted answer, either on this site or other Q/A sites.

Here's the error messages I'm getting from Jenkins.

ElementNotVisibleException: Message: element not visible
  (Session info: chrome=61.0.3163.79
  (Driver info: chromedriver=2.26.436382 (70eb799289ce4c2208441fc057053a5b07ceabac),platform=Linux 4.10.0-33-generic x86_64)

WebDriverException: Message: unknown error: Cannot read property 'innerHTML' of undefined
  (Session info: chrome=61.0.3163.79
  (Driver info: chromedriver=2.26.436382 (70eb799289ce4c2208441fc057053a5b07ceabac),platform=Linux 4.10.0-33-generic x86_64)

The other two are both element not visible exceptions identical to the first, both of which happen on a Click Button keyword that is not the first Click Button keyword in the test suite. The first one happens on a Click Element keyword that has worked perfectly since I wrote it, and the last one happens on tried-and-true JavaScript call to get the text of an element.

Why would something work locally on two different operating systems and then fail on Jenkins?

Upvotes: 6

Views: 42098

Answers (1)

Bryan Oakley
Bryan Oakley

Reputation: 386275

Why would something work locally on two different operating systems and then fail on Jenkins?

  • The most common might be that the jenkins system is running slower, and your tests aren't being hyper-vigilant about waiting for pages to finish loading before trying to interact with it. Jenkins boxes often can be under a heavy load, and if both the client and server are running on the same box, either or both could be contributing to the problem.

  • Another reason could be that you're running different versions of the browsers and/or selenium drivers on the jenkins box.

  • Another reason could be that the resolution of the (virtual?) displays is different, causing elements to be shifted to a different position.

  • The browsers on the jenkins box could have different profiles, resulting in a different set of plugins or antivirus software running. These can contribute to the speed at which a page renders, or could cause unwanted popups that cover portions of the screen.

Upvotes: 7

Related Questions