CODEWITHSUNDEEP
jenkinsselenium-webdriverautomationautomated-testsrobotframework
Arun Ramachandran
Arun Ramachandran

Reputation: 1372

Test automation with Jenkins + Robot framework

We are using Jenkins for the Continuous Integration.

I have also created a Jenkins job for robot framework test cases. But am facing an issue while running robot framework test cases using Jenkins.

Say, if I have to include around 300 test cases in Jenkins job for robot framework, while running the test cases, if any of the test case fails, then the remaining test scripts will not be get executed.

For example : while running the test cases , if 10th test case fails, then the remaining 290 test cases will not be get executed !

Is there any better option in Jenkins to run all test cases in a concurrent manner ? - So that failure of any test cases will not effect the execution of other test cases !

My only requirement is - All the test cases listed in Jenkins job must be get executed even if any test cases got failed in between.

Can you please suggest a solution for this issue ?

Do I have to include any other plugins for this ?

Here is the screenshot of this particular Jenkins Job - enter image description here

Upvotes: 0

Views: 4148

Answers (2)

aneroid
aneroid

Reputation: 15961

That's not a good way to be running your tests in a batch. For starters, the RobotFramework log would only contain the results of the last test, even if they all passed.

The issue is, when an 'Execute Shell' step in the build fails, Jenkins stops the build. As pointed out in the comments above, you could try changing the return state with exit 0.

However, instead of each test being a separate build step with:

pybot /home/arun/test-cases/228.txt
pybot /home/arun/test-cases/117.txt
...etc

You should be executing them together in one step:

pybot /home/arun/test-cases/

If you need to control which tests execute and not all the valid test files in that folder and its subfolders, then use tags.

Upvotes: 3

planetmaker
planetmaker

Reputation: 6044

https://wiki.jenkins-ci.org/display/JENKINS/Robot+Framework+Plugin lists as one of the necessary configuration steps:

  1. Force your Robot script to return successfully from shell with "exit 0" to empower the plugin in deciding if the build is success/failure (by default Robot exits with error code when there's any failed tests)

Thus if you have your test scripts return 0, then the jenkins build won't fail and you can decide upon the outcome what status you want to display.

  1. Set thresholds and optionally disable thresholds for critical tests only to count every test in the pass percentage.

Upvotes: 1

Related Questions