CODEWITHSUNDEEP
pythongitlabrobotframeworkgitlab-cigitlab-ci-runner
thekucays
thekucays

Reputation: 618

Why Pipeline Failed when there's test case failed

i'm running robotframework in Gitlab-CI.

the problem i'm facing is, if there's any test case failed on the run, it will failed the pipeline too. so how to prevent the pipeline to failed? because the test failed, not the entire build process.

for now, this is how i run the robotframework on gitlab-ci.yml

- robot --exitonfailure -i "android-test" --outputdir ./output Android/Androidtest.robot


for example, i have 3 test cases on Androidtest.robot Test Suite:
1. register
2. fillin_profile
3. checkout_order

if register case and fillin_profile passed, but checkout order failed, the Ci pipeline will failed. i don't want that to be failed, because on the next job is to send the robotframework test report to gdrive, and it will never sent if the pipeline failed.

is that because i add --exitonfailure parameter btw? how to solve this?

Upvotes: 0

Views: 3114

Answers (2)

rasjani
rasjani

Reputation: 7971

Replace --exitonfailure with --nostatusrc.

If there are test failures, robot will exit with exitcode other than 0. Gitlab and for that matter every ci ever, looks if any command it executes with exitcode other than 0 and thinks that there was a failure. With --nostatusrc robot will always exit with 0 and thus your ci doesnt think there where failures.

Do consider that if you go with suppressing exit codes, you either loose the ability to mark the job within CI as failed if there was test failures unless you provide some other mechanism to do that if you happen to need such feature.

Upvotes: 2

Michel Feldheim
Michel Feldheim

Reputation: 18250

The whole point of CI is to fail when tests fail. Uploading your test results or reports shouldn’t be an extra job in the pipeline. I don’t know about robotframework but GitLab supports publishing of artifacts after failed tests.

https://docs.gitlab.com/ee/ci/junit_test_reports.html

Upvotes: 1

Related Questions