CODEWITHSUNDEEP
phpunit-testingphpunit
rap-2-h
rap-2-h

Reputation: 31948

How to make PHPUnit fail on risky tests

I want PHPUnit to fail if one or more test is considered as risky. Actually:

PHPUnit 5.3.4 by Sebastian Bergmann and contributors.

..RRR..                                                         7 / 7 (100%)

Time: 2.83 seconds, Memory: 26.00Mb

OK, but incomplete, skipped, or risky tests!
Tests: 7, Assertions: 137, Risky: 3.

It says "OK, but incomplete", so my tests did not fail (and can be shipped in case of continous delivery). Is there any way to have a "fail" status? I want my test global status considered as failed on risky test, don't know if it's possible.

Upvotes: 5

Views: 1965

Answers (2)

Nic Wortel
Nic Wortel

Reputation: 11423

You can use the --fail-on-risky flag when calling the PHPUnit executable, or set the failOnRisky="true" attribute to the <phpunit> element in phpunit.xml.

In contrary to --stop-on-risky/stopOnRisky="true" this will not stop the test suite when PHPUnit encounters a risky test, but it will make PHPUnit exit with a non-zero status code, as it would when one of the tests fails.

Upvotes: 11

gontrollez
gontrollez

Reputation: 6538

There is a --stop-on-risky option that can be enabled from the command line or configured in the phpunit.xml.

The problem with this is that the run process wont execute all the tests. I'm not aware of a direct way of considering risky tests as failed.

The behaviour you want to obtain was proposed but not accepted, so don't expect to be implemented:

https://github.com/sebastianbergmann/phpunit/issues/1612

Upvotes: 6

Related Questions