CODEWITHSUNDEEP
phpunit-testingphpunit
Mike Graf
Mike Graf

Reputation: 5317

What does PHPUnit Strict mode do?

I am wondering about what "strict mode is in PHPUnit" ?

eg:

phpunit --strict

or in phpunit.xml

<phpunit strict="true"/>

I turned it on just to try it and my tests started failing with

PHP_Invoker_TimeoutException: Execution aborted after 1 second

Upvotes: 37

Views: 9124

Answers (2)

Mike Graf
Mike Graf

Reputation: 5317

Short answer: for long running tests use an annotation to increase the allowed run time:

@large // 10 seconds
@medium // 5 seconds
@small // 1 second max <-- Default, so no point using

Long Answer:

Here is an updated set of info that was derived with the help of @Crozin.

In my case the error was that a test was taking too long (>1 second.) (Doctrine ORM schema drop + create can slow things down, see this ZendCast for what I was doing). This was causing an issue (and some output) from PHP_Invoker. Strict mode doesnt allow any output.

By Reading / Reverse engineering /usr/share/php/pear/share/pear/PHPUnit/Util/Test.php::getSize() (and getGroups() on the same class) .. I figured out there are 3 undocumented annotations we can use:

@large  // 10 seconds
@medium // 5 seconds
@small // 1 second max run time

They can be specified on the class level or on the method level. Issue #490 on the PHPUnit github hints at issues with supplying both class level and method level so YMMV if you mix them. As crozin said, the allotted time outs are 10,5,1 seconds respectively.

A alternate solution was to increase how long an invoked function is allowed to run (on my slow computer).

sudo vi /usr/share/php/pear/share/pear/PHP/Invoker.php

Increase line 1 "declare(ticks = 1);" to
    "declare(ticks = 10);" // or any higher int that meets your needs

Here is a bunch of information about strict mode that helped me find the solution:

PHP_Invoker
A utility class for invoking callables with a timeout. This package is required to enforce test timeouts in strict mode. [PHPUnit Install Instructions]

Strict Mode Tests that do not assert anything are marked as incomplete Test that are incomplete (or skipped) yield no code coverage Slideshare by Sebastian Bergmann (slide 10)

Note
Please note that PHPUnit swallows all output that is emitted during the execution of a test. In strict mode, a test that emits output will fail. Testing output section of PHPUnit Manual

Upvotes: 22

Crozin
Crozin

Reputation: 44396

Please note that PHPUnit swallows all output that is emitted during the execution of a test. In strict mode, a test that emits output will fail.

That's all I could find in documentation, but I'd also checked the sources and I found that in strict mode:

  1. Tests with no assertions might be marked as incomplete/failures.

  2. Each test might be run with execution time limit depending on it's size and the presence of the pcntl extension and PHP_Invoker library. There are three timeouts values:

    • timeoutForSmallTests (default value: 1)
    • timeoutForMediumTests (default value: 10)
    • timeoutForLargeTests (default value: 60)

    The test size (small, medium or large) is determined by the PHPUnit_Util_Test::getSize() method:

    • Test cases descending from PHPUnit_Extensions_Database_TestCase or PHPUnit_Extensions_SeleniumTestCase are large.
    • Test cases and individual tests can also be made large or medium by adding them to the 'large' or 'medum' groups, respectively.
    • Otherwise, the test is small.

It's seems that strict mode does only the above three changes, but I'm not absolutely sure about that. I have never studied PHPUnit's sources before, nor used strict mode.

Upvotes: 35

Related Questions