Laravel/Pest not generating code coverage, worker crashing in parallel after Xdebug enabled

Question

Been banging my head against this one for a while and could use some help. I'm working to get xdebug enabled in our codebase at work to facilitate code coverage. I was able to get it installed and you can see that here:

I'll also add the xdebug.ini so that you can see configuration:

phpunit.xml configuration file:

If I scope the tests that run to the unit folder by running ./vendor/bin/pest --suite unit the tests run without a problem and a report is generated:

However, when I try to run the entire test suite in parallel I get the following output and I am just not sure what to do with it:

Some other notes:

• When Xdebug is disabled I can run parallel without a problem.
• When I run the entire test suite in non parallel mode it does run a lot of tests but it doesn't give me the summary/breakdown at the end. Which makes me believe it might be failing silently. Could it be Xdebug compatibility issue (something I was wondering)?
• I also noticed that when I run the test suite not in parallel mode the PHPUnit report does not update. More evidence that something is wrong with our test suite when Xdebug is enabled.
• I tried adding a tearDown with the following code based on an article I found to hopefully catch any memory leak issues, but no change.
• I can run a specific test not in parallel and the test run does complete.
• The failures in parallel are caused by the worker crashing.
• I do have some tests failing since I enabled Xdebug but I don't think that would cause the worker to crash (not sure).
• We do have tests that are in the old PHPUnit format. I went through and commented all of those out, but no change.

Update: I did find this in my laravel.log

[2024-07-30 14:52:30] testing.ERROR: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away {"userId":9,"exception":"[object] (PDOException(code: HY000): SQLSTATE[HY000]: General error: 2006 MySQL server has gone away at /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Concerns/ManagesTransactions.php:298)
[stacktrace]
#0 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Concerns/ManagesTransactions.php(298): PDO->exec('ROLLBACK TO SAV...')
#1 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Concerns/ManagesTransactions.php(267): Illuminate\\Database\\Connection->performRollBack(1)
#2 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Concerns/ManagesTransactions.php(102): Illuminate\\Database\\Connection->rollBack()
#3 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Concerns/ManagesTransactions.php(37): Illuminate\\Database\\Connection->handleTransactionException(Object(Illuminate\\Database\\QueryException), 1, 1)
#4 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/DatabaseManager.php(456): Illuminate\\Database\\Connection->transaction(Object(Closure))
#5 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(357): Illuminate\\Database\\DatabaseManager->__call('transaction', Array)
#6 /var/www/html/app/Core/Accounts/Actions/CreateAccount.php(30): Illuminate\\Support\\Facades\\Facade::__callStatic('transaction', Array)
#7 /var/www/html/vendor/lorisleiva/laravel-actions/src/Concerns/AsObject.php(24): App\\Core\\Accounts\\Actions\\CreateAccount->handle(Array)
#8 /var/www/html/app/Interfaces/Api/v1/Commercial/Controllers/CompanyController.php(66): App\\Core\\Accounts\\Actions\\CreateAccount::run(Array)
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/helpers.php(233): App\\Interfaces\\Api\\v1\\Commercial\\Controllers\\CompanyController->App\\Interfaces\\Api\\v1\\Commercial\\Controllers\\{closure}()
#10 /var/www/html/app/Interfaces/Api/v1/Commercial/Controllers/CompanyController.php(60): value(Object(Closure))
#11 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): App\\Interfaces\\Api\\v1\\Commercial\\Controllers\\CompanyController->store(Object(App\\Interfaces\\Api\\v1\\Commercial\\Requests\\CompanyStoreRequest))
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(43): Illuminate\\Routing\\Controller->callAction('store', Array)
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(260): Illuminate\\Routing\\ControllerDispatcher->dispatch(Object(Illuminate\\Routing\\Route), Object(App\\Interfaces\\Api\\v1\\Commercial\\Controllers\\CompanyController), 'store')
#14 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(206): Illuminate\\Routing\\Route->runController()
#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(806): Illuminate\\Routing\\Route->run()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#17 /var/www/html/app/Interfaces/Api/Middleware/APIVersion.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): App\\Interfaces\\Api\\Middleware\\APIVersion->handle(Object(Illuminate\\Http\\Request), Object(Closure), 'v1')
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(50): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Routing\\Middleware\\SubstituteBindings->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(161): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(127): Illuminate\\Routing\\Middleware\\ThrottleRequests->handleRequest(Object(Illuminate\\Http\\Request), Object(Closure), Array)
#23 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(89): Illuminate\\Routing\\Middleware\\ThrottleRequests->handleRequestUsingNamedLimiter(Object(Illuminate\\Http\\Request), Object(Closure), 'api', Object(Closure))
#24 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Routing\\Middleware\\ThrottleRequests->handle(Object(Illuminate\\Http\\Request), Object(Closure), 'api')
#25 /var/www/html/vendor/laravel/framework/src/Illuminate/Auth/Middleware/Authenticate.php(64): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Auth\\Middleware\\Authenticate->handle(Object(Illuminate\\Http\\Request), Object(Closure), 'api')
#27 /var/www/html/app/Interfaces/Api/Middleware/ForceJsonResponse.php(17): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#28 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): App\\Interfaces\\Api\\Middleware\\ForceJsonResponse->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#29 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#30 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(805): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))
#31 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(784): Illuminate\\Routing\\Router->runRouteWithinStack(Object(Illuminate\\Routing\\Route), Object(Illuminate\\Http\\Request))
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(748): Illuminate\\Routing\\Router->runRoute(Object(Illuminate\\Http\\Request), Object(Illuminate\\Routing\\Route))
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(737): Illuminate\\Routing\\Router->dispatchToRoute(Object(Illuminate\\Http\\Request))
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(200): Illuminate\\Routing\\Router->dispatch(Object(Illuminate\\Http\\Request))
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}(Object(Illuminate\\Http\\Request))
#36 /var/www/html/app/Interfaces/Api/Middleware/StripPhoneToDigits.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#37 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): App\\Interfaces\\Api\\Middleware\\StripPhoneToDigits->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#38 /var/www/html/app/Interfaces/Api/Middleware/AttachApiVersionHeaderToResponse.php(25): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#39 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): App\\Interfaces\\Api\\Middleware\\AttachApiVersionHeaderToResponse->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#40 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#41 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#42 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ConvertEmptyStringsToNull->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#43 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#44 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(51): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#45 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#46 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#47 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Http\\Middleware\\ValidatePostSize->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#48 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(110): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#49 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#50 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php(62): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#51 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Http\\Middleware\\HandleCors->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#52 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php(57): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#53 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Http\\Middleware\\TrustProxies->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#54 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#55 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(175): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))
#56 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(144): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter(Object(Illuminate\\Http\\Request))
#57 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php(585): Illuminate\\Foundation\\Http\\Kernel->handle(Object(Illuminate\\Http\\Request))
#58 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php(375): Illuminate\\Foundation\\Testing\\TestCase->call('POST', 'http://localhos...', Array, Array, Array, Array)
#59 /var/www/html/tests/System/Commercial/Http/Controllers/Company/CompanyStoreTest.php(75): Illuminate\\Foundation\\Testing\\TestCase->post('http://localhos...', Array)
#60 /var/www/html/vendor/pestphp/pest/src/Factories/TestCaseMethodFactory.php(110): P\\Tests\\System\\Commercial\\Http\\Controllers\\Company\\CompanyStoreTest->{closure}(Object(App\\Users\\Models\\User))
#61 [internal function]: P\\Tests\\System\\Commercial\\Http\\Controllers\\Company\\CompanyStoreTest->Pest\\Factories\\{closure}(Object(App\\Users\\Models\\User))
#62 /var/www/html/vendor/pestphp/pest/src/Concerns/Testable.php(337): call_user_func_array(Object(Closure), Array)
#63 /var/www/html/vendor/pestphp/pest/src/Support/ExceptionTrace.php(26): P\\Tests\\System\\Commercial\\Http\\Controllers\\Company\\CompanyStoreTest->Pest\\Concerns\\{closure}()
#64 /var/www/html/vendor/pestphp/pest/src/Concerns/Testable.php(337): Pest\\Support\\ExceptionTrace::ensure(Object(Closure))
#65 /var/www/html/vendor/pestphp/pest/src/Concerns/Testable.php(254): P\\Tests\\System\\Commercial\\Http\\Controllers\\Company\\CompanyStoreTest->__callClosure(Object(Closure), Array)
#66 /var/www/html/vendor/pestphp/pest/src/Factories/TestCaseFactory.php(196) : eval()'d code(60): P\\Tests\\System\\Commercial\\Http\\Controllers\\Company\\CompanyStoreTest->__runTest(Object(Closure), Object(Closure))
#67 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestCase.php(1122): P\\Tests\\System\\Commercial\\Http\\Controllers\\Company\\CompanyStoreTest->__pest_evaluable_it_can_create_an_account_for_an_admin_company(Object(Closure))
#68 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestCase.php(654): PHPUnit\\Framework\\TestCase->runTest()
#69 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestRunner.php(105): PHPUnit\\Framework\\TestCase->runBare()
#70 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestCase.php(488): PHPUnit\\Framework\\TestRunner->run(Object(P\\Tests\\System\\Commercial\\Http\\Controllers\\Company\\CompanyStoreTest))
#71 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php(349): PHPUnit\\Framework\\TestCase->run()
#72 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php(349): PHPUnit\\Framework\\TestSuite->run()
#73 /var/www/html/vendor/brianium/paratest/src/WrapperRunner/ApplicationForWrapperWorker.php(119): PHPUnit\\Framework\\TestSuite->run()
#74 /var/www/html/vendor/pestphp/pest/bin/worker.php(100): ParaTest\\WrapperRunner\\ApplicationForWrapperWorker->runTest('/var/www/html/t...')
#75 /var/www/html/vendor/pestphp/pest/bin/worker.php(105): {closure}()
#76 {main}
"}

Update: MySQL and Laravel Version

After further investigation I am more confidant that there is a memory leak. I ran docker stats php on my php docker container and if I run my test suite with Xdebug enabled my container memory eventually maxes out. Once the memory maxes out the container crashes and restarts. All tests after that point start to fail and the test suite eventually stops running. If I run the test suite without Xdebug the memory does continue to go up but it stays below the 7.6G limit (it goes up as high as 3G before finishing the run, well below the 7.6G limit).

Answer 1

I wasn't able to resolve this but I worked around it by going the pcov route and removing Xdebug.

Answer 2

With the detailed information provided and the error picture given, a couple of things we would consider (in order of check/apply) for the test suite and test runner configuration:

1. parent::tearDown() call should always be the last statement in a tearDown() method body. ¹˒²
2. Run the tests bare-metal before containerizing build jobs, which includes tests in our book. ³
3. Disable Xdebug (remote) debugging, you only need code coverage, then completely remove Xdebug and use Pcov for code coverage if there are still memory problems.
4. Do not run tests in parallel if memory is still a problem.
5. Organize your tests in test-suites and identify and isolate the memory intensive tests to verify if those were the culprit by running the non-memory intensive suite(s) in parallel (again)

Best guess chance for a lucky punch is 3.), if you know how to do it because it could confirm within brewing another can of coffee.

If 1. already solves it and it was only one place in the code base, please let us know about that, too.

The rest is only the hard work, therefore always remember: If you can't break it, you never knew it was for real.

¹ This is especially noteworthy as the PHP stack trace in your question shows you are making use of the Laravel Eloquent test-helpers (use RefreshDatabase; etc.) in the test that fails, but the advice is also in general.

² The foreach looks like an earlier debugging leftover to us, but we may be wrong.

³ Doing development bare-metal may or may not be in this current project of yours; if not, ignore it for now, first get operational again.