CODEWITHSUNDEEP
xcodexctestxcode-ui-testing
Alex
Alex

Reputation: 3971

Rerun failed unit test - Xcode XCTest

When I run my XCTests, I'd like to automatically rerun, once, any integration (unit/ui) test that fails. Is this possible?

This would be done in the same test, without having to hit 'run' again on my tests, or any part of my tests. I'm running all my tests from the command line.

I am doing UI tests that use the network to make calls to the server. If there is a significant server problem, the test will/should fail and report an error. However, if it is only a temporary problem with the request, it would be nice to rerun the test automatically and see if it passes. Also, with the current state of Xcode UI testing there are some occasional problems where it will crash for an obscure reason, and it would be nice to rerun the test automatically to see if it passes the second time.

It would be especially nice if it could output what happened, i.e. "The test failed the first time, because of failure getting refreshed snapshot, but passed the second time"

Upvotes: 14

Views: 5273

Answers (3)

Declan McKenna
Declan McKenna

Reputation: 4870

You can now do this in Xcode without any scripts.

  1. Press ⌘ + 6 or select the test assistant view.
  2. Filter for failed tests
  3. Select tests, right click and choose Run x test methods > Run in all classes
  4. (optional) If you want to run the same tests again press ⌃ + ⌥ + ⌘ + G for a quick way to run them right away.

Xcode image 1

enter image description here

Upvotes: 9

ablarg
ablarg

Reputation: 2490

Task list to accomplish this without fastlane and with granular test reruns

  • create a script to rerun the test and capture the list of failed tests, then rerun the tests using the -only-testing:TEST-IDENTIFIER flag of xcodebuild.
  • parse the resultBundle or .xcresult to find the failed tests
  • write code to erase the device or simulator executing the tests
  • reboot the device if it is a real one to clear any possible dialogs
  • use XCAttachment to collect things you care about, like app logs
  • use xchtmlreport to generate a nice webpage from the results bundle with your snapshots and attachments.

Upvotes: 4

user2206324
user2206324

Reputation: 356

Fastlane can make it possible here is the awesome blog post regarding same

Stabilizing the CI By Re-runing Flaky iOS XCUI Tests

Upvotes: 2

Related Questions