CODEWITHSUNDEEP
iosswiftperformanceperformance-testing
Declan McKenna
Declan McKenna

Reputation: 4870

How to make a performance test fail if it's too slow?

I'd like my test to fail if it runs slower than 0.5 seconds but the average time is merely printed in the console and I cannot find a way to access it. Is there a way to access this data?

Code

//Measures the time it takes to parse the participant codes from the first 100 events in our test data.
func testParticipantCodeParsingPerformance()
{
    var increment = 0
    self.measureBlock
    {
        increment = 0
        while increment < 100
        {
            Parser.parseParticipantCode(self.fields[increment], hostCodes: MasterCalendarArray.getHostCodeArray()[increment])
            increment++
        }
    }
    print("Events measured: \(increment)")
}

Test Data

[Tests.ParserTest testParticipantCodeParsingPerformance]' measured [Time, seconds] average: 0.203, relative standard deviation: 19.951%, values: [0.186405, 0.182292, 0.179966, 0.177797, 0.175820, 0.205763, 0.315636, 0.223014, 0.200362, 0.178165]

Upvotes: 16

Views: 2760

Answers (2)

uraimo
uraimo

Reputation: 19791

The only way to do something similar to what you describe is setting a time limit graphically like @andyvn22 recommends.

But, if you want to do it completely in code, the only thing you can do is extend XCTestCase with a new method that measure the execution time of the closure and returns it to be used in an assertiong, here is an example of what you could do:

extension XCTestCase{
    /// Executes the block and return the execution time in millis
    public func timeBlock(closure: ()->()) -> Int{
        var info = mach_timebase_info(numer: 0, denom: 0)
        mach_timebase_info(&info)
        let begin = mach_absolute_time()

        closure()

        let diff = Double(mach_absolute_time() - begin) * Double(info.numer) / Double(1_000_000 * info.denom)
        return Int(diff)
    }
}

And use it with:

func testExample() {
    XCTAssertTrue( 500 < self.timeBlock{
        doSomethingLong()
    })
}

Upvotes: 5

andyvn22
andyvn22

Reputation: 14824

You need to set a baseline for your performance test. Head to the Report Navigator:

Report Navigator

and select your recent test run. You'll see a list of all your tests, but the performance ones will have times associated with them. Click the time to bring up the Performance Result popover:

Performance Result

The "Baseline" value is what you're looking for--set it to 0.5s and that will inform Xcode that this test should complete in half a second. If your test is more than 10% slower than the baseline, it'll fail!

Upvotes: 14

Related Questions