CODEWITHSUNDEEP
xcodeunit-testingocunit
Nick L
Nick L

Reputation: 145

OCUnit tests failing, but Xcode Log Navigator says No Issues

Im having a very sneaky issue in Xcode and its "Log Navigator".

I've recreated the issue like so; I have a very simple OCUnit test:

-(void)testSimpleFailure{

    STFail(@"Please fail!");
}

I clean my build, I build my test scheme, and run my tests.

In the output console window I see all my test execute, including the one above. The console displays the test actually ran AND failed:

##teamcity[testFailed name='-|[SimpleTestClassTests.testSimpleFailure|]' message='Please fail!' details='']

HOWEVER, Xcode displays the "Tests Succeeded overlay" and the "Log Navigator" on the left says there are no issues and all is green.

Has anyone else had a similar issue? Does anyone know how to resolve this?

I would very much like to not get a false positive before I commit.

Edited to clarify my question.

Upvotes: 6

Views: 2561

Answers (6)

Rudiger
Rudiger

Reputation: 6769

If you are using XCTest, make sure you are deploying to an iOS 7 simulator. If you deploy to iOS 6 it will say build succeeds, test fails but not give any logging information in the debugger.

Upvotes: 0

EeKay
EeKay

Reputation: 6770

What worked for me: Reset the Simulator like mentioned by Nick L in his answer PLUS closing XCode and restarting. Then every build with an STAssertTrue(false) or STFail will give me a negative result.

Upvotes: 0

charles
charles

Reputation: 11762

I just read this long post about the issue that also affects Mac apps:

http://masonmark.com/the-xcode-fairy/

The trick seems to be to quit Xcode and open the project again. The bug is intermittent and will sometimes be here, or never be here, and it stays in this state until the next restart of Xcode.

It's a good read too.

Upvotes: 2

Shazron
Shazron

Reputation: 2478

I'm running my application-tests and unit-tests on a device. I'm getting the same results as you - tests are failing but the Log Navigator says everything passes. Not only that, I can't set breakpoints in my code.

The solution for me was to "Edit Scheme...", chose the "Test" scheme, and change my Debugger from "LLDB" to "GDB". I'm using Xcode 4.3.2, with my compiler set for all targets to LLVM 3.1

After changing my debugger for the Test scheme to GDB, The Log Navigator reports the test failures, and I can set breakpoints and debug as well while the tests are running.

link: search for "Unit tests crash before executing any code" (LLDB bug) https://github.com/kstenerud/iOS-Universal-Framework

Upvotes: 11

Nick L
Nick L

Reputation: 145

Ok, after about 3 developers over 5 days looking at this problem, we have found a solution:

rm -rf Library/Application\ Support/iPhone\ Simulator

Or, alternatively, if blowing away an entire directory tree isn't your thing:

Go to your iOS Simulator application. Choose iOS Simulator from the top menu and choose "Reset Content and Settings..."

Apparently something gets corrupted between the simulator and Xcode, so Xcode cannot hook to the simulator to watch the tests. Xcode believes no tests ran, therefore the tests pass.

Exactly what causes this, we aren't sure, but after 5 grueling days and 1 solution later, we don't care because we're heading to the nearest bar.

Cheers.

Upvotes: 7

zaph
zaph

Reputation: 112857

Yes, I've seen this, intermittent and annoying.

Do make sure the top (most recent) log message is selected.

Upvotes: 0

Related Questions