Open Radar

 
Summary:
At Realm we've had a reproducible case where "xcodebuild test" fails, with the end of the output saying only ** TEST FAILED **. Digging through the log file, in the middle of the build output we see:

2015-10-08 13:17:33.051 xcodebuild[57711:240667]  iPhoneSimulator: Timed out waiting 120 seconds for simulator to boot, current state is 1.

Steps to Reproduce:
1. git clone https://github.com/realm/realm-cocoa.git
2. git checkout 0afd9939eaa6ffe829c0fd8f322b1f6ff00baa60
3. rm -rf build && xcodebuild -IDECustomDerivedDataLocation=build/DerivedData -project RealmSwift.xcodeproj -scheme RealmSwift -configuration Release -sdk iphonesimulator -destination 'name=iPhone 6' test

Expected Results:
The simulator launches and the tests run.

Actual Results:
The simulator doesn't launch, and the tests fail.

Version:
Xcode 7.0.1
Build version 7A1001

ProductName:	Mac OS X
ProductVersion:	10.11.1
BuildVersion:	15B30a


Notes:
I poked at xcodebuild in the debugger for a little while. What I found is that when configuring the "test" operation, xcodebuild immediately kicks off a background thread to wait for the simulator to boot. However, it doesn't start booting the simulator until after the dependencies of the test target have been built. If building the dependencies takes longer than the timeout (120 seconds), the tests will always fail to run. Using "xcodebuild build test" results in the dependencies of the test target being up to date, which avoids the issue.

I determined this by way of breakpoints on the following methods:

1. -[DVTiPhoneSimulator makeTransportForTestManagerService:], which blocks on a dispatch semaphore for up to 120 seconds or until it sees the simulator running. This method emits the "Timed out waiting 120 seconds" message seen in the output.

2. -[IDELaunchiPhoneSimulatorLauncher start], which is responsible for launching the simulator and simulated application.

With the two breakpoints in place, you'll see that -makeTransportForTestManagerService: is called relatively early. After it is called, xcodebuild goes on to compile the dependencies (in this case by way of shelling out in a script phase). Only once the dependencies have been built does -[IDELaunchiPhoneSimulatorLauncher start] run, at which point -makeTransportForTestManagerService: has already timed out waiting for the simulator.

We were able to work around this by using "xcodebuild build test", as this forces the dependencies to be built prior to the initialization of the test subsystem (and thus prior to -makeTransportForTestManagerService: waiting on the simulator to boot). Obviously this shouldn't be necessary.