Open Radar

 
Summary:
iOS is smart and shows context relevant tips and tutorials when they seem useful, like the help sheet that one can swipe to enter numbers in the keyboard. Unfortunately this breaks our automated UI tests as it’s something that only happens on first run and thus has implicit state. A fresh system behaves different for tests. This is hard to manage.

Steps to Reproduce:
Run a pristine Simulator install, enter text, see Keyboard Tutorial.

Expected Results:
Things that are not predictable, such as Keyboard Tutorials BUT ALSO iCloud Auth logins or really anything that randomly comes from the system and requires user input should be block-able in a UI testing mode. 

Actual Results:
The Keyboard Tutorial blocks our tests and we can fix it by VNC-ing into our Jenkins worker and click once on “OK” until the next Simulator reset…

Version:
11.0 GM

Notes:
Here’s the relevant issue where we discuss this + potential workaround hacks in the Google EarlGrey UI Testing Framework: https://github.com/google/EarlGrey/issues/633

Please don’t make that mode only for XCUI. This doesn’t work for us. We need graybox testing where we can go into the model and don’t have a typical app flow.