Test Coverage reports from xcodebuild CLI ignore coverage from UI tests
Originator: | sinophilic | ||
Number: | rdar://29033348 | Date Originated: | 10/31/2016 |
Status: | Duplicate of 28900146 (Open) | Resolved: | |
Product: | Developer Tools | Product Version: | |
Classification: | Reproducible: | yes |
Summary: A project with both Unit Tests and UI Tests will generate code coverage when ran in the XCode IDE. If the same project is run using `xcodebuild test` from CLI, code coverage from the Unit Tests will exist but coverage from UI Tests will be missing Sample project with UI Tests and Unit Tests, showing a difference in coverage from `xcrun llvm-cov report` between the project built via IDE and via CLI https://github.com/iwllyu/UITestCoverageBug thelvis4 found a discrepancy between how the two building processes differ - https://github.com/nakiostudio/xcov/issues/22#issuecomment-232014641 Steps to Reproduce: 1. install XCode 8.1 with Swift 3 2. clone the UITestCoverageBug project from https://github.com/iwllyu/UITestCoverageBug 3. Open the project in XCode and run the tests. See both Unit and UI tests get run. See both `testedViaUnitTest` and `testedViaUITest` are covered 4. in terminal, run `xcrun llvm-cov report` (see github for details) 5. check that coverage for ViewController.swift is 80% 6. in terminal, run `xcodebuild test` for project (see github for details) 7. in terminal, run `xcrun llvm-cov report` (see github for details) 8. check that coverage for ViewController.swift is now 40% Expected Results: coverage for ViewController.swift is 80% Actual Results: coverage for ViewController.swift is now 40% Version: XCode 8.1 OSX 10.11.5 Notes: Configuration: Attachments:
Comments
Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!
This is still occurring in Xcode 9.2
Issue appears resolved in Xcode 9
https://github.com/iwllyu/UITestCoverageBug/issues/4