Skip to content

ci: use custom test runner & enable swift_library_tests#338

Closed
rebello95 wants to merge 27 commits intomasterfrom
swiftlib-tests
Closed

ci: use custom test runner & enable swift_library_tests#338
rebello95 wants to merge 27 commits intomasterfrom
swiftlib-tests

Conversation

@rebello95
Copy link
Copy Markdown
Contributor

@rebello95 rebello95 commented Aug 16, 2019 •
edited
Loading

Introduces a custom test runner for iOS which works around a problem being fixed by bazelbuild/rules_apple#549, where we previously were unable to set a device/OS on which to run tests.

This allows us to re=enable swift_library_tests on CI.

Signed-off-by: Michael Rebello me@michaelrebello.com

@rebello95
WIP: ci: re-enable swift_library_tests
7a28fcc 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
lol command slash
8f70e1b 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
keith
keith previously approved these changes Aug 16, 2019
View reviewed changes
@rebello95
Copy link
Copy Markdown
Contributor Author

rebello95 commented Aug 16, 2019 •
edited
Loading

Something very odd is going on here - it’s launching the wrong sim in an invalid state, which is why we disabled this test previously:

19-08-16 05:18:22,640 Creating a new simulator:
Name: New-iPhone Xʀ-12.2
OS: iOS 12.2
Type: iPhone Xʀ2019-08-16 05:18:23,540 Created new simulator FAFCAC18-EC85-4178-AB6A-17193FDC5CDA.
An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=165):Invalid device state

@rebello95
Merge remote-tracking branch 'origin/master' into swiftlib-tests
0f85534 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
custom runner
a5699dd 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
take 2
fa86789 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
namespace
d664fa9 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95 rebello95 changed the title WIP: ci: re-enable swift_library_tests ci: use custom test runner & enable swift_library_tests Aug 17, 2019
@rebello95
Update BUILD
1a242eb 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
Update .bazelrc
313e7c9 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95 rebello95 marked this pull request as ready for review August 17, 2019 04:46
@rebello95 rebello95 requested a review from keith August 17, 2019 04:47
@rebello95
buildifier
0d6d699 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
keith
keith previously approved these changes Aug 17, 2019
View reviewed changes
@rebello95
public visibility, re-lint
ca3e1ca 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
update comment
5fc01de 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
Update BUILD
9b83838 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
version
9a96336 
> xctestrunner.shared.ios_errors.IllegalArgumentError: The simulator os version 12.1 is not supported. Supported simulator os versions are [u'8.4', u'9.0', u'9.1', u'9.2', u'9.3', u'10.0', u'10.1', u'10.2', u'12.2'].

Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
Update BUILD
543e015 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
doesn't like 12.2
ef96089 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
Copy link
Copy Markdown
Contributor Author

rebello95 commented Aug 17, 2019

@keith everything is green locally, but it looks like on CI it can't seem to find a viable OS version for the simulator. I've tried 12.1, 12.2, and 12.4 - all of which say the versions are invalid.

What's strange is that when running with 12.4, I get the following error:

xctestrunner.shared.ios_errors.IllegalArgumentError: The simulator os version 12.1 is not supported. Supported simulator os versions are [u'8.4', u'9.0', u'9.1', u'9.2', u'9.3', u'10.0', u'10.1', u'10.2', u'12.2'].

When when switching to 12.2, it says 12.2 is invalid and to switch to 12.1. 12.1 yields a similar error:

xctestrunner.shared.ios_errors.IllegalArgumentError: The simulator os version 12.1 is not supported. Supported simulator os versions are [u'8.4', u'9.0', u'9.1', u'9.2', u'9.3', u'10.0', u'10.1', u'10.2', u'12.2'].

Basically seems like we're in a loop. Any ideas?

@keith
Copy link
Copy Markdown
Member

keith commented Aug 19, 2019

Ok it sounds like you want 12.2, and we need to debug the invalid error. That must mean that CI is still on Xcode 10.2.x?

@rebello95
Revert "doesn't like 12.2"
4c43299 
This reverts commit ef96089.

Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
debugging
2c08a17 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
try
9fc7758 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
fixup
ea3f030 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
Revert "try"
ee03f11 
This reverts commit 9fc7758.

Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
fixup
11af556 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
12.4
7e28361 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
Update mac_ci_setup.sh
d7d8756 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
keith
keith reviewed Aug 19, 2019
View reviewed changes
ci/mac_ci_setup.sh Outdated
done

# /Applications/Xcode.app/Contents/Developer
sudo xcodebuild -runFirstLaunch
Copy link
Copy Markdown
Member

@keith keith Aug 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be after the switch, assuming the switch is actually working

Copy link
Copy Markdown
Contributor Author

@rebello95 rebello95 Aug 19, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, looks like we're still getting the following error:

xctestrunner.shared.ios_errors.IllegalArgumentError: The simulator os version 12.4 is not supported. Supported simulator os versions are [u'8.4', u'9.0', u'9.1', u'9.2', u'9.3', u'10.0', u'10.1', u'10.2'].

The Xcode path is still indicating 10.3, though:

/Applications/Xcode_10.3.app/Contents/Developer

And iPhone X on 12.4 is listed as being available:

-- iOS 12.4 --
    iPhone 5s (22DA5C87-561D-4147-A2ED-62F6C88C3690) (Shutdown) 
    iPhone 6 (9B3290A6-1E5F-4ACA-832D-C327C93A3A28) (Shutdown) 
    iPhone 6 Plus (29B6D028-AA03-481F-926E-0CAB108C3B1E) (Shutdown) 
    iPhone 6s (8841751E-A7FB-420C-9C6A-00C7646E120D) (Shutdown) 
    iPhone 6s Plus (57CEE6E3-0F66-42CD-A9C1-0212BA92522A) (Shutdown) 
    iPhone 7 (7852BB53-4751-4B0D-B0E7-2D5CC82FB2E5) (Shutdown) 
    iPhone 7 Plus (62FE3DA8-7C06-4353-AA46-02C50D0574ED) (Shutdown) 
    iPhone 8 (F4E5A2DE-1609-4C86-8AC4-0B7D799BDF78) (Shutdown) 
    iPhone 8 Plus (33DCB7FA-E504-4BC1-812A-82CD2E16CBEC) (Shutdown) 
    iPhone SE (6A941539-C0FD-4396-965C-3BC164B42840) (Shutdown) 
    iPhone X (067FE3A9-9FBB-4309-ACAA-5C6CBC6CA06F) (Shutdown)

@rebello95
hard life
3499fcf 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
Update mac_ci_setup.sh
5a26c2b 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
[!] No simulator matching iOS 12.4 was found. Please specify a versio…
3d78de7 
…n from the following available simulators:

Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
Please authenticate to install iOS 12.2 Simulator...
183659e 
Signed-off-by: Michael Rebello <me@michaelrebello.com>
@rebello95
Copy link
Copy Markdown
Contributor Author

rebello95 commented Aug 19, 2019

Giving up for now. After talking with @keith, it looks like the CI machines are somehow in a bad state and can't properly start the simulator(s) that are already installed.

Hopefully when we move to GitHub Actions this will start working. // @junr03

@rebello95
Copy link
Copy Markdown
Contributor Author

rebello95 commented Aug 21, 2019

Replacing with #364 now that we're on GitHub Actions

@rebello95 rebello95 closed this Aug 21, 2019
@rebello95 rebello95 deleted the swiftlib-tests branch August 21, 2019 18:47
This was referenced Aug 21, 2019
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@keith keith keith left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@rebello95 @keith