Ready the iOS simulator before Maestro and restart a driver that exits on first launch - #74
Merged
Merged
Conversation
…s on first launch
Pull request footprint
User-delivered artifact sizes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
On a runner that is still settling, the iOS lanes (the three
native-e2e-ios-lightshards andui-screenshots-ios-dark) lost the Maestro XCUITest driver during the first flow's launch command. In run 36213528963 (first attempt, detail shard)maestro.logends the firstLaunch app "app.cuetracker" with clear statewithTransport unreachable while processing setPermissions, latchingand the JUnit failure ismaestro.DeviceUnreachableException: Device became unreachable during setPermissions. No assertion had run; the flow never got past its first command.What changes
Boot.
scripts/prepare-ios-ui.shalready boots the simulator first (xcrun simctl boot, overlapping the Maestro download) and blocks onxcrun simctl bootstatus <udid> -bbefore installing. This PR keeps that.App warm-up. After
simctl install, the script launches the app once withxcrun simctl launch --terminate-running-process, pollsxcrun simctl spawn <udid> launchctl listfor a numeric PID on theapp.cuetrackerlabel (for up to 30 s), then runsxcrun simctl terminate. If the app never shows up as running,terminatefails and the job fails in the prepare step instead of inside Maestro. Maestro'sclearStatereinstalls the app, so the app's own container is recreated. What stays warm is the system side: SpringBoard, the dyld cache, and the launch services path.One bounded driver restart.
scripts/maestro-ios-test.shwrapsmaestro --device "$DEVICE_ID" test --debug-output <dir>. When the run fails, it rerunsmaestro testexactly once, and only if the run'smaestro.logmeets both conditions:Transport unreachable while processing, andLaunch app(no assertion, tap, or other step had begun).A new
maestro testprocess restarts the XCUITest runner (restartXCTestRunneruninstalls, installs, and starts it at session start). Assertion failures, launch failures without the latch, and driver deaths later in a flow fail on the first attempt with no retry. Run 36213699490's activity shard is a mid-flow driver death (the latch fires duringTap on id: snackbar-undo, after several assertions passed), so it is deliberately not retried.Tests
test/ci/ios-maestro-readiness.test.tsruns the wrapper against a fakemaestrothat replays log excerpts copied from the failing runs:A workflow-structure test checks that every macOS job that runs Maestro is exactly
native-e2e-ios-lightandui-screenshots-ios-dark, that each runsscripts/prepare-ios-ui.shbeforescripts/maestro-ios-test.sh, and that neither callsmaestrodirectly.Mutations, each caught by a failing test:
maestrocallThe job parser from
release-paths.test.tsmoved totest/support/workflow-jobs.tsso both tests share it.Cost on a healthy runner
Device already booted, nothing to do.by the timebootstatusran. The other two waited inbootstatuswhile the boot overlapped the Maestro download, as before.simctlcalls timed inside Maestro's own log in run 36213699490 (activity shard):simctl launchtook 2.8 s (launch start to command completion) andsimctl terminatetook 1.9 s. This PR adds onelaunchctl listpoll that I have not timed.Waiting for flows to complete...).Not measured: whether the warm-up reduces first-launch driver exits. That needs CI runs on this branch.
Sources
xcrun simctl help bootstatus(Xcode 27.0 locally; CI selects Xcode 26.6): "-b Boot the device if it isn't already booted ... prints boot status information until the device finishes booting."xcrun simctl help launch: "--terminate-running-process Terminate any running copy of the application."maestro-client/src/main/java/maestro/DeviceUnreachableException.kt: the exception is "an infrastructure failure, not a test failure ... (no test-error reporting, no flow-level retry)", and "Once a driver records a transport failure it should fail-fast on subsequent calls". This is why the restart happens at themaestro testlevel and not through a flow-levelretry. https://github.com/mobile-dev-inc/maestro/blob/cli-2.10.0/maestro-client/src/main/java/maestro/DeviceUnreachableException.ktmaestro-ios-driver/src/main/kotlin/xcuitest/XCTestDriverClient.kt:transportCalllogsTransport unreachable while processing $callName, latchingon any transport IOException, andrestartXCTestRunnerreinstalls and restarts the runner. https://github.com/mobile-dev-inc/maestro/blob/cli-2.10.0/maestro-ios-driver/src/main/kotlin/xcuitest/XCTestDriverClient.ktsimctl spawn <udid> launchctl listand strips theUIKitApplication:prefix and[...]suffix from the label: https://github.com/mobile-dev-inc/Maestro/blob/8c24ce06/maestro-ios-driver/src/main/kotlin/util/XCRunnerCLIUtils.kt