Repository navigation
ci: add E2E inventory and retry reporting - #291
Draft
piotr-iohk wants to merge 18 commits into
Draft
piotr-iohk wants to merge 18 commits into
piotr-iohk wants to merge 18 commits into
Conversation
This was referenced Nov 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #264.
Companion: synonymdev/bitkit-ios#1199
This PR supplies the inventory and attempt reporting consumed by the smaller iOS gate. The iOS repository owns path policy and automatic selection. Android's workflow selection and legacy retry behavior stay unchanged.
Description
ciIt()pass locks: cases passed earlier in this run are skipped, so only failed or uncompleted cases repeat. The iOS workflows expose three numbered attempts, stopping on success. The final report combines current-run passes, retains the first failure and fails after three unsuccessful attempts. App/E2E/artifact identities remain pinned.Merge this producer before its iOS consumer. The consumer pins the reusable fixture workflow so development branches can be validated before merge.
Validation
At E2E
b691df2/ iOS7380b949, controlled Mocha executions through the real attempt CLI verify that passed bodies run once, only failures repeat, recovery works on attempts two and three, persistent failures fail, and startup/timeouts/failed hooks/stale locks retain correct outcomes. The workflow check verifies three visible attempts and a final report that fails the job.The runner contract tests use controlled fixtures; they are not hosted Appium evidence. The initial hosted five-case gate passed first attempt at E2E
39f8964/ iOSf0a4aa3(run); that is historical runtime evidence before policy relocation. The current hosted policy check, full lanes and optional cost comparison are recorded separately. Existing iOS mainnet/skip gaps are documented, not counted as passes.Current full-suite validation
Keep one automatic local shard. All 42 local cases passed on both runs at iOS
5ba352f7/ E2Eee7a429. Two shards took 37.6% longer and used 71.0% more estimated Namespace compute on this pair.The 24-case shard passed first try in 30m44s of execution. In the 18-case shard,
boost_1timed out waiting forBoostButton. The retained screenshot and recording show iOS Settings in the foreground. Its other 17 cases passed, but the then-current runner repeated all 18 and then passed. The retry correction restores failed-only retries; this historical pair does not measure that behavior. This is one observation with a recovered failure. Estimates use measured runner/backend lifetimes and published rates. They exclude storage, rounding and account discounts. One-shard baseline · Two-shard experiment · Namespace ratesThe baseline completed 54/54 cases under the preceding workflow wiring. The regular two-shard run completed 48/48, covering 42 local and six staging cases. Its selection contained no migration. Only local elapsed time and paid Namespace compute are compared. An earlier two-shard dispatch mistakenly included migration and was cancelled. Its estimated $1.55 prepaid-equivalent / $2.32 overage is recorded separately. Cancelled dispatch
In the draft at iOS
94088e98,e2e-tests.ymlcontains no migration job, call, output or status dependency.e2e_migration.ymlowns a separate build, nightly schedule, release-push trigger and result. All six case assertions passed in the historical run. The independent build and artifact handoff will be verified on its next nightly or release run. Both PRs remain drafts.The preceding run restored the correct 449,502 sats in RN, but its upgrade and passphrase tests could not read the nested amount on iOS. The corrected test reads the balance from its parent accessibility label. A local iOS simulator using the hosted target build and funded RN fixture passed the passphrase upgrade, including balances, history and tags after relaunch. Preceding run
The previous full run passed all six staging cases on the first attempt and five of six migration cases within three attempts. All 12 RN source wallets were prepared. The RN passphrase case failed. One recording showed the correct 449,502 sats behind a USDT introduction. Another showed a truncated passphrase and an empty wallet before the upgrade. The harness now checks the exact passphrase and funded RN balance before installing the native app, then dismisses the introduction before verifying balances and history. Migration job
The previous local job reached its 90-minute limit three times, with 37, 38 and 39 of 42 cases passing on the respective attempts. Security and transfer each passed on a subsequent attempt. The first PIN reset showed a regtest log-deletion error and returned to home instead of onboarding. The transfer recording showed QuickPay covering the activity assertions. The security assertion remains in the suite. The transfer case handles the one-time introduction and still verifies the balance and both activity rows. Local job
That unsuccessful run consumed 271.72 Namespace Mac job-minutes and 271.00 observed backend minutes. Published rates imply $18.47 prepaid-equivalent or $27.71 overage for compute. The estimate excludes storage, credits, rounding and account discounts. Run · Namespace pricing
The initial diagnostic also exposed a redundant Android Back after RN send success that backgrounded the source wallet. Its correction was exercised by all three sweep wallets in the next run. Initial diagnostic
Models used
gpt-6-astra(reasoning:high),gpt-6-astra(reasoning:xhigh),gpt-6.1-sol(reasoning:high)gpt-6-astra(reasoning:high),gpt-6-astra(reasoning:xhigh),gpt-6.1-sol(reasoning:high)gpt-6-astra(reasoning:high),gpt-6.1-sol(reasoning:high)