Skip to content

fix(executor): retry's maxRetries counts retries, not attempts - #182

Merged
omnarayan merged 2 commits into
devicelab-dev:mainfrom
vyrahealth:upstream-pr/retry-counts-retries
Oct 1, 2026
Merged

omnarayan merged 2 commits into
devicelab-dev:mainfrom
vyrahealth:upstream-pr/retry-counts-retries

Conversation

@bulatgaleev

@bulatgaleev bulatgaleev commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Maestro runs a retry block's commands once, then up to maxRetries more times: (maxRetries?.toIntOrNull() ?: 1).coerceAtMost(3), then while (attempt <= maxRetries) from 0 (Orchestra.kt). So maxRetries: 1 is two attempts, the default is two, and no retry runs more than four times. maestro-runner ran exactly maxRetries attempts, three when unset and with no cap, so a flow written with maxRetries: 1 never retried, and it failed the step when maxRetries was not an integer. Both forms of retry now count as Maestro does.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)

Changes Made

  • The inline and the file form of retry run the commands once and then up to maxRetries more times, capped at 3, with a default of 1.
  • A negative maxRetries runs no attempt and passes, since Maestro's loop never starts.
  • A value that is not an integer is logged and read as 1. TestExecuteRetry_InvalidMaxRetries, which expected that to fail the step, is replaced by tests of the new reading.
  • pkg/executor/retry_attempts_test.go, and a CHANGELOG.md entry.

Related Issues

No existing issue found.

Testing

  • go test ./pkg/executor/
  • Added tests for new functionality
  • Running on a physical iPhone (iOS 27, WebDriverAgent reached through a forwarded port) in our nightly 46-flow suite, as part of our fork's build
  • go vet ./... and make fmt-check pass (run on the top of the stack, fix(wda): checked selectors work on iOS #185, which contains all five changes)
  • make test as a whole: not run, because its device tests drive whatever device is attached to the machine
  • make lint: the Makefile has no lint target, and the linters make check runs (staticcheck, revive, errcheck, nilaway, gosec) are not installed here

Checklist

  • Code follows project style guidelines
  • Self-reviewed the code
  • Added/updated documentation as needed (no documentation change needed)
  • No breaking changes (or documented if breaking)
  • CHANGELOG.md updated (for notable changes)

Additional Notes

A flow that relied on the old count now retries as it does under Maestro: maxRetries: 1 gets its second attempt, and an unset maxRetries gets two attempts instead of three.

Stack 2 of 5: #181 → #182 → #183 → #184 → #185. Merge in that order. This branch is built on #181, so it also contains its commit. This PR's own change is the top commit, 48922eb. Once the PRs below it merge, the rest of the diff disappears, and nothing conflicts. All five sit on main at fe3dd53 (the 1.1.28 release).

The WDA driver reads an element's name, rect, text and displayed in
parallel, and a tap looks an element up four ways at once. The client used
Go's default transport, which keeps two idle connections per host, so every
such burst closed two connections and opened two new ones.

On a real iPhone reached through a forward (SSH, then iproxy over USB) a
new connection costs about 300 ms, and new ones opened together fail at
once with EOF. In one 44-flow run, 297 of 996 element bursts lost exactly
two reads (the two new connections), 2 lost one, and none lost a read on a
kept connection: 976 EOFs, each sent again by the dropped-connection
retry. WebDriverAgent does not close an idle keep-alive connection
(FBHTTPServer exempts them from its reaper), so a stale reuse was not the
cause.

The client now keeps up to eight idle connections per host. In the new
tests, 50 bursts of four reads open 4 connections instead of 102, and
through a server that drops every new connection past the first four, no
read fails (the old client lost 23 reads there, after 196 dropped
connections).
Maestro runs a retry's commands once and then up to maxRetries more times:
`(maxRetries?.toIntOrNull() ?: 1).coerceAtMost(3)`, then
`while (attempt <= maxRetries)` from 0 (Orchestra.kt:934-955, with
MAX_RETRIES_ALLOWED at 1844 and the "1" default at YamlFluentCommand.kt:614).
So `maxRetries: 1` is two attempts, the default is two, and no retry runs
more than four times. The value is evaluated first, so ${...} works
(Commands.kt:986), and anything that is not an integer counts as 1.

The runner ran exactly maxRetries attempts, three when unset, with no cap,
so a flow written with `maxRetries: 1` never retried at all. It also failed
the step when maxRetries was not an integer. The inline and the file form
of retry now count as Maestro does. A negative maxRetries runs no attempt
and passes, since Maestro's loop never starts. A value that is not an
integer is logged and read as 1.
@bulatgaleev
bulatgaleev force-pushed the upstream-pr/retry-counts-retries branch from 102dfe3 to 48922eb Compare September 30, 2026 14:49
@omnarayan

Copy link
Copy Markdown
Contributor

Thank you @bulatgaleev. We checked this against Maestro's source too (Orchestra.kt: (maxRetries ?: 1).coerceAtMost(3), then while (attempt <= maxRetries)), and you're right: maxRetries: N is N+1 attempts, the default is two, and the cap is four.

This changes behaviour on every driver, not only WDA. A flow with no maxRetries goes from three attempts to two, and maxRetries: 1 now actually retries. So we'll call it out as a behaviour change in the release notes.

Merging into main now; it ships in 1.1.29.

@omnarayan
omnarayan merged commit 070f49c into devicelab-dev:main Oct 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants