Skip to content

Generate the Jetpack App Store screenshots from the app - #26339

Draft
jkmassel wants to merge 10 commits into
jkmassel/ui-tests-targets-structurefrom
jkmassel/app-store-screenshots
Draft

jkmassel wants to merge 10 commits into
jkmassel/ui-tests-targets-structurefrom
jkmassel/app-store-screenshots

Conversation

@jkmassel

@jkmassel jkmassel commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Generates the Jetpack App Store screenshots from the app: the UI tests from #26119 take them against the fixtures, in every App Store language, on iPhone, iPad and iPhone Duo, and a compositor frames them below a headline fitted to each language.

Stacked on #26119. The screenshots below were taken from this branch rebased onto trunk, which has the iPhone Duo fixes (#26207–#26212) that #26119's base doesn't.

Summary

  • bundle exec fastlane jetpack_screenshots builds the app with the fixtures and runs the new AppStoreScreenshots UI test once per device and language, saving the raw screenshots. Devices: iPhone 18 Pro Max (6.9"), iPad Pro 13-inch (M5) and iPhone Duo (outer and inner displays).
  • bundle exec fastlane create_jetpack_promo_screenshots composes them with Scripts/app-store-screenshots/compose.swift: a title screen, then each screenshot in a device frame below its headline. update_jetpack_metadata_on_app_store_connect with_screenshots:true uploads them, as before.
  • Headlines are fitted to each language: as large as they fit in two lines, one size for the whole set, balanced lines, Japanese and Chinese broken only between phrases, and a serif that ships with macOS for each script.
  • bundle exec fastlane trigger_jetpack_screenshots does the same on CI: .buildkite/app-store-screenshots.yml builds once on the Xcode 27.1 image, runs a job per device and language (51, in a matrix step per device, since a matrix makes at most 50) at a low priority, so they only use Mac agents nothing else is waiting for, and composes whatever was taken into downloadable artifacts.
  • On-screen content is translated too, for languages with a fastlane/appstoreres/jetpack_screenshot_content/<locale>.json: the lane writes translated copies of the fixture files and the app loads them over the originals. The fixtures the UI tests assert on don't change.
  • The fixtures can serve images (WireMock's bodyFileName) and take an overlay. The new images are the screens' header images and photos, and generated placeholder avatars.
  • The headlines are new GlotPress strings: a feature name ("Stats.") and the rest ("Your metrics, wherever you are."), so every language gets its bold lead without translators keeping markup.
  • Removed: the old Jetpack screenshot layout (jetpack_screenshots.json), its assets and strings, and Scripts/fix-screenshots.rb. The WordPress app's screenshots are unchanged.

docs/app-store-screenshots.md covers all of it.

What a reviewer will want to know

The iPhone Duo needs Xcode 27.1. An app built with an older SDK runs letterboxed on its outer display, so the lane stops if the selected Xcode, or a build it would reuse, is older. App Store Connect takes both Duo displays in one set (APP_IPHONE_DUO), which deliver uploads from fastlane 2.241.1, hence the Gemfile bump.

The simulator only models the Duo's outer display. For the inner one, the lane moves the app to a 669×951 pt resizable display with xcrun devicectl device appResize, as unfolding does. XCUITest can't tap there, so the test asks the lane for each inner screenshot through a file and waits. On the Duo the test relaunches the app before each screen, since not every screen lays itself out for the outer display again after unfolding.

The test finds its way by identifiers and the fixtures' content, never the app's text, since it runs in every language. That needed one app change: an accessibility identifier on the Reader post's back button.

Nothing runs on PRs. The test is skipped unless the lane gives it somewhere to save screenshots, and the screenshot pipeline only runs when it's started. A full local run (17 languages × 3 devices) takes about three hours; on CI it's 51 jobs that can run side by side.

The translations are temporary. Every App Store language has headlines and on-screen content that Claude translated, not translators, so the screenshots can be generated in all 17 now. They're placeholders until both go through the translation pipeline: the headlines are replaced the first time download_jetpack_promo_strings runs after GlotPress has them, and the on-screen content has no pipeline yet. The headline English is the current App Store copy.

Screenshots

Taken on this branch rebased onto trunk. Each image is one language and device side by side, as create_jetpack_promo_screenshots writes them to fastlane/jetpack_screenshots_overview. The Japanese and German headlines are stand-ins.

English

English, iPhone 6.9 inch: title screen, Stats, Editor, Reader and Notifications

English, iPad 13 inch: title screen, Stats, Editor, Reader and Notifications

English, iPhone Duo inner display

English, iPhone Duo outer display

Japanese and German, with translated content

Japanese, iPhone, with the posts and notifications translated

German, iPhone, with the posts and notifications translated

Japanese, iPad, with translated content

German, iPad, with translated content

Not in this PR

  • Putting the headlines and on-screen content through the translation pipeline, to replace the temporary translations.
  • A manifest for choosing screens, appearance and theme without editing code.
  • Fixing the app retrying a failed image request over 1,000 times, which the image fixtures sidestep here.

Test plan

  • swift test -Xswiftc -DUI_TEST_HTTP_FIXTURES --filter HTTPFixturesTests: 58 tests passed, including the new body file and overlay tests.
  • ruby fastlane/lanes/screenshot_content_helper_test.rb: 10 runs, 0 failures. All fastlane/lanes/*_test.rb pass, and RuboCop reports no offenses in fastlane and the Gemfile.
  • SwiftLint passes on the changed Swift files, which are stable under swift format.
  • On this branch rebased onto trunk, with Xcode 27.1: jetpack_screenshots passed on all three devices in English and Japanese, and with translated content in Japanese and German on iPhone and iPad.
  • On this branch as pushed, with Xcode 27.1: jetpack_screenshots passed on all three devices in English and Japanese. iPad in English first failed once, tapping a Reader card while the list replaced it; after the fix in the last commit it passed, as did iPhone in English and Japanese.
  • The temporary translations are valid for all 16 languages: each language's content generates its fixture overlay with every notification's names and titles in place, and every headline fits within two lines (three for the title) on the iPhone, iPad and both iPhone Duo layouts, with no warnings from the compositor.
  • .buildkite/app-store-screenshots.yml passes Buildkite's schema validation, and trigger_jetpack_screenshots is registered.
  • Run the screenshot pipeline on CI. Not done yet: it starts 51 Mac jobs, and whether devicectl appResize works on a CI VM, for the iPhone Duo's inner display, is still unknown.
  • Upload the screenshots to App Store Connect. Not done: it publishes to the live listing.

@jkmassel jkmassel added Testing Unit and UI Tests and Tooling Tooling Build, Release, and Validation Tools labels Nov 5, 2026
@jkmassel jkmassel self-assigned this Nov 5, 2026
@jkmassel jkmassel added this to the 27.5 milestone Nov 5, 2026
@dangermattic

Copy link
Copy Markdown
Collaborator
1 Warning
⚠️ This PR is larger than 500 lines of changes. Please consider splitting it into smaller PRs for easier and faster reviews.
1 Message
📖 This PR is still a Draft: some checks will be skipped.

Generated by 🚫 Danger

@wpmobilebot

wpmobilebot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor
App Icon📲 You can test the changes from this Pull Request in WordPress by scanning the QR code below to install the corresponding build.
App NameWordPress
ConfigurationRelease-Alpha
Build Number35790
VersionPR #26339
Bundle IDorg.wordpress.alpha
Commitd93fc1e
Installation URL5ni6r8lmpe690
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot

wpmobilebot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor
App Icon📲 You can test the changes from this Pull Request in Jetpack by scanning the QR code below to install the corresponding build.
App NameJetpack
ConfigurationRelease-Alpha
Build Number35790
VersionPR #26339
Bundle IDcom.jetpack.alpha
Commitd93fc1e
Installation URL023gigrggpffg
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot

wpmobilebot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Build Failure Analysis

This build has failures. Claude has analyzed them - check the build annotations for details.

A mapping can now answer with a file from the fixtures' `__files` directory, by naming it in `bodyFileName` as WireMock does. That's how images are served.

A second fixtures directory can also be laid over the first: its mapping files replace the ones at the same paths. The app loads one from the `-ui-test-http-fixtures-overlay` launch argument. The App Store screenshots use one to show the fixtures' content translated, without changing what the UI tests assert on.
Every image request used to get a 404, which the app retries over and over: over 1,000 times for one Reader header image in a single run. These are the Reader posts' header images, the editor draft's photo, and avatars for the notifications and the Reader post's author.

The avatars are generated placeholders, not anyone's Gravatar. Any other Gravatar gets a plain silhouette.
`AppStoreScreenshots` opens Stats, the editor, a Reader post and Notifications against the fixtures, and saves a screenshot of each. It runs in any language, so it finds its way by identifiers and the fixtures' content rather than the app's text, and it handles the iPad's sidebar and the iPhone Duo.

The Duo's inner display is shown by the lane on the Mac, which the test asks for through a file. The test is skipped unless the lane gives it somewhere to save the screenshots.

The Reader post's back button gets an accessibility identifier, since its label changes with the language.
`compose.swift` draws each screenshot in a device frame below its headline, plus a title screen, for the iPhone, the iPad and both iPhone Duo displays.

Each language's headlines are fitted to it: set as large as they fit in two lines, all at one size, with balanced lines. Japanese and Chinese only break between phrases. Scripts New York doesn't cover get a serif font that ships with macOS.
The screenshots' content comes from the fixtures, in English. `ScreenshotContentHelper` writes, for one language, a copy of every fixture file with content to translate, translated from `jetpack_screenshot_content/<locale>.json`.

Notifications mark names and titles in their text with UTF-16 ranges, which are moved to match the translation. A translation that leaves one out is an error.

The Japanese and German translations are drafts, to be reviewed.
- `jetpack_screenshots` builds the app with the fixtures, and runs `AppStoreScreenshots` for each device and language. Devices: iPhone 18 Pro Max, iPad Pro 13-inch and iPhone Duo. For the Duo it also shows and screenshots the inner display, which needs Xcode 27.1.
- `create_jetpack_promo_screenshots` composes them, ready for `update_jetpack_metadata_on_app_store_connect`.
- The headlines are new GlotPress strings: a feature name and the rest of the headline, for each screenshot.

The old Jetpack screenshot layout, its assets and strings, and `Scripts/fix-screenshots.rb` are removed. fastlane goes to 2.241.1 or later, which uploads the iPhone Duo's screenshots.
The Reader stream replaces its cards as images arrive. Tapping a card through its element can then find it with an infinite frame, which raises an exception that fails the test, as it did once on the iPad. The test now reads the card's frame once and taps that point on the screen, and tries again if the post didn't open.
Headlines and on-screen content for the 16 App Store languages besides English, so the screenshots can be generated in all of them now. Claude wrote them, not translators, and they're placeholders until the headlines and content go through the translation pipeline. Every content translation keeps the names and titles the notifications mark, and every headline fits on the iPhone, iPad and both iPhone Duo displays.
`.buildkite/app-store-screenshots.yml` builds the app once on the Xcode 27.1 image, then takes each device's screenshots in a matrix of the 17 App Store languages: 51 jobs, since a matrix makes at most 50. A last job composes whatever was taken, and uploads the composed screenshots and each language's overview.

`bundle exec fastlane trigger_jetpack_screenshots` starts it on the current branch.
The pipeline's 50-odd jobs now only take Mac agents nothing else is waiting for, so they never hold up a PR or trunk build. Everything else runs at the default of 0, and the release pipelines at 1.
@jkmassel
jkmassel force-pushed the jkmassel/ui-tests-targets-structure branch from b12ef60 to 8c4fae5 Compare November 6, 2026 20:27
@jkmassel
jkmassel force-pushed the jkmassel/app-store-screenshots branch from d93fc1e to 64b1bf7 Compare November 6, 2026 20:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Testing Unit and UI Tests and Tooling Tooling Build, Release, and Validation Tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants