Skip to content

docs(kaneai): Visual Comparison in app tests - #3715

Merged
Ishavyas9 merged 1 commit into
LambdaTest:stagefrom
chaitanyas-maker:docs/kaneai-app-visual-comparison
Oct 16, 2026
Merged

Ishavyas9 merged 1 commit into
LambdaTest:stagefrom
chaitanyas-maker:docs/kaneai-app-visual-comparison

Conversation

@chaitanyas-maker

Copy link
Copy Markdown
Contributor

Summary

Adds app test coverage to Visual Test Creation with KaneAI. The page used to cover web tests only, although the Visual Comparison step also works in app tests.

  • A web vs app overview table near the top of the page.
  • A new Visual Comparison in App Tests section. It covers starting an app test, adding and configuring the step, reviewing the baseline, saving and running, finding app results in SmartUI, and handling the device status bar.
  • Metadata: the description mentions app tests, two keywords are added, and dateModified is bumped.
  • The existing web steps are unchanged.

How the content was verified

The page is written from a live prod run on 16 Oct 2026: KaneAI app test on Android (Galaxy S25 Ultra, Android 15) with the Proverbial sample app, followed by two Execute runs. I cross-checked the result against the KaneAI source (auteur, lt-web-platform) and the test-run service (lambda-test-forge-service).

Behaviour seen on prod and documented:

  • App tests offer only View Port. Full Page is hidden.
  • With Manual Interaction on, the step stays Queued until Manual Interaction is turned off.
  • Screenshot names are sanitised: Proverbial Home becomes Proverbial_Home.
  • Two steps with the same name overwrite each other in a build.
  • Where app results land in SmartUI:
    • Authoring goes to project kane_ai_<id>, variant default.
    • The sample execution goes to the same project, on the device used for the sample run.
    • Execute and test runs go to a project named after the KaneAI project ID, on the device the test ran on.
  • Baselines are per device. The first screenshot on a device is New.
  • The device status bar is part of the screenshot. A rerun of an unchanged app showed Changes Found at 0.0878%, all of it in the clock and status icons.

Not verified live:

  • iOS. It uses the same code path.
  • The Test Runs dialog for app tests. The statement that Visual Regression settings apply only to web tests is based on the code.
  • The Bounding Boxes workaround for the status bar.

Images

There are 10 new files in assets/images/kane-ai/smartui/app/, all under 100 KB (the largest is 98 KB). The set includes one 4-frame GIF of the authoring flow, made from 1:1 crops.

Checks

  • Local Docusaurus preview: the page builds and renders, with both tables, the 3 callouts and all images.
  • Callouts use info and warning only. caution does not render on this site.

🤖 Generated with Claude Code

Adds a "Visual Comparison in App Tests" section and a web vs app
overview table to kaneai-smartui-visual-testing. Content is based on a
prod run on 16 Oct 2026 (Android, Galaxy S25 Ultra, Proverbial app) and
the auteur, lt-web-platform and lambda-test-forge-service source.

Covers: View Port only for apps, name sanitising, same-name overwrite,
Manual Interaction queueing, where app results land in SmartUI,
per-device baselines, and the status bar false-positive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Ishavyas9
Ishavyas9 merged commit 5404c46 into LambdaTest:stage Oct 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants