Skip to content

test: migrate e2e tests from mocha to playwright#248

Merged
gmegidish merged 1 commit into
mainfrom
test/migrate-mocha-to-playwright
Jun 2, 2026
Merged

test: migrate e2e tests from mocha to playwright#248
gmegidish merged 1 commit into
mainfrom
test/migrate-mocha-to-playwright

Conversation

@gmegidish
Copy link
Copy Markdown
Member

What

Migrates the test/ E2E suite from mocha + chai to @playwright/test (Playwright's built-in runner and expect).

Changes

  • Renamed specs for Playwright auto-discovery: server.tsserver.spec.ts, simulator.tssimulator.spec.ts, emulator.tsemulator.spec.ts.
  • Converted constructs: describe/it/before/aftertest.describe/test/beforeAll/afterAll; chai expect().to.* → Playwright expect().to*; this.skip()test.skip(condition, reason); this.timeout() removed in favor of a global config timeout.
  • Added playwright.config.ts with one project per suite (server/simulator/emulator), workers: 1, retries: 0, fullyParallel: false, timeout: 180000, reporter: 'list' to preserve mocha's serial, single-device behavior.
  • Deleted index.ts (the mocha aggregator — Playwright discovers files itself).
  • package.json: dropped mocha, chai, @types/mocha, @types/chai, ts-node, tsx; added @playwright/test. Scripts now run playwright test --project=… (added test and test:ios).
  • tsconfig.json: types reduced to ["node"].
  • CI (build.yml): dropped the no-op --grep "Android API …" from the Android job; iOS/server jobs unchanged. Makefile test-e2e: fixed test-simulatortest:simulator. .gitignore: ignore Playwright output dirs.

Verification

All three suites pass against real targets locally:

  • server: 10/10 passed
  • simulator (iOS 26): 20 passed, 1 skipped (the intentionally-disabled device lifecycle test)
  • emulator (Android): 15/15 passed against a connected device

Go coverage collection (GOCOVERDIR) is preserved unchanged.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 2, 2026

Walkthrough

This PR migrates the test suite from Mocha/Chai to Playwright test runner. The infrastructure layer adds playwright.config.ts, updates package.json to replace Mocha/Chai dependencies with Playwright, and removes deprecated test framework type declarations from tsconfig.json. The server, emulator, and simulator test suites are then converted to use Playwright's test.describe, test.beforeAll/test.afterAll, and test() APIs, with all assertions updated from Chai syntax to Playwright/Jest-style matchers. Response validation helpers are updated to consistently assert status fields. Finally, the CI workflow removes matrix-based grep filtering, the Makefile uses the new Playwright-driven test command names, and .gitignore ignores new Playwright report directories.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely describes the primary change: migrating E2E tests from mocha to playwright.
Description check ✅ Passed The description provides clear context for all changeset modifications, including file renames, framework conversions, configuration additions, and CI updates.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/migrate-mocha-to-playwright

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
test/server.spec.ts (1)

87-91: ⚡ Quick win

Use if guard pattern for consistency.

Lines 89-90 use the non-null assertion operator (!) to access error.code and error.data, while the rest of the test suite consistently uses an if (jsonResp.error) guard (e.g., lines 65-68, 113-116, 139-142). Aligning this test with the established pattern improves readability and consistency.

♻️ Align with the if-guard pattern used elsewhere
 	expect(data.jsonrpc).toBe('2.0');
 	expect(data.error).toBeDefined();
 	expect(data.error).not.toBeNull();
-	expect(data.error!.code).toBe(ErrCodeInvalidRequest);
-	expect(data.error!.data).toBe("'jsonrpc' must be '2.0'");
+
+	if (data.error) {
+		expect(data.error.code).toBe(ErrCodeInvalidRequest);
+		expect(data.error.data).toBe("'jsonrpc' must be '2.0'");
+	}
 });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/server.spec.ts` around lines 87 - 91, Replace the non-null assertions on
data.error with the same if-guard pattern used elsewhere: check if (data.error)
and then inside that block assert data.error.code equals ErrCodeInvalidRequest
and data.error.data equals "'jsonrpc' must be '2.0'"; ensure the test still
fails if error is absent (the if-guard preserves behavior and matches the
pattern used in other tests like the jsonResp.error checks).
test/simulator.spec.ts (1)

257-257: 💤 Low value

Consider using more idiomatic Playwright assertions for array checks.

The pattern expect(Array.isArray(x)).toBe(true) works but is less idiomatic than directly asserting on array properties.

♻️ Alternative approaches
-expect(Array.isArray(entries)).toBe(true);
+// Option 1: Just assert on array properties directly
+expect(entries.length).toBeGreaterThanOrEqual(0);

+// Option 2: If you want to explicitly check type
+expect(entries).toBeInstanceOf(Array);

Similarly for line 586:

-expect(Array.isArray(rawData.children)).toBe(true);
+expect(rawData.children).toBeInstanceOf(Array);

Also applies to: 586-586

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/simulator.spec.ts` at line 257, Replace the non-idiomatic check
expect(Array.isArray(entries)).toBe(true) with the Playwright-style assertion
expect(entries).toBeInstanceOf(Array) (and do the same for the other occurrence
around line 586); this targets the entries variable directly and uses the
built-in expect matcher to assert it's an Array.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/simulator.spec.ts`:
- Around line 290-296: The assertion is checking for the exact name
'mobilecli-test' but the created directory uses a timestamp suffix; update the
test "should remove the test directory from the app container" to detect the
timestamped directory by inspecting the fsList result (used via
fsList(simulatorId, `${containerPath}/Documents`) and variable names) and assert
that no entry name startsWith or matches the pattern 'mobilecli-test-' (e.g.,
use names.some(name => name.startsWith('mobilecli-test-')) and expect that to be
false, or use a regex match check) after calling fsRm(simulatorId, remoteDir,
true).

---

Nitpick comments:
In `@test/server.spec.ts`:
- Around line 87-91: Replace the non-null assertions on data.error with the same
if-guard pattern used elsewhere: check if (data.error) and then inside that
block assert data.error.code equals ErrCodeInvalidRequest and data.error.data
equals "'jsonrpc' must be '2.0'"; ensure the test still fails if error is absent
(the if-guard preserves behavior and matches the pattern used in other tests
like the jsonResp.error checks).

In `@test/simulator.spec.ts`:
- Line 257: Replace the non-idiomatic check
expect(Array.isArray(entries)).toBe(true) with the Playwright-style assertion
expect(entries).toBeInstanceOf(Array) (and do the same for the other occurrence
around line 586); this targets the entries variable directly and uses the
built-in expect matcher to assert it's an Array.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b52d1520-d677-4b56-80ed-cb767507bfd8

📥 Commits

Reviewing files that changed from the base of the PR and between 59afe0f and 85610ba.

⛔ Files ignored due to path filters (1)
  • test/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (10)
  • .github/workflows/build.yml
  • .gitignore
  • Makefile
  • test/emulator.spec.ts
  • test/index.ts
  • test/package.json
  • test/playwright.config.ts
  • test/server.spec.ts
  • test/simulator.spec.ts
  • test/tsconfig.json
💤 Files with no reviewable changes (1)
  • test/index.ts

Comment thread test/simulator.spec.ts
Comment on lines +290 to 296
test('should remove the test directory from the app container', async () => {
test.skip(!simulatorId, 'simulator not found');
fsRm(simulatorId, remoteDir, true);
const entries = fsList(simulatorId, `${containerPath}/Documents`);
const names = entries.map((e: any) => e.name);
expect(names).to.not.include('mobilecli-test');
expect(names).not.toContain('mobilecli-test');
});
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Directory removal check may not detect test directory.

The test directory is created with a timestamp suffix (mobilecli-test-${timestamp}), but line 295 checks for the exact string 'mobilecli-test' without the suffix. Since .toContain() checks for exact element matches in the array, this assertion won't detect if the timestamped directory still exists.

🐛 Proposed fix
 fsRm(simulatorId, remoteDir, true);
 const entries = fsList(simulatorId, `${containerPath}/Documents`);
 const names = entries.map((e: any) => e.name);
-expect(names).not.toContain('mobilecli-test');
+// Check that no directory starts with 'mobilecli-test-'
+const hasTestDir = names.some(name => name.startsWith('mobilecli-test-'));
+expect(hasTestDir).toBe(false);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/simulator.spec.ts` around lines 290 - 296, The assertion is checking for
the exact name 'mobilecli-test' but the created directory uses a timestamp
suffix; update the test "should remove the test directory from the app
container" to detect the timestamped directory by inspecting the fsList result
(used via fsList(simulatorId, `${containerPath}/Documents`) and variable names)
and assert that no entry name startsWith or matches the pattern
'mobilecli-test-' (e.g., use names.some(name =>
name.startsWith('mobilecli-test-')) and expect that to be false, or use a regex
match check) after calling fsRm(simulatorId, remoteDir, true).

@gmegidish gmegidish merged commit 9fd9f9c into main Jun 2, 2026
19 checks passed
@gmegidish gmegidish deleted the test/migrate-mocha-to-playwright branch June 2, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant