diff --git a/plugins/codeceptjs/skills/ci-fix-tests/SKILL.md b/plugins/codeceptjs/skills/ci-fix-tests/SKILL.md index e98c0d9..8090a3c 100644 --- a/plugins/codeceptjs/skills/ci-fix-tests/SKILL.md +++ b/plugins/codeceptjs/skills/ci-fix-tests/SKILL.md @@ -1,6 +1,6 @@ --- name: ci-fix-tests -description: "Use on CI in non-interactive mode when a CodeceptJS run failed — automatically attempt safe fixes (locator drift, missing waits), rerun only the failing scenarios, compare against the baseline, roll back any edit that didn't help, and always write a markdown report at `output/ci-fix.md` for the CI step to consume. Conservative by design: no refactors, no config edits, no auth fixes, no flake-masking retries. Trigger on \"ci fix tests\", \"auto-fix failing tests\", \"attempt repair on CI\", or as a CI step after a failed run." +description: "Use on CI in non-interactive mode after a failed CodeceptJS run — attempts safe fixes only (locator drift, missing waits), reruns failing scenarios, rolls back edits that didn't help, reports at `output/ci-fix.md`. No refactors, no config edits, no flake-masking. Trigger on \"ci fix tests\", \"auto-fix failing tests\", \"attempt repair on CI\", or as a CI step after a failed run." --- # Auto-fix CodeceptJS Tests on CI @@ -25,7 +25,7 @@ Save this as the **baseline** — `count`, `failing_set`, `clusters`. ### 2. Pick safe fix candidates Only attempt fixes from this allowed list: -- **Locator drift** — the failed locator no longer matches anything but a similar element exists. Use the **codeceptjs-exploration** skill (headless) to find candidates; pick one with high semantic stability (ARIA `{ role, name }` → visible text → `data-testid` → composed CSS). Replace the locator at the failing step only. +- **Locator drift** — the failed locator no longer matches anything but a similar element exists. Use the **codeceptjs-exploration** skill (headless) to find candidates; pick a short locator scoped to a stable region — `I.click('Save', '.toolbar')` — in this order: visible text / accessible name → ARIA `{ role, name }` → `$name` (`customLocator`) → composed CSS. Replace the locator at the failing step only. - **Missing wait for a spinner / loader / modal** — the failed step's ARIA snapshot shows a spinner or skeleton present, or the target element appears later. Add a single matching `I.waitFor*` immediately before the failing step. - **`I.wait(N)` replacement** — when a hardcoded sleep is the only thing between a failing assertion and a passing one and the gating element is identifiable, replace the sleep with a specific `waitFor*`. @@ -81,7 +81,7 @@ Failing scenarios: N - ... ## Attempted fixes -- `tests/foo_test.js:42` — locator drift: `'Save'` → `{ role: 'button', name: 'Save' }` +- `tests/foo_test.js:42` — locator drift: `I.click('Save')` → `I.click('Save', '.toolbar')` - `tests/bar_test.js:15` — added `I.waitForInvisible('.spinner')` before checkout click - `tests/baz_test.js:7` — replaced `I.wait(3)` with `I.waitForVisible('.confirmation-dialog', 10)` @@ -112,9 +112,8 @@ The first line of `Status:` is the machine-parseable signal. The rest is for the - Writing a partial or missing `output/ci-fix.md`. CI depends on it; the absence of a report is itself a failure mode. - Running the **whole** suite for the verification step — only the originally-failing scenarios, to keep CI time bounded. -## Pointers +## Related skills -- `codeceptjs-run-analysis` — read trace artifacts, cluster failures, build the baseline set. -- `codeceptjs-exploration` — find replacement locators when one has drifted. -- `codeceptjs-fundamentals` — confirm helper, config, which env the CI run used. -- `node_modules/codeceptjs/docs/aitrace.md` — trace format. +- `codeceptjs-run-analysis` — baseline, clustering, post-fix comparison +- `codeceptjs-exploration` — replacement locators for drift +- `codeceptjs-fundamentals` — config, helper, CI environment diff --git a/plugins/codeceptjs/skills/codeceptjs-auth/SKILL.md b/plugins/codeceptjs/skills/codeceptjs-auth/SKILL.md index db9742f..b3ef57e 100644 --- a/plugins/codeceptjs/skills/codeceptjs-auth/SKILL.md +++ b/plugins/codeceptjs/skills/codeceptjs-auth/SKILL.md @@ -1,51 +1,45 @@ --- name: codeceptjs-auth -description: Use when a CodeceptJS test needs login, when different user roles are involved, or when the writing-codeceptjs-tests skill identifies authorization is required. Configures the `auth` plugin for session reuse, derives the login flow from the actual login page HTML (not guesses), keeps the real flow inside `steps_file.js` so `I.login*()` is callable directly and the conf stays small, loads credentials from a `.env` file via the modern Node `process.loadEnvFile()` API (no `dotenv` dependency), and supports multiple roles. Trigger on mentions of login, sign-in, sign-up, authentication, sessions, "logged in", admin/editor/user roles, or auth-related test failures. +description: > + Use when a CodeceptJS test needs login, user roles, or session reuse — or + when another CodeceptJS skill identifies authorization is required. Also on + auth-related test failures. Trigger on login, sign-in, sign-up, + authentication, sessions, "logged in", admin/editor/user roles. --- # CodeceptJS Auth Plugin -The `auth` plugin logs each user in once, captures cookies (or local-storage / token via overrides), and restores the session for subsequent tests. Stale sessions trigger a fresh login automatically. +The `auth` plugin logs each user in once, captures cookies (or localStorage/token via overrides), and restores the session for subsequent tests. Stale sessions trigger a fresh login automatically. -Source of truth: `node_modules/codeceptjs/lib/plugin/auth.js` — JSDoc lists every recipe. Reference doc: `node_modules/codeceptjs/docs/auth.md`. - -## When to add it - -Suggested if tests have repeatable authentication and session is needed to be persisted accross tests. Also optimizes start time of tests by saving previous sessision cookies in files. - -Suggested as performance optimization, refactoring measure. - -If `auth` plugin already exists in the project (check fundamentals' output), reuse the existing `inject` name and user keys. +If the project already has `auth` configured (fundamentals' discovery output), **reuse** its existing inject name and user keys — don't reconfigure. ## Decide first — ask the user -Four answers shape the plugin. Don't guess any. If the project or test plan doesn't make them obvious, ASK. +Four answers shape the plugin. Don't guess any. -1. **Is a session needed at all?** Public flows (landing, signup) don't need one — skip the plugin. -2. **One user or many?** Default to one. Add more only when more than one is actually exercised. -3. **If many — what splits them?** Don't assume "admin / editor / viewer". Real systems split users by role, workspace / tenant, plan tier, sign-in provider (Google vs SSO vs password — same person), or per-test fixture. ASK the user; use the answer to name the `users.` entries. -4. **What's the auth type?** Form login is the default; others need a different `login()` body: - - **Form** — `fillField` → `click Sign in`. Canonical shape below. - - **OAuth / SSO** — click provider button, drive the IdP page (often a separate origin). - - **Magic link / passwordless** — UI flow is rarely worth automating. Prefer a backdoor — API mint, or read the link from a test mailbox. - - **API token** — skip the form; `I.executeScript` to write the token into `localStorage`, or `I.setCookie(...)`. - - **2FA / OTP** — `async login`; fetch the code from a test mailbox / backdoor before submitting. +1. **Is a session needed at all?** Public flows (landing, signup) → skip the plugin. +2. **One user or many?** Default one; add more only when actually exercised. +3. **If many — what splits them?** Role, workspace/tenant, plan tier, sign-in provider, per-test fixture — real systems vary. Ask; use the answer to name `users.` entries. +4. **What's the auth type?** + - **Form** — default; canonical shape below + - **OAuth / SSO** — click provider button, drive the IdP page (often separate origin) + - **Magic link / passwordless** — UI flow rarely worth automating; prefer an API mint or reading the link from a test mailbox + - **API token** — skip the form; write the token into `localStorage` via `executeScript`, or `I.setCookie(...)` + - **2FA / OTP** — async `login`; fetch the code from a test mailbox / backdoor before submitting ## Rules -1. **Login flow must not be written in the conf.** Better to place it nto steps_file.js (if it is includded) or page object. The conf better to reference it like: `login: (I) => I.login()`. -2. **Credentials should not be stored in configs. Use `.env` via `process.loadEnvFile()`** (modern Node, no `dotenv` package). Passwords wrapped with `secret(...)`. No literal credentials anywhere — conf, steps file, test, or git history. -3. **`.env` is gitignored; `.env.example` is committed** with the var names and no values. `output/*_session.json` is gitignored too. +1. **Login flow never lives in the conf.** Put it in `steps_file.js` (if included) or a page object; conf only references it: `login: (I) => I.login()`. +2. **Credentials from env only** — `.env` loaded via `process.loadEnvFile()` (no dotenv dependency). Passwords wrapped with `secret(...)`. No literals anywhere — conf, steps file, test, git history. +3. **`.env` is gitignored; `.env.example` is committed** with names, no values. Gitignore `output/*_session.json` too. ## Canonical shape ```js // codecept.conf.js — first line of the file -process.loadEnvFile() // or dotenv.load() if this package availble -// ... +process.loadEnvFile() export const config = { - // ... include: { I: './steps_file.js' }, plugins: { auth: { @@ -79,56 +73,46 @@ export default function () { } ``` -``` -# .env (gitignored) # .env.example (committed) -USER_EMAIL=USER@example.com USER_EMAIL= -USER_PASSWORD= USER_PASSWORD= +```sh +# .env (gitignored) # .env.example (committed) +USER_EMAIL=... USER_EMAIL= +USER_PASSWORD= USER_PASSWORD= ``` ## Pre-flight (before writing config) -1. **Read the real login page HTML.** Don't guess locators. MCP: `run_code` to login page and inspect the ARIA snapshot. Field labels / `name` / `id` / submit control — from the actual page. It's ok to ask user about authorization if it is not clear how to makeit. -2. **Pick a role-specific post-login marker.** Something the page renders only for *this* user (navbar username, `data-user-role`). -3. **Confirm session storage.** Cookies (default) for server-rendered apps; `localStorage`/`sessionStorage` for SPAs — verify with `I.executeScript(() => Object.keys(localStorage))` after a manual login. Default cookie `fetch`/`restore` silently no-op for token storage. +1. **Read the real login page** — MCP `run_code` to `/login`, inspect the ARIA snapshot (`codeceptjs-exploration`). Field labels / `name` / `id` / submit control from the actual page, not guesses. Unclear authorization mechanism → ask the user. +2. **Pick a role-specific post-login marker** — something rendered only for *this* user (navbar username, `data-user-role`). +3. **Confirm session storage** — cookies (default) for server-rendered apps; localStorage/sessionStorage for SPAs. Verify after a manual login with `I.executeScript(() => Object.keys(localStorage))`. Cookie fetch/restore silently no-op against token storage. ## Verify -Or MCP `run_test` against a one-Scenario file that calls `login()` then asserts on the post-login marker. - -Enable `saveToFile: true` only after the verification round-trip succeeds — a bad saved session masks a broken `login`. - -## Refactor - -Add to before hooks (if applied to all tests in suite) or to exact tests in syute - -`Before(({ login }) => login())` - -`login` was declared in auth plugin configuration - -Run real, not dry: +Run a one-Scenario file that calls `login()` then asserts on the post-login marker: ```bash -npx codeceptjs run --grep '' --debug +npx codeceptjs run --grep '' --debug # real run, not dry — dry-run doesn't init plugins ``` +Enable `saveToFile: true` only after this round-trip succeeds — a bad saved session masks a broken `login`. + +Then wire into hooks/tests: `Before(({ login }) => login())` for suite-wide, or per-test as needed. -## Multi-session shape +## Multi-role shape -Only after question 3 is answered. Name `users.` after whatever splits them in *this* system; one matching login method per key in `steps_file.js`. +Only after question 3 is answered. Keys named after whatever splits users *in this system*; one matching actor method per key: ```js -// keys named after the dimension (role / workspace / provider / …) users: { admin: { login: (I) => I.loginAsAdmin() }, workspaceB: { login: (I) => I.loginToWorkspaceB() }, } ``` -Don't parameterise into a single `login(key)` — the plugin keys sessions by name and explicit methods read better. Switching between sessions in one Scenario: `session('')` opens a separate browser context (see `node_modules/codeceptjs/docs/sessions.md`). +Don't parameterise into a single `login(key)` — the plugin keys sessions by name, explicit methods read better. Switch mid-Scenario: `session('')` opens a parallel browser context (fundamentals § Writing tests). -## Token / local-storage auth +## Token / localStorage auth -When sessions live outside cookies, override `fetch` and `restore`: +Override `fetch` / `restore` when sessions live outside cookies: ```js admin: { @@ -142,17 +126,16 @@ admin: { } ``` -`check(I, session)` receives whatever `fetch` returned — throw inside `check` to force a fresh login (e.g., when a `/me` endpoint shows the wrong user). - -## Common pitfalls +`check(I, session)` receives whatever `fetch` returned — throw inside `check` to force fresh login (e.g. `/me` endpoint shows wrong user). -- **Credentials inlined in conf or test.** Always env-driven. A code review showing a literal email or password = skill failed. -- **Forgetting to gitignore `.env` and `output/*_session.json`.** Both leak credentials. +## Pitfalls +- Credentials inlined in conf/test — always env-driven + `secret()`. +- Forgetting to gitignore `.env` and `output/*_session.json` — both leak credentials. -## Pointers +## Related skills -- `node_modules/codeceptjs/lib/plugin/auth.js` — JSDoc recipes (cookie / multi-user / local-storage / async / session-validation) -- `node_modules/codeceptjs/docs/auth.md` — full reference -- `node_modules/codeceptjs/docs/sessions.md` — `session()` for multi-user Scenarios -- `node_modules/codeceptjs/docs/secrets.md` — the `secret()` wrapper +- `codeceptjs-fundamentals` — secrets rule, sessions, config mutation trap +- `codeceptjs-exploration` — reading the live login page +- `writing-codeceptjs-tests` / `refactoring-codeceptjs-tests` — invoke this skill when auth is identified +- `debugging-codeceptjs-tests` — auth-related failure patterns diff --git a/plugins/codeceptjs/skills/codeceptjs-exploration/SKILL.md b/plugins/codeceptjs/skills/codeceptjs-exploration/SKILL.md index a6adf4b..81403ad 100644 --- a/plugins/codeceptjs/skills/codeceptjs-exploration/SKILL.md +++ b/plugins/codeceptjs/skills/codeceptjs-exploration/SKILL.md @@ -1,50 +1,54 @@ --- name: codeceptjs-exploration -description: Use to explore a page in CodeceptJS — read its ARIA tree, inspect candidate elements, pick a stable locator. Drives the live browser through MCP `run_code`, prefers ARIA over HTML, uses `I.grabWebElement` / `I.grabWebElements` with permissive XPaths to enumerate candidates and `toSimplifiedHTML` / `toAbsoluteXPath` to disambiguate. Other skills (`writing-codeceptjs-tests`, `debugging-codeceptjs-tests`, `refactoring-codeceptjs-tests`) invoke this whenever they need to learn what's on a page. +description: > + Use when an agent needs to learn what's on a page in CodeceptJS — read the + ARIA tree, inspect candidate elements, pick or disambiguate a stable locator. + Drives the live browser via MCP `run_code` / `snapshot`. Invoked by + writing-codeceptjs-tests, debugging-codeceptjs-tests, and + refactoring-codeceptjs-tests whenever page inspection is needed. --- # CodeceptJS Page Exploration -Authoring a test, debugging a failure, and refactoring a stale locator all share one task: open a page, find the right element, pick a stable locator. This skill is the playbook. +Authoring a test, debugging a failure, and refactoring a stale locator share one task: open a page, find the right element, pick a stable locator. This is that playbook. -## How to look at a page +## Tools -Drive everything through MCP. Two tools matter for exploration: +- **`run_code`** — runs CodeceptJS code, returns produced values, captures `console.*`, saves a final-state snapshot. For *do something and look at the result*. +- **`snapshot`** — captures state without acting (URL, cookies, localStorage, HTML, ARIA, screenshot, console). For "what's on the page right now". -- **`run_code`** — runs CodeceptJS code and returns the value the code produced, captures `console.*` output, and saves a final-state snapshot. Use when you want to *do something and look at the result* (try a locator, grab a value, navigate). -- **`snapshot`** — captures the current state without performing any action: URL, cookies, localStorage, HTML, ARIA, screenshot, console. Use when you want to look at "what's on the page right now" between two actions, without re-running anything. +Artifact sources, in preference order: -Three artifact sources, in order of preference: +1. **ARIA snapshot** — structured, no styling noise, easy duplicate/accessibility-name scanning +2. **Screenshot** — visual confirmation; catches layout breaks ARIA can't show +3. **HTML** — only when ARIA lacks context (custom widgets without accessible names, attribute-driven behaviour) -1. **ARIA snapshot first.** Structured, free of styling noise, easy to scan for duplicates and accessibility names. -2. **Screenshot.** Visual confirmation — catches layout breaks, missing icons, "rendered but wrong" cases that ARIA can't show. -3. **HTML / outer markup.** Pull only when ARIA is missing crucial context: custom widgets without accessible names, attribute-driven behaviour, dynamic content with no roles. +## Inspect an element -## Inspect a known element +`I.grabWebElement(locator)` → one WebElement; `I.grabWebElements(locator)` → array. Same cross-helper API on Playwright / Puppeteer / WebDriver. -`I.grabWebElement(locator)` returns one WebElement; `I.grabWebElements(locator)` returns an array. Same cross-helper API on Playwright / Puppeteer / WebDriver, returns values back through MCP `run_code`. - -| You want to … | Method on WebElement | +| You want to … | Method | |---|---| | Confirm rendered / visible / enabled | `exists()`, `isVisible()`, `isEnabled()` | | Read text / value / attribute / property | `getText()`, `getValue()`, `getAttribute(n)`, `getProperty(n)` | -| Where is it on the page | `getBoundingBox()` — flags offscreen / zero-sized | -| The actual rendered markup | `toOuterHTML()`, `toSimplifiedHTML(300)` (truncated, MCP-friendly) | +| Position on page | `getBoundingBox()` — flags offscreen / zero-sized | +| Rendered markup | `toOuterHTML()`, `toSimplifiedHTML(300)` (truncated, MCP-friendly) | | Stable selector for a fix | `toAbsoluteXPath()` | -| Look inside an iframe | `inIframe(async (body) => { ... })` | +| Inside an iframe | `inIframe(async (body) => { ... })` | | Drill into children | `$(loc)`, `$$(loc)` | -| Run a browser-side function | `evaluate(fn, ...args)` | +| Browser-side function | `evaluate(fn, ...args)` | ## Discover candidates when the obvious locator misses -When `Edit` matches nothing, the control might say "Change", carry `aria-label="Edit user"`, or live in a `.btn-edit` class. Cast a wide net: pass a permissive XPath to `I.grabWebElements`, then disambiguate. +When `Edit` matches nothing, the control may say "Change", carry `aria-label="Edit user"`, or live in `.btn-edit`. Cast a wide net with a permissive XPath via `I.grabWebElements`, then disambiguate. + +OR together in the XPath: -Build the XPath by ORing: -- visible text — `text()` (or `.` to match descendants too) -- relevant attributes — `@class`, `@aria-label`, `@title`, `@data-action`, `@id` -- **synonyms** — "edit" / "change" / "modify"; "delete" / "remove" / "trash"; "submit" / "send" / "save" +- visible text — `text()` (or `.` for descendants) +- attributes — `@class`, `@aria-label`, `@title`, `@data-action`, `@id` +- **synonyms** — edit/change/modify; delete/remove/trash; submit/send/save -Wrap each match with `translate(...)` for case-insensitive `contains`: +Case-insensitive via `translate(...)`: ``` //*[contains(translate(., 'EDIT', 'edit'), 'edit') @@ -53,39 +57,44 @@ Wrap each match with `translate(...)` for case-insensitive `contains`: or contains(translate(., 'CHANGE', 'change'), 'change')] ``` -Then iterate `toSimplifiedHTML(150)` over the result, review the candidates, pick the right one, and commit a stable locator built from its discriminating attribute or text — or `toAbsoluteXPath()` if nothing else is stable. +Then iterate `toSimplifiedHTML(150)` over the results, pick the right candidate, commit a stable locator from its discriminating attribute or text. ## Pick a stable locator -Once the right element is identified, choose the locator with the highest semantic value that is still unique: +Two decisions in order: **which region scopes the lookup** (context), **what identifies the element inside it**. Region first keeps the identifier short and semantic — the discriminator found during disambiguation belongs in the context argument: + +```js +I.click('Edit user', '.user-row') // ✅ region + what the user sees +I.click('#user-row-42 button.edit') // ❌ same element, brittle, unreadable +``` + +Stable regions: landmarks (`nav`, `main`, `{ role: 'dialog' }`), app-shell containers (`.sidebar`, `.toolbar`, `.modal`), rows/cards identified by data via `locate(...)`. Identifier priority (full rationale: `codeceptjs-fundamentals` § Locators): -1. ARIA — `{ role: 'button', name: 'Edit user' }`. Survives CSS refactors. -2. Visible label / semantic text — `'Edit user'`. Easy to read. -3. `[data-testid="edit-user"]` if the team uses test attributes. -4. Composed CSS with a stable parent — `#user-row-42 button.edit`. -5. `toAbsoluteXPath()` from the candidate review — last resort; flag for the team to add a `data-testid`. +1. Visible label / accessible name — plain string already matches `aria-label`; don't expand to `{ css: '[aria-label="..."]' }` +2. ARIA role when ambiguous within context or role is part of the check +3. `$name` via `customLocator` when team test attributes exist +4. Composed CSS, still scoped: `I.click('button.edit', '#user-row-42')` +5. `toAbsoluteXPath()` — last resort; flag the team to add a `data-testid` -Don't commit a locator you didn't verify. After picking, run `I.seeElement()` (or `grabWebElement()`) through MCP `run_code` to confirm it matches exactly one element. +**Never commit an unverified locator** — confirm via `run_code` (`I.seeElement(loc, context)` or `grabWebElement(loc)`) that it matches exactly one element. ## Common patterns -- **Strict mode found 2 matches** — `grabWebElements('Save')`, `toSimplifiedHTML(200)` each, find a discriminator (parent class, `data-*`, surrounding text). -- **Button rendered but doesn't act** — `grabWebElement('Submit')`, then `isEnabled()` + `getBoundingBox()` — disabled? offscreen? zero-sized? -- **Wrong row in a list** — `grabWebElements('.user-row')`, `getText()` per row, identify, then `getAttribute('data-id')` for a stable hook. -- **Inside an iframe** — `(await I.grabWebElement('iframe.editor')).inIframe(async (body) => body.$('button'))`. -- **No semantic name on the element** — `toAbsoluteXPath()` for now; flag the team to add a `data-testid`. +- Strict mode 2+ matches → `grabWebElements('Save')` + `toSimplifiedHTML(200)` each, find discriminator, pass as context: `I.click('Save', '.modal')` +- Button rendered but doesn't act → `grabWebElement('Submit')` + `isEnabled()` + `getBoundingBox()` — disabled? offscreen? zero-sized? +- Wrong row in a list → `grabWebElements('.user-row')`, `getText()` per row to identify, `getAttribute('data-id')` for stable hook +- Inside iframe → `(await I.grabWebElement('iframe.editor')).inIframe(async (body) => body.$('button'))` ## Things to avoid -- Choosing a locator without seeing the candidates first — you'll guess wrong and the test will be flaky. -- Committing `toAbsoluteXPath()` when a semantic locator is right there. -- Ignoring the screenshot — "element exists in HTML" doesn't mean "user can see it". -- Reaching for `usePlaywrightTo` / `useWebDriverTo` when WebElement methods cover the case (cross-helper code is preferred). +- Choosing a locator without seeing candidates first. +- Committing `toAbsoluteXPath()` when a semantic locator is available. +- Committing unscoped locators where a context keeps them short. +- Ignoring the screenshot — "exists in HTML" ≠ "user can see it". +- `usePlaywrightTo` / `useWebDriverTo` when WebElement methods cover it. -## Pointers +## Related skills -- `node_modules/codeceptjs/docs/web-element.md` — full WebElement API -- `node_modules/codeceptjs/docs/locators.md` — locator strategies and priorities -- `node_modules/codeceptjs/docs/element-selection.md` — `step.opts({ elementIndex })`, strict mode -- `node_modules/codeceptjs/docs/mcp.md` — MCP tool list -- `codeceptjs-fundamentals` skill — locator priority and the `await` rule +- `codeceptjs-fundamentals` — locator priority, await rule +- `writing-codeceptjs-tests` — invokes this during Mode B exploration +- `debugging-codeceptjs-tests` — invokes this for live inspection; offline variant via `codeceptq` diff --git a/plugins/codeceptjs/skills/codeceptjs-fundamentals/SKILL.md b/plugins/codeceptjs/skills/codeceptjs-fundamentals/SKILL.md index 497c82f..8048cd3 100644 --- a/plugins/codeceptjs/skills/codeceptjs-fundamentals/SKILL.md +++ b/plugins/codeceptjs/skills/codeceptjs-fundamentals/SKILL.md @@ -1,158 +1,196 @@ --- name: codeceptjs-fundamentals -description: "Run first when working with any CodeceptJS 4 project. Compact primer on the internals you must know — configuration, the `I` actor and helpers, the DI container and `inject()`, custom helpers (and the rule that `I` is unreachable from inside one), plugins as hook listeners, and the `await` rule. Then runs a four-step discovery against this project: `codeceptjs check` to verify the setup loads, read the config, run `codeceptjs list` to enumerate available `I.*` actions, run `codeceptjs dry-run` to enumerate existing tests — and reports which helper, plugins, env switching, page objects, custom actions, and test suites are actually active. Other CodeceptJS skills depend on this output." +description: > + Run first when working with any CodeceptJS 4 project — before writing, + debugging, refactoring, or migrating tests. Teaches the framework's + non-obvious rules and runs four-step discovery (`check` → read config → + `list` → `dry-run`) reporting which helpers, plugins, page objects, custom + actions, and tests are active. Other CodeceptJS skills depend on this output. --- # CodeceptJS Fundamentals -Two jobs: teach the concepts you need to read CodeceptJS code without making things up, and report what *this* project has configured. Do both, in order. +Two jobs, in order: learn the rules below, then discover what *this* project has configured. ---- +## Gate -## Concepts +- CodeceptJS 4 is **ESM/TypeScript only**; tests, configs, page objects, helpers use `import`/`export`. +- No `"type": "module"` in package.json → add it before anything else. +- TypeScript: config `codecept.conf.ts`, TS loader entry in `require: [...]`. +- Project on 3.x or CommonJS (`require()`, removed plugins like `autoLogin`, helper `Nightmare`) → stop, recommend `migrate-codeceptjs-4`. Don't patch files piecemeal — migration is whole-project. -### Module system -CodeceptJS 4 is **ESM and TypeScript only**. Tests, configs, page objects, and helpers use `import`/`export`; `package.json` **must** have `"type": "module"` — if it isn't there yet, add it before doing anything else (without it, every `.js` file is parsed as CommonJS and imports fail). **TypeScript** is first-class: name the config `codecept.conf.ts`, add a loader entry like `require: ['tsx/cjs']` (or `ts-node/register`), and write tests as `.ts` files. +## Main rule -If the project is on **CodeceptJS 3.x or still uses CommonJS** (`require()` / `module.exports`, no `"type": "module"`, removed helpers/plugins like `autoLogin` or `Nightmare`), stop here and run the **`migrate-codeceptjs-4`** skill — it walks the full upgrade path (Node bump, ESM conversion, helper/plugin replacements, AI/Zod/effects API changes, `noGlobals`, dependency bumps, verify). Don't try to half-fix individual files; the migration is a whole-project change. +- Tests are written from the user's perspective: a linear scenario of actions, readable as prose. + - Good: `I.click('Login')`, `I.fillField('Email', ...)`, `I.see('Welcome')` +- **Tests are declarative, helpers imperative — recommended layering:** + - Scenario shows *what* the user does via `I.*`; implementation details live below + - Low-level access (`this.helpers['Playwright'].page`, fetch, filesystem) works fine inside a Scenario, but it's recommended to push it into a helper and expose one `I.*` action instead +- Keep tests short: repeated sequences → actor method, page object, or step object. +- Prefer semantic locators over selectors so tests survive markup churn. -### Configuration -`codecept.conf.{js,ts,mjs,cjs}` at the repo root. Top-level keys: `helpers`, `plugins`, `include`, `ai`, `bootstrap`/`teardown`, `tests`, `output`, `timeout`. TypeScript configs declare a loader in `require: [...]` (`tsx/cjs`, `ts-node/register`, `ts-node/esm`). Multiple env-specific files (`codecept.ci.conf.js`, …) are selected via `--config `. The `@codeceptjs/configure` package mutates the resolved config at load time (`setHeadlessWhen`, `setBrowser`, `setCommonPlugins`, `setWindowSize`) — static fields can lie until you grep for that import. +## Where things go (recommended placement) -### `I` and helpers -`I` is the actor. Every `I.(...)` is dispatched to whichever active helper provides that method. Built-in helpers contribute different surfaces: web (Playwright, Puppeteer, WebDriver — overlapping core actions plus helper-specific extras), API (REST, GraphQL), AI, mobile (Appium), utility (FileSystem). The active helpers are exactly the keys under `helpers` in config. +- Site-wide actions (`login`, dropdowns, rich text editors) → **actor file** (custom steps) +- Page/screen actions + locators → **page object**; SPA screen = one page object +- Site-wide widgets (nav, modals, datepickers) → **page fragments / component objects** +- Low-level driver access (DB connections, email, filesystem, complex mouse) → **helper** +- Data creation/cleanup via API → **data objects** (REST/GraphQL helper + `_after()` cleanup), or `ApiDataFactory` (`I.have(...)`) +- Don't overengineer: no page object until an abstraction is reused across tests. -### `inject()` and the DI container -Everything testable lives in a global container — the actor, every helper, every page object listed in `include`, every custom step module, every support object. Inside a Scenario you destructure from the test signature: `Scenario('...', ({ I, loginPage }) => { ... })`. Inside a *file* (page object class, data factory, custom helper module) call `const { I } = inject()` once at the top to pull what you need from the container. The names available are exactly the keys in `include`. +## Shortcuts -### Custom helpers -Custom helpers extend `Helper` (from `codeceptjs`) and contribute new `I.` calls. They register under `helpers` in config alongside built-ins. **You cannot call `I.*` from inside a custom helper — `I` does not exist in helper scope.** To compose with another helper, reach for it via `this.helpers['']` (e.g. `this.helpers['Playwright'].page` or `this.helpers['REST'].sendGetRequest(...)`). Helpers exist to expose new low-level capabilities; tests stay in the `I.*` vocabulary. +- Login needed → `autoLogin` plugin or actor method, not inline steps per test +- Test data → create via API before the test, not through the UI +- Long test → break into several; long tests are fragile and hard to follow +- Optional UI element / conditional flow → `await tryTo(...)` instead of `if (await I.grab...)` — keeps scenarios linear -### Plugins and hooks -Plugins are event listeners. CodeceptJS emits lifecycle events on a global dispatcher; any plugin can subscribe. Events include `suite.before`/`after`, `test.before`/`started`/`passed`/`failed`/`after`, `step.before`/`started`/`passed`/`failed`/`after`, `hook.passed`/`failed`, `multiple.before`/`after`. Plugins react — taking screenshots, retrying, healing, writing artifacts, pausing. Built-ins live under `node_modules/codeceptjs/lib/plugin/`; the full event list is in `node_modules/codeceptjs/lib/event.js`. A plugin is registered under `plugins` with `enabled: true`. `setCommonPlugins()` from `@codeceptjs/configure` enables a recommended bundle silently. +## Architecture -### Plugins worth knowing -- `retryFailedStep` — re-runs a transient action failure -- `screenshot` — saves a screenshot when a step matches the trigger (default `on: 'fail'`); set `slides: true` to also produce a `output/records.html` slideshow (replaces the old `stepByStepReport`) -- `pageInfo` — dumps URL, HTML, console output on failure -- `auth` — session reuse for login (see the `codeceptjs-auth` skill) -- `aiTrace` — per-step screenshots/HTML/ARIA/console for AI debugging (default `on: 'step'`; set `on: 'fail'` to capture only failures) -- `pause` — interactive pause (replaces `pauseOn` / `pauseOnFail`; default `on: 'fail'`) -- `heal` — AI-suggested fixes for broken action steps (disabled in `--debug` mode) -- `screencast` — records a video / animated frames of the run (replaces `subtitles`) -- `browser` — CLI-only override of browser helper config; see the next section +- **Config**: `codecept.conf.{js,ts,mjs,cjs}` at repo root; multiple files selected via `--config `. +- **Helpers execute; `I` delegates.** Every `I.` is routed to whichever active helper implements it (Playwright, WebDriver, Puppeteer, Appium share one API surface). Active helpers = keys under `helpers`. Tests call the actor, never the engine — backends stay swappable. +- **DI container** — why it exists: + - Everything shared (actor, helpers, page objects, support objects) registers under one container; `include` maps names → modules + - Classes are auto-instantiated by the container — no `new`, no manual wiring + - **`inject()` returns lazy proxies**: destructuring at module top resolves at call time, so circular page-object references work where plain `import` would give `undefined` + - Access: destructure in Scenario signature (`Scenario('...', ({ I, loginPage }) => ...)`) or `const { I } = inject()` once per file +- **Custom helpers** extend `Helper`, register under `helpers`, add new `I.*` methods. + - **`I` does not exist inside a helper.** Compose via `this.helpers['']` (e.g. `this.helpers['Playwright'].page`, `this.helpers['REST'].sendGetRequest(...)`). +- **Plugins** are event listeners on lifecycle events (`suite.*`, `test.*`, `step.*`, `hook.*`, `multiple.*`). Full list: `node_modules/codeceptjs/lib/event.js`. Register under `plugins` with `enabled: true`. -### Running plugins from the CLI -Plugins are normally enabled in `codecept.conf.{js,ts}`, but any plugin can be turned on or reconfigured for a single run via `-p ` on the runner. Args chain with `:` (or `;` inside one arg): +## Config mutation trap -```bash -npx codeceptjs run -p aiTrace # enable aiTrace for this run -npx codeceptjs run -p screenshot:on=step # screenshot every step -npx codeceptjs run -p pause:on=file:path=tests/login_test.js;line=43 -npx codeceptjs run -p browser:hide:browser=firefox:windowSize=1280x800 -``` +- `@codeceptjs/configure` mutates resolved config at load time (`setHeadlessWhen`, `setBrowser`, ...). Static values can lie — grep for its import before trusting `show:` / `browser:` fields. +- `setCommonPlugins()`: **enables** `retryFailedStep` + `screenshot`; **registers** (off until `-p`) `pause`, `browser`, `aiTrace`, `heal`. + +## Plugins worth knowing + +- `retryFailedStep` — retries transient step failures +- `screenshot` — screenshots on failure; `slides: true` → `output/records.html` slideshow +- `pageInfo` — dumps URL/HTML/console on failure +- `auth` — session reuse for login (see `codeceptjs-auth` skill) +- `aiTrace` — per-step screenshots/HTML/ARIA/console for AI debugging +- `pause` — interactive pause +- `heal` — AI-suggested fixes for broken steps (off in `--debug`) +- `screencast` — video of the run +- `customLocator` — maps `$name` prefix to team's test attribute (`data-testid`, `data-qa`) +- `browser` — CLI-only override of browser helper config (see below) -`screenshot`, `pause`, `aiTrace`, and `heal` share a unified **`on=` parameter** that picks when they fire: +Note: `tryTo`, `retryTo`, `eachElement` are not plugins in 4.x — import them from `codeceptjs/effects`. -| `on=` value | Fires when | Extra args | -|---|---|---| -| `fail` | a step fails (default for screenshot / pause / heal) | — | -| `step` | every step (default for aiTrace) | — | -| `test` | after each test | — | -| `file` | execution reaches a file/line | `path=[;line=]` | -| `url` | browser URL matches a pattern | `pattern=` (`*` wildcards) | +## Plugins from CLI -The **`browser` plugin** is CLI-only and overrides the active browser helper without touching the config file — useful for one-off env variants and CI matrix legs: +Any plugin can be enabled/reconfigured per-run with `-p `, args chained with `:`: -```bash -npx codeceptjs run -p browser:show # force visible -npx codeceptjs run -p browser:hide # force headless -npx codeceptjs run -p browser:browser=firefox # switch browser -npx codeceptjs run -p browser:windowSize=1024x768 -npx codeceptjs run -p browser:hide:browser=webkit:windowSize=800x600 +```sh +npx codeceptjs run -p aiTrace # enable for this run +npx codeceptjs run -p screenshot:on=step # reconfigure inline +npx codeceptjs run -p pause:on=file:path=tests/login_test.js;line=43 ``` -Requires `@codeceptjs/configure` installed. It translates `browser=` per helper (Puppeteer's `product`, Playwright's `browser`) and injects `--headless` into WebDriver capability args when toggling `hide`. +- `screenshot`, `pause`, `aiTrace`, `heal` share an `on=` trigger: `fail` (default except aiTrace) | `step` | `test` | `file:path=...;line=N` | `url:pattern=` +- `browser` plugin overrides without touching config — CI matrix legs, one-off env variants: + - `-p browser:hide` / `-p browser:show` / `-p browser:browser=firefox` / `-p browser:windowSize=1280x800` + - Requires `@codeceptjs/configure` -### Test file structure -One `Feature(...)` per file with one or more `Scenario(...)` blocks inside it. CodeceptJS does **not** allow nested suites or multiple Features in the same file. Hooks: `Before`, `After`, `BeforeSuite`, `AfterSuite`, plus `Fail((test, err) => { ... })` for failure-only cleanup. Page objects can expose `_before`, `_after`, `_afterSuite` lifecycle methods so per-page setup lives next to the page. +## Effects (`codeceptjs/effects`) -### Locators -Most actions accept a locator as a plain string (semantic — visible text, label, placeholder, `name`) or an object (`{ css }`, `{ xpath }`, `{ role, name }`, `{ id }`, `{ aria }`). Prefer ARIA `{ role, name }` for resilience to markup changes; semantic strings for prototyping; CSS / XPath as fallback. The `locate(...)` builder composes complex queries (`.withClass`, `.withText`, `.inside`, `.and`, `.andNot`). Almost every action method takes an optional context arg that narrows the search to a subtree: `I.click('Save', '.modal')`. +Flow-control functions imported from `codeceptjs/effects`. In 4.x these are no longer plugins/globals. -### Auto-waiting -Action methods (`click`, `fillField`, `selectOption`, …) automatically wait for the element to exist and become interactable before acting. Explicit `I.waitFor*` calls are needed only when the next condition isn't tied to an interaction — a modal appearing after a network call, a spinner disappearing, a value updating. Avoid `I.wait(N)` (raw seconds) unless nothing else fits. +- **`tryTo(() => ...)`** — runs steps that may fail without stopping the test; returns `boolean`. + - **Prefer `tryTo` over `if`:** scenarios should stay linear — instead of branching on a grabbed value to decide whether a UI state exists, attempt the optional steps and branch on the boolean result: + ```js + const banner = await tryTo(() => { I.see('Cookie banner'); I.click('Accept cookies') }) + if (!banner) I.say('No cookie banner') + ``` + - Auto-retries are disabled inside `tryTo` blocks. +- **`retryTo(() => ..., maxTries, pollInterval = 200)`** — retries a step block until it succeeds (flaky elements, animations); callback receives the current attempt count. +- **`hopeThat(() => ...)`** — soft assertions (see Assertions); end with `hopeThat.noErrors()`. +- **`within(locator | { frame }, fn)`** — scopes resolution to subtree or iframe; can return values (`await`). Prefer the context parameter of individual actions (`I.click('Save', '.toolbar')`) when possible — reserve `within` for genuinely scoped blocks. -### Assertions -CodeceptJS ships built-in browser assertions: `I.see`, `I.dontSee`, `I.seeElement`, `I.dontSeeElement`, `I.seeInCurrentUrl`, `I.seeInTitle`, `I.seeInField`, `I.seeNumberOfElements`, `I.seeCookie`, `I.seeCheckboxIsChecked`, etc. Use these instead of an external `expect()` library — they produce clear failure messages and integrate with the recorder. For non-DOM assertions, use `grab*` plus any assertion library: `const title = await I.grabTitle(); expect(title).toEqual('My App')`. +All effects return Promises — `await` them. -### `await` inside tests -CodeceptJS queues steps onto an internal recorder; the framework chains them, you do not. **Use `await` only when you need a return value** — `await I.grabTextFrom(...)`, `await I.grabCookie(...)`, or when calling a user-defined `async` function. Plain action steps (`I.click`, `I.fillField`, `I.see`) do not need `await`. Same rule inside `within(...)` and `pause()` callbacks. Sprinkling unnecessary `await` doesn't break anything, but signals you don't trust the recorder. +## Element-based API (`codeceptjs/els`) -### `secret()` for sensitive values -Wrap passwords, tokens, API keys so they're masked in logs, step output, and trace artifacts: `I.fillField('Password', secret(process.env.PASSWORD))`. Imported from `codeceptjs`. Use anywhere a value would otherwise leak through verbose output, trace files, or AI prompts. +Hybrid style: mix `I.*` with direct element access. Import `{ element, eachElement, expectElement, expectAnyElement, expectAllElements } from 'codeceptjs/els'`. -### Sessions and `within` -- `session(name, fn)` runs `fn` in a parallel browser context — for multi-user Scenarios (chat, multi-tenant). Combined with the `auth` plugin, each session can log in as a different role. -- `within(locator, fn)` scopes locator resolution inside `fn` to the subtree under `locator`. `within({ frame: '#editor' }, fn)` switches into an iframe for the callback. Both can return values (`await within(..., () => I.grabTextFrom(...))`). +- `element(locator, async el => { ... })` — scoped access to one element; chain `el.$(locator)` into children without re-querying +- `eachElement(locator, async (el, index) => ...)` — iterate collections +- `expectElement` / `expectAnyElement` / `expectAllElements(locator, fn)` — custom conditions +- Elements are `WebElement` wrappers — same API on all helpers: `getText()`, `getAttribute()`, `isVisible()`, `isEnabled()`, `getBoundingBox()`, `exists()`, `$$()` +- Optional purpose string improves debug logs: `element('verify discount applied', '.price', ...)` +- Use when built-ins don't cover it: collections, layout checks (`getBoundingBox`), per-element loops, chaining ops on one element. Prefer `I.*` for readability otherwise. -### Parallel runs -`npx codeceptjs run-workers ` splits Scenarios across N Node worker threads; results aggregate in the main process. The config can also describe **profiles** (different browsers, viewports, environments) via the `multiple` block; launch with `npx codeceptjs run-multiple `. +## Writing tests ---- +- Structure: one `Feature(...)` per file, one or more `Scenario(...)` inside. No nested suites, no multiple Features per file. +- Hooks: `Before`, `After`, `BeforeSuite`, `AfterSuite`, `Fail(...)`. +- Page object lifecycle hooks: `_before()` (lazy, once per test, on first use), `_after()` (skipped if unused), `_beforeSuite()`, `_afterSuite()`. +- **`await` required for**: `grab*` methods, imported functions, page-object methods containing async ops (elsewhere: unhandled rejections). Never for plain action steps — the recorder chains them. +- Secrets: `I.fillField('Password', secret(process.env.PASSWORD))` — masks logs, traces, AI prompts. Import from `codeceptjs`. +- Sessions: `session(name, fn)` — parallel browser context for multi-user Scenarios (chat, multi-tenant). -## Discover this project +## Locators -Four steps, in order. Don't skip — guesses about helpers, custom actions, or what tests exist will be wrong as often as they're right. +- **ARIA locators are strongest** — resilient to CSS refactors, describe what the user sees: + - `I.click({ role: 'button', name: 'Save' })` +- Actions accept plain strings (visible text, label, placeholder, `name`, `aria-label`) or objects (`{ css }`, `{ xpath }`, `{ id }`). +- Plain string already matches `aria-label` — no `'aria-label=...'` prefix needed. +- **Pass context as last argument** — scoped semantic locator beats long unscoped one: + - `I.click('Save', '.toolbar')` not `I.click('#toolbar .btn-save')` +- Avoid style-based class names (`.bg-green`); prefer semantic ones (`.btn-save`). +- `data-testid`/`data-qa` apps → enable `customLocator`, write `$name`. +- No semantic name fits → `locate(...)` builder (`.withClass`, `.withText`, `.inside`, `.and`): `locate('.button').withText('Click me')`. -### 1. Verify the setup loads +## Waiting -```sh -npx codeceptjs check -c # validates config, container, helpers, plugins, page objects, hooks, tests, defs -``` +- Action steps auto-wait for existence + interactability. Add explicit `waitFor*` only when the condition isn't tied to an interaction (modal after network call, spinner hiding). +- Avoid `I.wait(N)` — last resort. -Each item prints a pass/fail line, so the output doubles as a quick inventory of what the project has wired up. If anything fails here, fix it before running `list` or `dry-run` — a broken helper or unresolved page object will distort their output. Skipping this step also means you won't notice a missing dependency, an `auth` plugin pointed at a non-existent login route, or a custom helper that throws at construction. +## Assertions -### 2. Read the active config +Built-in browser assertions come first: `I.see`, `I.seeTextEquals`, `I.seeElement`, `I.seeInField`, `I.seeNumberOfElements`, `I.seeInCurrentUrl` (+ `dontSee*` counterparts). Clear failures, recorder-integrated. `see` matches *visible* text; hidden DOM content needs `seeInSource` / `seeElementInDOM`. -Open `codecept.conf.{js,ts,mjs,cjs}` (resolve via `package.json` scripts and CI workflows if multiple files exist — note the path; you'll pass it to `-c` in steps 3 and 4). Extract: which helper(s) and any non-default behaviour (browser, strict, navigation, base URL, viewport, env-driven values); which plugins (incl. anything `setCommonPlugins()` injects); AI provider + the env var its key requires; how environments are selected (`--config` vs `process.env.*` branching, plus any `setHeadlessWhen`-style mutations); page object names from `include`; any custom helpers (entries pointing at local files). +For what built-ins don't cover, in order of preference: -### 3. List available actions +1. **Reusable custom assertion** in a helper — `I.seeTableIsOrdered('Price', 'desc')`; name positives `see*`, negatives `dontSee*`; use `codeceptjs/assertions` inside, never raw `throw new Error()` +2. **ExpectHelper** (`@codeceptjs/expect-helper`) — chai matchers on `I`: `I.expectEqual`, `I.expectDeepEqualExcluding`, `I.expectMatchesPattern`, `I.expectJsonSchema`; appears in step log like other steps +3. **`codeceptjs/assertions`** directly — dependency-free factories: `equals(subject).assert(actual, expected)` / `.negate(...)`; failure messages match `I.see` formatting +4. **Any library** on grabbed data (`grab*` always needs `await`) — chai/jest/`node:assert`; fails the test but won't show as a step -```sh -npx codeceptjs list -c # every I., grouped by helper, with signature -npx codeceptjs list --docs -c # adds JSDoc + docs/webapi/* prose under each action -npx codeceptjs list --action -c # single action; I. prefix optional; implies --docs -``` +Soft assertions: `hopeThat(() => I.see(...))` from `codeceptjs/effects` — logs each failure and continues; end with `hopeThat.noErrors()` to fail if any were recorded. -Run `list` against the discovered config before suggesting any method — especially in projects with custom helpers, where the available `I.*` surface differs from the built-in catalog. The CodeceptJS MCP server's `list_actions` tool returns the same data programmatically. +## Parallel runs -### 4. List existing tests +- `run-workers ` — splits Scenarios across worker threads +- `run-multiple ` — profiles via `multiple` block in config (browsers, viewports) -```sh -npx codeceptjs dry-run -c # suite + test names that the config would load -npx codeceptjs dry-run --steps -c # also prints queued I.* steps inside each test -npx codeceptjs dry-run --grep "@smoke" -c # filter by name; --features / --tests narrow file kind -npx codeceptjs dry-run --debug --grep '' --numbers --no-ansi -c # numbered steps, no ANSI -``` +## Config organization (recommended) -`dry-run` walks the test files the active config picks up and prints them without executing — confirming both **which tests exist** and (with `--steps`) **what each Scenario would do** before any browser spins up. +- Multiple config files per environment (`codecept.conf.js`, `codecept.ci.conf.js`, ...); share parts via modules in a `config/` dir +- `.env` files + `dotenv` for secrets/env-specific values +- Bulk-register page objects/components by spreading exported maps into `include` +- Pass data from config/bootstrap into tests via `codeceptjs.container.append({ testUser })` — injectable by name -`--numbers` (paired with `--debug`, `--steps`, or `--verbose`) prefixes each leaf step with a per-test 1-based index. The numbering matches the `pauseAt: N` parameter on the MCP `run_test` tool — so this is the canonical way to discover step indices for programmatic breakpoints. `--no-ansi` strips colors / ANSI escapes so the output is clean for LLM consumption or piping to other tools. +## Discover this project -For Gherkin step definitions specifically, `npx codeceptjs gherkin:steps -c ` lists registered step patterns. +In order; skipping steps produces wrong guesses: -## Report +1. **Verify setup loads**: `npx codeceptjs check -c ` — validates everything; output doubles as inventory. Fix failures before continuing. +2. **Read the active config**: helpers (+ browser/baseURL/viewport/env-driven values), plugins (incl. anything `setCommonPlugins()` injects), AI provider + required env var, env selection mechanism, page objects from `include`, custom helpers. +3. **List actions**: `npx codeceptjs list -c ` (`--docs` adds JSDoc; `--action ` for one). The actual `I.*` surface differs from built-ins when custom helpers exist — always check before suggesting a method. +4. **List tests**: `npx codeceptjs dry-run -c ` — `--steps` shows queued actions, `--grep` filters, `--numbers` gives per-test step indices matching MCP `pauseAt`. + - ⚠ `dry-run --grep` and `run --grep` do **not** select the same set (4.1.0): `run --grep` matches `Feature` + `Scenario`, `dry-run --grep` matches the Scenario title only. `dry-run --grep 'Dialogs'` lists 0 tests where `run --grep 'Dialogs'` executes all 11. Never size a run from a dry-run's grep, and target a whole Feature by file path (`run tests/foo_test.ts`) when the selection must be exact. + +Gherkin projects: `npx codeceptjs gherkin:steps -c `. -Short prose summary covering the items above. Flag env-driven values explicitly — don't claim a fixed value when it's `process.env.BROWSER || 'chromium'`. Flag conflicts (static `show: true` overridden by `setHeadlessWhen(CI)`; `auth` configured but the credential env vars are missing from the current shell or `.env.example`). If no config exists at the repo root and no `--config` is referenced anywhere, recommend `npx codeceptjs init .` and stop. **If the project is on CodeceptJS 3.x or CommonJS, recommend the `migrate-codeceptjs-4` skill and stop** — discovery output for a pre-4 project will misrepresent the available APIs. +Reference docs live under `node_modules/codeceptjs/docs/` — read them instead of guessing APIs. + +## Report -## Pointers +Short prose summary. Must include: -- `node_modules/codeceptjs/docs/configuration.md` — config reference -- `node_modules/codeceptjs/docs/typescript.md` — TS loader options -- `node_modules/codeceptjs/docs/helpers.md` — helper concepts and method catalogs -- `node_modules/codeceptjs/docs/custom-helpers.md` — writing your own -- `node_modules/codeceptjs/docs/plugins.md` — plugin authoring + built-ins -- `node_modules/codeceptjs/docs/hooks.md` — suite/test/step hook semantics -- `node_modules/codeceptjs/lib/event.js` — every event the dispatcher emits -- `@codeceptjs/configure` (npm) — the mutator API surface +- Env-driven values flagged as env-driven (`process.env.BROWSER || 'chromium'`, not just `'chromium'`) +- Conflicts flagged (static `show: true` vs `setHeadlessWhen(CI)`; `auth` configured but credential env vars missing) +- No config at root and no `--config` referenced → recommend `npx codeceptjs init .`, stop +- 3.x/CommonJS detected → recommend `migrate-codeceptjs-4`, stop diff --git a/plugins/codeceptjs/skills/codeceptjs-run-analysis/SKILL.md b/plugins/codeceptjs/skills/codeceptjs-run-analysis/SKILL.md index b12888e..abef7b7 100644 --- a/plugins/codeceptjs/skills/codeceptjs-run-analysis/SKILL.md +++ b/plugins/codeceptjs/skills/codeceptjs-run-analysis/SKILL.md @@ -1,90 +1,83 @@ --- name: codeceptjs-run-analysis -description: Use after running CodeceptJS tests with the `aiTrace` plugin enabled — analyse the trace artifacts (trace.md, per-step HTML/ARIA/screenshots, console logs) via bash tools. Toolkit, not a workflow — use cases include verifying a fix, clustering errors across a CI fail-storm, diagnosing flakiness across reruns, or investigating a single failure. Other skills (`writing-codeceptjs-tests`, `debugging-codeceptjs-tests`, `refactoring-codeceptjs-tests`) invoke this whenever a run has happened and needs to be reviewed. Trigger on phrases like "what failed", "analyse the run", "cluster these errors", "is it flaky", "did the fix hold". +description: > + Use after running CodeceptJS tests with the `aiTrace` plugin enabled and the + results need review — verify a fix held, investigate a single failure, cluster + errors across a CI fail-storm, diagnose flakiness across reruns. Invoked by + other CodeceptJS skills whenever a run has happened. Trigger on "what failed", + "analyse the run", "cluster these errors", "is it flaky", "did the fix hold". --- # CodeceptJS Run Analysis -After `npx codeceptjs run`, the trace artifacts land in `output/`. This skill is the playbook for reading them efficiently — pulling out the right step, the right file, and the right slice of a giant HTML snapshot using bash tools rather than re-running the test through MCP. +After `npx codeceptjs run`, trace artifacts land in `output/`. This skill reads them efficiently — right step, right file, right slice of a giant HTML snapshot — via bash tools rather than re-running through MCP. -There's no single end goal — pick the use case that matches the situation. The foundations (where artifacts live, how to read them, what tools to reach for) apply to all of them. +No single end goal: pick the use case matching the situation. The foundations apply to all of them. ## Foundations -### Ensure aiTrace is on -The whole skill leans on `output/trace__/trace.md` and its sibling artifacts. Confirm `plugins: { aiTrace: { enabled: true } }` in the active config (run `codeceptjs-fundamentals` if you don't already know). Without aiTrace there are screenshots and `pageInfo` dumps at most — useful but partial; suggest enabling and re-running before deep analysis. +- **aiTrace must be on.** Everything leans on `output/trace__/trace.md`. Confirm in the active config (run `codeceptjs-fundamentals` if unknown). Without it there are only screenshots and `pageInfo` dumps — suggest enabling and re-running before deep analysis. +- `run_step_by_step` is interactive only; `aiTrace` is the sole source of per-step files. Ad-hoc `run_code` / `snapshot` still produce single-shot bundles under `output/trace_run_code_*` / `output/snapshot_*`. +- **Locate traces**: reruns create new dirs — when unclear, most recent wins (`ls -dt output/trace_*`). +- **Read trace.md first** — it's the index linking each step to its artifacts. + - Focus on the failed step; none marked → last step in the trace. + - Multiple failures marked → the **first** is usually the cause; the rest cascade. -The MCP `run_step_by_step` tool is now interactive (pauses after every step; agent advances via `continue`) and no longer auto-writes a per-run artifact bundle. `aiTrace` is the only source of per-step trace files now — for ad-hoc `run_code` and `snapshot` calls you still get a single-shot artifact set under `output/trace_run_code_*` / `output/snapshot_*`. - -### Locate traces -Trace directories are `output/trace__/`; reruns produce a new dir per run. **When the right trace isn't obvious, sort by modification time and take the most recent** (`ls -dt`) — almost always the run you just kicked off. - -### Read trace.md first -trace.md is the index — each step block links to its screenshot/HTML/ARIA/console. **Focus on the step marked failed; if no failure is explicitly marked, jump to the last step in the trace** — execution usually stops where the failure happened. When more than one step shows failed, the **first** is usually the cause; the rest are cascading side effects. - -### Open artifacts in the right order -For the focus step `NNNN_`: +### Artifact order for the focus step (`NNNN_`) | Artifact | When | |---|---| -| `NNNN_*_aria.txt` | First read — leaner than HTML, structured, easy to scan for duplicates | +| `NNNN_*_aria.txt` | First read — lean, structured, easy to scan for duplicates | | `NNNN_*_screenshot.png` | Visual confirmation — layout, animation, "rendered but wrong" | -| `NNNN_*_page.html` | Only when ARIA is missing context. **Use `grep`, not `cat`** | -| `NNNN_*_console.json` | JS errors, 4xx/5xx, deprecation warnings explaining "vanished" elements | -| `NNNN_*_storage.json` | Cookies + localStorage at this step. First place to look when auth is the suspected culprit. | +| `NNNN_*_console.json` | JS errors, 4xx/5xx, deprecation warnings explaining vanished elements | +| `NNNN_*_storage.json` | Cookies + localStorage at this step — first stop when auth suspected | +| `NNNN_*_page.html` | Last resort, and only via `grep` | ### Never read big files whole -HTML snapshots can be megabytes; `console.json` arrays can be long. -- Use `grep` to search HTML for expected elements / attributes / locator variants (text, class, aria-label, `data-*`). Line numbers and context flags make the match readable. -- Use `jq` to filter `console.json` for errors or specific event types instead of reading every entry. -- When scanning many files, `grep -l` returns filenames-only. +- HTML snapshots: search with `grep` (text, class, aria-label, `data-*`) — line numbers + context flags keep matches readable. +- `console.json`: filter with `jq`, don't read every entry. +- Scanning many files: `grep -l` for filenames only. ## Use cases ### Verify a fix held -After editing a test (e.g. via MCP) and re-running it via CLI: locate the latest trace, read trace.md, confirm no FAILED markers. Glance at `console.json` for warnings worth fixing while you're there. +Locate latest trace → read trace.md → confirm no FAILED markers. Glance at `console.json` for warnings worth fixing while you're there. ### Cluster errors across a CI batch -When many tests failed in one run: extract the failing-step lines from every `trace.md` in the batch, group by signature (`grep` + `sort` + `uniq -c`), rank by frequency. - -- **Same error in many tests = systemic.** Env var missing, auth broken, base URL wrong, deploy regression. Fix the root cause once and rerun the batch. -- **Different errors per test = local issues.** Triage one at a time. - -Pick the most-frequent root cause first; rerun to see how many tests came back along with it. - -### Diagnose flakiness across reruns -Run the same test 5–10 times. Extract which step failed in each trace and look at how the answer varies. If the failing step is the *same* but the surrounding state differs, `diff` the ARIA snapshots at that step between two runs to spot what changed. - -- **Different step each run** → timing, environment, external service. -- **Same step, different reason** → likely a missing or wrong wait (see Waiting in writing/debug skills). -- **`console.json` differs between runs** → transient backend / network errors. -- **Bounding box differs** → layout reflow or late-loading content shifting things. +Extract failing-step lines from every `trace.md`, group by signature (`grep` + `sort` + `uniq -c`), rank by frequency. +- Same error in many tests = **systemic** (env var, auth, base URL, deploy regression) — fix root cause once, rerun the batch. +- Different errors per test = local — triage one at a time. +Start with the most frequent root cause; rerun to see how many tests came back with it. + +### Diagnose flakiness +Rerun the same test 5–10 times; compare which step failed in each trace. +- Different step each run → timing, environment, external service +- Same step, different state → missing/wrong wait — `diff` the ARIA snapshots of that step between runs +- `console.json` differs between runs → transient backend/network errors +- Bounding box differs → layout reflow or late-loading content ### Investigate a single failure -Locate the trace, jump to the failed-or-last step, open ARIA + screenshot first, console.json second, HTML last (and only via `grep`). Form a hypothesis. If the trace alone isn't enough — the failure resists static analysis or the page state needs live poking — hand off to **debugging-codeceptjs-tests** for the MCP-driven loop. +Failed-or-last step → ARIA + screenshot first, console second, HTML last (grep only). Form a hypothesis. Trace not enough / page needs live poking → hand off to `debugging-codeceptjs-tests`. -## After the analysis +## After analysis -- **Systemic cause across many tests** — fix the root once (env, auth, deploy, base URL). Not per test. -- **Locator drift** — invoke **codeceptjs-exploration** to pick a new stable locator. -- **Timing / wait issue** — apply the Waiting guidance from the writing/debugging skills, replacing `I.wait(N)` with a specific `waitFor*`. -- **Failure resists static analysis** — invoke **debugging-codeceptjs-tests** for the live MCP loop. -- **Test passed cleanly** — done. Still glance at `console.json` for warnings worth fixing now. +- Systemic cause → fix root once (env, auth, deploy), not per test +- Locator drift → `codeceptjs-exploration` +- Timing/wait issue → Waiting guidance from fundamentals/writing skills; replace `I.wait(N)` with specific `waitFor*` +- Resists static analysis → `debugging-codeceptjs-tests` (live MCP loop) +- Clean pass → done, but glance at `console.json` anyway ## Things to avoid -- Reading large HTML or `console.json` files whole — search them with `grep` / `jq` instead. -- When more than one step is marked failed, stopping at the last one — the **first** failure is usually the cause and the rest are cascading. -- Triaging individual failures before clustering — fixing 12 symptoms of one bug is wasted effort. -- Drawing flakiness conclusions from a single run — needs 5+ reruns. -- Deleting `output/` mid-investigation; the artifacts are the only record of the run. +- Reading large HTML or `console.json` whole — `grep` / `jq`. +- Stopping at the last marked failure instead of the first. +- Triaging individual failures before clustering. +- Flakiness conclusions from a single run — needs 5+ reruns. +- Deleting `output/` mid-investigation. -## Pointers +## Related skills -- `node_modules/codeceptjs/docs/aitrace.md` — trace format and config knobs -- `node_modules/codeceptjs/docs/debugging.md` — verbose flags, `pause` plugin's `on=` modes -- `node_modules/codeceptjs/docs/reports.md` — alternative reporters -- `codeceptjs-fundamentals` — what's actually configured -- `codeceptjs-exploration` — when locator drift is the cause -- `debugging-codeceptjs-tests` — when post-mortem isn't enough +- `codeceptjs-fundamentals` — what's configured, aiTrace `-p` overrides +- `codeceptjs-exploration` — locator drift fixes +- `debugging-codeceptjs-tests` — live loop + offline locator resolution via `codeceptq` diff --git a/plugins/codeceptjs/skills/debugging-codeceptjs-tests/SKILL.md b/plugins/codeceptjs/skills/debugging-codeceptjs-tests/SKILL.md index 2066370..7c72475 100644 --- a/plugins/codeceptjs/skills/debugging-codeceptjs-tests/SKILL.md +++ b/plugins/codeceptjs/skills/debugging-codeceptjs-tests/SKILL.md @@ -1,234 +1,126 @@ --- name: debugging-codeceptjs-tests -description: "Use when a CodeceptJS 4 test is failing, flaky, or behaving unexpectedly — stack traces from `npx codeceptjs run`, intermittent failures, locator drift, timing issues, \"works locally fails in CI\", \"step through this test\", \"pause at step N\", \"set a breakpoint\". For AI agents the primary path is **MCP with pause** — drop a `pause()` in the test (or pass `pauseAt: N` to `run_test` for a no-edit breakpoint), inspect via `run_code` / `snapshot` against the live browser, release with `continue`. Step indices for `pauseAt` come from `npx codeceptjs dry-run --debug --grep --numbers --no-ansi`. CLI debugging (`npx codeceptjs run --debug`, `DEBUG=\"codeceptjs:*\"`) is the fallback for humans, CI repros, and framework-internal issues (recorder hangs, leaks, plugin races). Don't fix from the error message alone; capture page state and read it. Trigger on broken or flaky tests, run errors, \"why does this fail\", trace/screenshot/console mentions, breakpoint/pause/step-through requests." +description: > + Use when a CodeceptJS 4 test fails, flakes, or behaves unexpectedly. Trigger + on run errors and stack traces from `npx codeceptjs run`, intermittent + failures, locator drift, timing issues, "works locally fails in CI", "why + does this fail", trace/screenshot/console mentions, and breakpoint / + step-through / "pause at step N" requests. --- # Debugging CodeceptJS 4 Tests -Failures lie. The error usually points at a step that's a side effect of something earlier — auth expired, a frame switch missed, a network call still pending. Reproduce, capture state, and read it before fixing. +Failures lie — the error usually points at a step that's a side effect of something earlier (auth expired, frame switch missed, network call pending). Reproduce, capture state, read it, then fix. -Two paths, picked by who's driving: +## Paths -- **MCP-first (for AI agents)** — drive the test through the MCP server. In-test `pause()` and the `pauseAt: N` option on `run_test` both yield control back to the agent in-process — same `I` / browser the test is using. Inspect via `run_code` / `snapshot`, advance one step at a time via `run_step_by_step` + `continue`, release a pause via `continue`. `aiTrace` artifacts cover the prior steps. -- **CLI fallback (for humans / CI / framework internals)** — `npx codeceptjs run --debug` for verbose framework output. Escalate to `DEBUG="codeceptjs:*"` when the *framework itself* looks at fault: recorder hangs, plugin races, event leaks, "step never ran". Use this path for CI repros, headless servers, and framework-internal bugs. +- **MCP-first (AI agents)**: `run_test` / in-test `pause()` yield control to the agent on the same `I` / browser the test uses. Inspect via `run_code` / `snapshot`; step through via `run_step_by_step` + `continue`. +- **CLI fallback (humans / CI / framework internals)**: `--debug` → `--verbose` → `DEBUG="codeceptjs:*"`. The DEBUG escape hatch is only for framework-internal suspicion: recorder hangs, plugin races, event leaks, "step never ran". Namespaces: `codeceptjs:recorder`, `codeceptjs:pause`, `codeceptjs:ai`, `codeceptjs:plugin:`. -In-test `pause()` adapts to who's driving: at a TTY it opens the readline REPL; under MCP it yields control to the agent (same in-process `I` / browser); in a non-TTY non-MCP subprocess it prints a notice and resolves immediately so leftover `pause()` calls don't deadlock CI. **Adding `pause()` is now the primary MCP breakpoint** — drop it where you want to look, run via `run_test`, drive the live page through `run_code`, release with `continue`. +`pause()` adapts to who's driving: TTY → readline REPL; MCP → yields control to the agent; non-TTY non-MCP subprocess → prints a notice and resolves (no CI deadlock). ## Workflow -### 1. Read the project (fundamentals) -Run the **codeceptjs-fundamentals** skill. You need: helper, plugins on (especially `aiTrace`, `screenshot`, `pageInfo`, `retryFailedStep`, `pause`, `auth`), env vars, whether MCP is wired up. If `aiTrace` is **not** declared, add it **once** with `plugins: { aiTrace: { enabled: true } }` — most of this skill leans on its output. - -**Declare `aiTrace` once; never edit config to change its trigger.** The capture mode is controlled per-run from the CLI, exactly like the `pause` plugin: - -```bash -npx codeceptjs run -p aiTrace:on=step # persist every step (default) -npx codeceptjs run -p aiTrace:on=fail # persist only the failed step -npx codeceptjs run -p aiTrace:on=test # persist only the last step of each test -npx codeceptjs run -p aiTrace:on=file:path=tests/login_test.js;line=43 -npx codeceptjs run -p aiTrace:on=url:pattern=/checkout/* -``` - -`-p aiTrace:on=...` overrides the config-declared mode for that run only. Reach for `on=fail` to keep a CI repro lean, `on=step` while actively diagnosing. Don't flip `on:` in `codecept.conf.js` between runs — it churns the config and the change leaks into other runs. - -### 2. Reproduce minimally -Run only the failing test, with steps printed: -```bash -npx codeceptjs run --grep '' --steps -``` -Add `--config codecept.ci.conf.js` if the failure is CI-specific. Confirm reproduction before instrumenting further. - -### 3. Pick a path - -**MCP (primary for AI agents):** -- `run_test ` — runs a specific test in-process; shares `I` / browser with `run_code` and `snapshot`. Returns the JSON reporter result on completion, **or** `{ status: 'paused', pausedAfter, page, suggestions }` if the test calls `pause()` or hits the optional `pauseAt: N` breakpoint. From a paused state, drive the live page via `run_code` / `snapshot` and release with `continue`. -- `run_step_by_step ` — interactive: pauses after every step. After each `continue`, the test advances one step and re-pauses (or completes). Use when you want to watch the whole flow tick by; use `run_test` with `pauseAt: N` instead for a single targeted breakpoint. -- `continue` — releases a paused test. After `pause()` or `pauseAt`: runs to completion (or to the next `pause()`). After `run_step_by_step`: advances one step. -- `run_code ` — runs arbitrary CodeceptJS code in the live session (works fresh **and** while a test is paused — same container). Returns the **value the code produced**, captures `console.log` / `info` / `warn` / `error` / `debug` output, and saves a final-state snapshot (URL, ARIA, HTML, screenshot, storage). Use to test a locator hypothesis or grab a value at the failure point. -- `snapshot` — captures current browser state without performing any action (URL, cookies, localStorage, HTML, ARIA, screenshot, console). Use between actions when you want to reason about what to do next without re-running anything. -- `list_actions` — sanity-check that an `I.*` method exists on the active helper. - -**CLI (fallback):** -- `npx codeceptjs run --grep '' --debug` — first move when MCP isn't available. Steps + helper internals + URLs + plugin events. -- `npx codeceptjs run ... --verbose` — adds promise-queue / retry / timeout logs on top of `--debug`. -- `DEBUG="codeceptjs:*" npx codeceptjs run ...` — turns on CodeceptJS's internal debug streams. Reach for this when `--debug` doesn't explain the failure: orphaned timers, event leaks, recorder hangs, plugin races, double-emitted events, "step disappeared from the queue". Narrow with namespaces: `codeceptjs:recorder` (promise queue), `codeceptjs:pause`, `codeceptjs:ai`, `codeceptjs:plugin:`. Most user-level test failures don't need this — it's the framework-internal escape hatch. - -### 4. Set a breakpoint with `pause()` or `pauseAt` - -The MCP server installs an in-process pause handler at startup. Whenever a test running through `run_test` hits `pause()` (or completes the `pauseAt: N` step), control yields back to the agent on the same `I` / browser. There's no subprocess, no IPC, and `run_code` / `snapshot` work against the live page — exactly what a paused REPL would give you. - -Two ways to land at a breakpoint: - -- **In-test `pause()`** — drop `pause()` directly in the test where you want to look. Best when you're already editing the file or want to break inside a `within` / loop / hook. -- **`pauseAt: N` on `run_test`** — programmatic, no test edit required. Pauses after the Nth leaf step completes. - -To pick `N`, list the steps with their indices: - -```bash -npx codeceptjs dry-run --debug --grep '' --numbers --no-ansi -``` - -Output is one numbered line per leaf step (1-based, per-test). The number on the line you want to stop *after* is the value to pass as `pauseAt`. `--no-ansi` strips colors so the output is clean for parsing. - -Once paused (`{ status: 'paused', pausedAfter, page, suggestions }`): - -1. **Inspect with `run_code`** — `await I.grabCurrentUrl()`, `await I.grabWebElement(...)`, `await I.seeElement({ role: 'dialog' })`. Each call returns URL + ARIA + console + storage from the live page. -2. **Capture clean state with `snapshot`** between hypotheses — no action, just the artifact bundle. -3. **Walk earlier steps via `aiTrace`** — `output/trace__/trace.md` has the per-step state for everything that ran before the breakpoint. -4. **Release with `continue`** — runs to completion (or to the next `pause()`). For a step-by-step walk, use `run_step_by_step` instead of `run_test`; each `continue` then advances one step. - -### 5. Read the trace -Hand off to **codeceptjs-run-analysis** to walk `output/trace__/trace.md` and the per-step artifacts. Focus on the **first** failed step — late failures are usually side effects of an earlier silent miss. The run-analysis skill also covers grepping into large HTML, clustering errors across many traces, and comparing reruns when flakiness is in play. - -For locator-level questions on a saved snapshot ("would `.btn-primary` have matched here?", "is `Username` a field at step 7?") use **`codeceptq`** against the per-step `__page.html` — see the "Query trace HTML with `codeceptq`" section below. Faster feedback loop than `run_code` when you're iterating selector candidates. - -### 6. Form a hypothesis - -| Symptom | Likely cause | -|---|---| -| Element missing, page is `/login` | Auth: stale `check`, expired session, missing env var | -| Element in HTML but `display: none` | `waitForVisible`, not `waitForElement` | -| Locator matches 2+ (strict mode) | Disambiguate: ARIA role, `step.opts({ elementIndex })`, `within` | -| Element in screenshot N+1, missing in N | Animation / lazy load — `waitForVisible(loc, t)` | -| 401/403 in console.json | API token expired or env var missing | -| Steps pass, next `I.see` fails | Frame switch missed — wrap in `within({ frame })` | -| Different result CI vs local | `setHeadlessWhen(CI)`, viewport, timing, env var | -| Recorder hangs, step never fires | `DEBUG="codeceptjs:recorder"` to inspect the queue | -| Plugin misbehaves | `DEBUG="codeceptjs:plugin:"` | - -### 7. Verify the fix on the live page -For agents driving MCP, use `run_code` to try the candidate fix in the live session **before editing the file**. If it works there, it'll work in the test. While paused (in-test `pause()` or `pauseAt`), `run_code` operates on the same `I` / browser the test is using, so a candidate replacement step can be tried in place. Humans running with `--debug` at a TTY can use in-test `pause()` for the same purpose at a readline REPL. - -### 8. Apply and re-run -Edit the test, then `npx codeceptjs run --grep '' --steps`. Use **codeceptjs-run-analysis** to verify the trace looks right after the fix — and to confirm the failure didn't shift to another step. If the fix introduces a `waitFor*` or `step.opts`, leave a one-line `Why:` comment — those are the comments worth keeping. - -## When to reach for which plugin / mode - -| You want to … | Use | -|---|---| -| Per-step artifacts after a run | `aiTrace` plugin (`output/trace_*/`), declared once in config | -| Capture every step's state | `npx codeceptjs run -p aiTrace:on=step` | -| Capture only the failed step (lean CI repro) | `npx codeceptjs run -p aiTrace:on=fail` | -| Capture last step per test | `npx codeceptjs run -p aiTrace:on=test` | -| Capture steps from a file/line or URL | `-p aiTrace:on=file:path=;line=` / `-p aiTrace:on=url:pattern=` | -| REPL on first failure | `npx codeceptjs run -p pause` (default `on=fail`) | -| Single-step interactively | `npx codeceptjs run -p pause:on=step` | -| Break on a file or URL | `pause:on=file:path=;line=` / `pause:on=url:pattern=` | -| Programmatic breakpoint at step N (no test edit) | MCP `run_test` with `pauseAt: N` (discover N via `dry-run --numbers`) | -| In-test breakpoint at a specific line | drop `pause()` in the test, then MCP `run_test` | -| Step-by-step REPL from an AI agent | MCP `run_step_by_step`, then `continue` between steps | -| Release a paused test | MCP `continue` | -| Test a hypothesis on the live page (agent) | MCP `run_code` (works fresh **and** while paused) | -| Capture state without acting (agent) | MCP `snapshot` | -| Test a hypothesis on the live page (human, TTY) | in-test `pause()` + `npx codeceptjs run --debug` | -| List steps with their indices (for `pauseAt`) | `npx codeceptjs dry-run --debug --grep '' --numbers --no-ansi` | -| Visual replay slideshow | `screenshot:slides=true` → `output/records.html` | -| Auto-suggest fixes for broken locators | `heal` plugin + `--ai` (disabled in `--debug`) | -| Diagnose framework-internal behaviour | `DEBUG="codeceptjs:*"` (or a specific namespace) | -| Inspect specific elements — state, markup, position, children | `I.grabWebElement` / `I.grabWebElements` (cross-helper WebElement API) | -| Drop to native helper APIs when nothing else works | `I.usePlaywrightTo` / `I.usePuppeteerTo` / `I.useWebDriverTo` | -| Verify a locator against a saved trace snapshot (offline) | `codeceptq --file output/trace_*/__page.html` | - -## Waiting (a common cause of flakes) - -Most "intermittent" failures are missed waits. Use the trace HTML / ARIA to find the *actual* gating element rather than adding a generic delay: -- a **loader / spinner / skeleton** still on the page → `I.waitForInvisible('.spinner')` / `I.waitForDetached('.skeleton')` -- a **modal / drawer / panel** that hasn't appeared yet → `I.waitForVisible('.modal')` / `I.waitForElement({ role: 'dialog' })` -- async data — list rows, cards, charts, async-rendered text → `I.waitForElement('.user-row', 10)` / `I.waitForText('Loaded', 10, '.status')` - -`I.wait(N)` (raw seconds) is fine **during debugging** to confirm a timing hypothesis — if a 5-second sleep makes the test pass, you've found the cause. **Replace it with the specific `I.waitFor*` before committing.** Raw sleeps are slow on fast machines, flaky on slow ones, and hide the real sync point so the next person to touch the test inherits the same problem. - -## Inspect the page when the trace isn't enough - -When the trace tells you *what* failed but you need more page-state detail to diagnose — "is this button actually disabled?", "are there really two Save buttons?", "what's the rendered markup of this row?" — hand off to the **codeceptjs-exploration** skill. It covers the WebElement API (`I.grabWebElement` / `I.grabWebElements`, state checks, `toSimplifiedHTML`, `toAbsoluteXPath`, iframe walking) and the broad-XPath candidate-discovery technique. - -Debug-specific reaches into that toolkit: - -- **Button rendered but the click had no effect** — `grabWebElement('Submit')`, then `isEnabled()` + `getBoundingBox()`. Disabled? offscreen? zero-sized? -- **Strict-mode "matched 2 elements"** — exploration's broad-XPath + iterate-and-disambiguate pattern is the canonical fix. -- **Iframe content** — exploration's `inIframe` pattern; the failing step likely needs to be wrapped in `within({ frame })`. - -Prefer this over `usePlaywrightTo` / `useWebDriverTo` for inspection: same code across helpers, less boilerplate. +1. **Fundamentals** — run `codeceptjs-fundamentals`: helper, plugins (`aiTrace`, `screenshot`, `pageInfo`, `retryFailedStep`, `pause`, `auth`), env vars, MCP availability. If `aiTrace` isn't declared, add it once: `plugins: { aiTrace: { enabled: true } }` — most of this skill leans on its output. + - **Declare once; never edit config to change its trigger.** Override per-run with `-p aiTrace:on=step|fail|test|file|url` (fundamentals § Plugins from CLI). `on=fail` for lean CI repros, `on=step` while diagnosing. Flipping `on:` in config churns the repo and leaks into other runs. +2. **Reproduce minimally**: `npx codeceptjs run --grep '' --steps`. Add `-c codecept.ci.conf.js` if CI-specific. Confirm reproduction before instrumenting. +3. **Pick tools**: + - MCP `run_test ` — runs in-process; returns reporter result or `{ status: 'paused', pausedAfter, page, suggestions }` + - MCP `run_step_by_step` + `continue` — pause after every step; for watching the whole flow + - MCP `run_code` — arbitrary code in the live session; works fresh *and* paused; returns produced values + console output + final-state snapshot + - MCP `snapshot` — current browser state without acting (URL, cookies, storage, HTML, ARIA, screenshot, console) + - MCP `list_actions` — sanity-check an `I.*` method exists + - CLI `npx codeceptjs run --grep '' --debug` — first move without MCP; `--verbose` adds promise-queue/retry/timeout logs +4. **Breakpoint**: + - In-test `pause()` — best when already editing or breaking inside `within`/loop/hook + - `pauseAt: N` on `run_test` — no test edit; pauses after the Nth leaf step + - Find N: `npx codeceptjs dry-run --debug --grep '' --numbers --no-ansi` (1-based, per-test; number of the line to stop *after*) + - While paused: inspect with `run_code`, capture clean state with `snapshot`, walk prior steps in `output/trace__/trace.md`, release with `continue` +5. **Read the trace** — hand off to `codeceptjs-run-analysis`; focus on the **first** failed step — late failures are usually side effects of an earlier silent miss. It also covers grepping large HTML, clustering errors across traces, comparing reruns. +6. **Form a hypothesis**: + + | Symptom | Likely cause | + |---|---| + | Element missing, page is `/login` | Auth: stale session cache, missing env var | + | Element in HTML but `display: none` | `waitForVisible`, not `waitForElement` | + | Locator matches 2+ (strict mode) | Scope it: context arg, then ARIA role, then `step.opts({ elementIndex })` | + | Element present in screenshot N+1, missing in N | Animation / lazy load → `waitForVisible(loc, t)` | + | 401/403 in console.json | API token expired or env var missing | + | Steps pass, next `I.see` fails | Frame switch missed → `within({ frame })` | + | Different result CI vs local | `setHeadlessWhen(CI)`, viewport, timing, env vars | + | Recorder hangs, step never fires | `DEBUG="codeceptjs:recorder"` | + | Plugin misbehaves | `DEBUG="codeceptjs:plugin:"` | + +7. **Verify the fix on the live page** — try the candidate replacement step via `run_code` (works while paused, same `I`) **before editing the file**. Humans at a TTY get the same via in-test `pause()`. +8. **Apply and re-run**: edit, then `npx codeceptjs run --grep '' --steps`; confirm via `codeceptjs-run-analysis` that the failure didn't shift steps. Leave a one-line `Why:` comment when the fix introduces `waitFor*` or `step.opts` — those comments are worth keeping. ## Query trace HTML with `codeceptq` -`aiTrace` writes a per-step `__page.html` snapshot of the live DOM for every step (formatted so each element sits on its own line — line numbers map 1:1 to elements). To answer "would my locator have matched at step N?", use `codeceptq` — a CLI that resolves any CodeceptJS locator (CSS / XPath / fuzzy / semantic) against a saved HTML snapshot and prints the matched elements with their source lines. +`aiTrace` writes per-step `__page.html` snapshots (one element per line — line numbers map 1:1 to elements). `codeceptq` resolves any CodeceptJS locator against a saved snapshot. -**Never load the page HTML into your context to inspect it manually.** Real-world `*_page.html` files are thousands of lines and burn context for nothing — `codeceptq` does the locator resolution and returns only the relevant elements. Reach for it instead of `Read`-ing the snapshot. +- **Never load page HTML into context manually** — snapshots are thousands of lines; `codeceptq` returns only matched elements with their line numbers. +- Test candidate locators offline before applying via `run_code` — a hit is a green light to try live, not a guarantee (visibility, re-renders). +- Multiple matches → don't write a brittler XPath; disambiguate with `step.opts({ elementIndex })` following the order `codeceptq` prints. ```bash -# does this CSS resolve? -npx codeceptq '#submit-btn' --file output/trace__/0007_I_click_Submit_page.html - -# semantic field lookup against a saved snapshot -npx codeceptq 'Email' --field --file output/trace_*/0003_*_page.html - -# semantic clickable, scoped to a context -npx codeceptq 'Save' '.modal' --click --file output/trace_*/0005_*_page.html - -# pipe directly from stdin -cat output/trace_*/0001_*_page.html | npx codeceptq './/form//input[@required]' - -# machine-readable for chained tooling -npx codeceptq 'Username' --field --json --file output/trace_*/0002_*_page.html +npx codeceptq '#submit-btn' --file output/trace_*/0007_*_page.html # CSS +npx codeceptq 'Email' --field --file output/trace_*/0003_*_page.html # semantic field +npx codeceptq 'Save' '.modal' --click --file output/trace_*/0005_*_page.html # scoped clickable +npx codeceptq 'Username' --field --json --file ... # machine-readable ``` -What you get back: a count, the resolved XPath, and one entry per match with the **line number** in the snapshot file plus the element's outerHTML. +Key flags: `--field/--click/--checkable/--select` force semantic strategies; `--xpath`/`--css` override auto-detection (a bare tag name like `select.foo` is treated as fuzzy text); exit codes `0` match / `1` none / `2` invalid input. -Flags worth knowing: -- `--field` / `--click` / `--checkable` / `--select` — force a CodeceptJS semantic strategy (label, button text, checkbox, option). Without a flag, the locator type is auto-detected (CSS if it starts with `#`/`.`/`[`; XPath if it starts with `//` or `./`; fuzzy text otherwise). -- `--xpath` / `--css` — force interpretation when auto-detection wouldn't pick the right one (e.g., a bare tag name like `select.foo` without `--css` would be treated as fuzzy text). -- `[context]` — second positional arg restricts matches to descendants of the context locator (e.g., `'Save' '.modal' --click`). -- `--limit N` (default 20), `--snippet N` (default 500), `--full`, `--json`. -- Exit codes: `0` matches, `1` no match, `2` invalid input/XPath — useful for scripted "did this locator break?" checks. +> ⚠ **The `[context]` second positional does not scope** (CodeceptJS 4.1.0). It prints `N matches within ''` but returns page-wide results — `lib/command/query.js` evaluates an absolute XPath (`//…`) against the context node, and `//foo` re-roots at the document. It will report a match inside a container that does not hold the element. To check a scoped locator offline, pass one composed selector (`codeceptq '.modal button[aria-label="Save"]'`) and compare its count against the unscoped form; a context-dependent locator is only truly verified by running the step. -Use `codeceptq` to test locators against the snapshot **before** applying them via `run_code`. If a candidate matches, you've validated the locator string against the DOM as captured. It can still fail live — element not visible, removed by a re-render, or the snapshot is from a different step — so treat a hit as a green light to try, not a guarantee. +## Inspect deeper -When `codeceptq` returns multiple matches, **don't write a brittler XPath** — disambiguate with `step.opts({ elementIndex })`. Indexing is 1-based and follows the order `codeceptq` prints; supports `'first'`, `'last'`, and negatives. +Hand off to `codeceptjs-exploration` when the trace says *what* failed but you need more page-state detail ("is this button actually disabled?", "are there really two Save buttons?"). Debug-specific reaches: -```js -I.click('Edit', step.opts({ elementIndex: 2 })) -I.fillField('input', 'value', step.opts({ elementIndex: 'last' })) -``` +- Button rendered but click had no effect → `grabWebElement('Submit')` + `isEnabled()` + `getBoundingBox()` (disabled? offscreen?) +- Strict-mode multi-match → exploration's broad-XPath iterate-and-disambiguate pattern +- Iframe content → wrap failing steps in `within({ frame })` -## Native helper API escape hatch +Prefer this over `usePlaywrightTo`/`useWebDriverTo` for inspection — same code across helpers, less boilerplate. -When `I.grabWebElement` and the rest of the `I.*` surface still don't cover it — listening to network requests, manipulating storage, calling a Playwright-only API, raw browser context work — drop down to the underlying helper: +## Native helper escape hatch -- **Playwright** — `I.usePlaywrightTo('description', async ({ browser, browserContext, page }) => { ... })` -- **Puppeteer** — `I.usePuppeteerTo('description', async ({ page }) => { ... })` -- **WebDriver** — `I.useWebDriverTo('description', async ({ browser }) => { ... })` +Only when the `I.*` surface truly doesn't cover it (network interception, storage manipulation, helper-only APIs): -The first arg is a label that shows up in step output and traces. The callback receives the helper's native objects. Use these to inspect or manipulate state CodeceptJS doesn't expose — `page.evaluate(() => performance.timing)`, `page.context().cookies()`, `browserContext.on('request', …)`, raw `executeScript` chains. They work inside MCP `run_code` too, so you can poke at internals during a live debug session. +- Playwright: `I.usePlaywrightTo('label', async ({ browser, browserContext, page }) => { ... })` +- Puppeteer: `I.usePuppeteerTo('label', async ({ page }) => { ... })` +- WebDriver: `I.useWebDriverTo('label', async ({ browser }) => { ... })` -Try the regular `I.*` API first — these escape hatches couple the test to a specific helper. Reach for them only when nothing else works. +The label shows up in step output and traces. Works inside MCP `run_code` too. These couple tests to a specific helper — last resort. -## Helper-specific gotchas +## Helper gotchas -- **Playwright** — `strict: true` throws on multi-match. `trace: 'on'` produces `output/trace.zip` (open with `npx playwright show-trace`). Prefer `'load'` / `'domcontentloaded'` over `'networkidle'`. -- **Puppeteer** — `'networkidle0'` can hang on long-polling pages; try `'networkidle2'` or `'domcontentloaded'`. -- **WebDriver** — `smartWait` applies to actions only, not assertions. `executeScript` args must be JSON-serializable. +- Playwright: `strict: true` throws on multi-match; prefer `'load'`/`'domcontentloaded'` over `'networkidle'`; `trace: 'on'` → `output/trace.zip` (`npx playwright show-trace`) +- Puppeteer: `'networkidle0'` hangs on long-polling pages — use `'networkidle2'` or `'domcontentloaded'` +- WebDriver: `smartWait` covers actions only, not assertions; `executeScript` args must be JSON-serializable ## Auth-related failures -If the trace shows a redirect to `/login` mid-test, or 401/403 in console, fix **auth**, not the failing step. Check the `auth` plugin's `check`, that credential env vars are exported, and that the cached session under `output/_session.json` isn't stale (delete it to force re-login). The **codeceptjs-auth** skill has the full pattern. +Redirect to `/login` mid-test or 401/403 in console → fix **auth**, not the failing step: check the plugin's `check`, credential env vars, and stale cached session under `output/_session.json` (delete to force re-login). Full pattern in `codeceptjs-auth`. + +## Flakiness and waits + +Most "intermittent" failures are missed waits. Use trace HTML/ARIA to find the actual gating element instead of adding generic delay (fundamentals § Waiting has the mapping). `I.wait(N)` confirms a timing hypothesis while debugging — replace with the specific `waitFor*` before committing. ## Things to avoid - Fixing from the error message without reading the trace. -- Editing the test before verifying the fix in `run_code` — you'll iterate without ground truth. -- Committing `pause()` calls. They're a debugging tool — remove (or replace with `pauseAt`) before merging. A `pause()` left in a test that runs in a non-TTY non-MCP CI subprocess will print a notice and skip, but it's still noise on every run. -- Adding `waitFor*` blindly instead of identifying the real gating element from HTML/ARIA. -- Leaving `I.wait(N)` (raw seconds) in committed tests — keep them only while debugging, then replace with the specific `waitFor*`. -- Editing `aiTrace`'s `on:` in `codecept.conf.js` to switch capture modes — declare it once and override per-run with `-p aiTrace:on=...`. Repeated config edits churn the repo and leak the mode into unrelated runs. -- Skipping the config check — `setHeadlessWhen(CI)` or env-driven URLs explain many "works locally fails in CI" reports. -- Hiding the failure with `retries` instead of fixing the cause. - -## Pointers - -- `node_modules/codeceptjs/docs/mcp.md` — MCP tool list and client config -- `node_modules/codeceptjs/docs/aitrace.md` — plugin config, trace.md format -- `node_modules/codeceptjs/docs/debugging.md` — in-test `pause()`, the `pause` plugin's `on=` modes, IDE setup, DEBUG namespaces -- `node_modules/codeceptjs/docs/heal.md` — self-healing recipes -- `node_modules/codeceptjs/docs/retry.md` — retry semantics across step / scenario / hook -- `node_modules/codeceptjs/lib/plugin/aiTrace.js`, `lib/plugin/pause.js`, `lib/plugin/screenshot.js`, `lib/plugin/browser.js`, `bin/mcp-server.js` — source if docs and code disagree +- Editing the test before verifying the fix via `run_code`. +- Committing `pause()` calls — debugging tool only. +- Blind `waitFor*` instead of identifying the real gating element. +- Leaving `I.wait(N)` in committed tests. +- Editing `aiTrace`'s `on:` in config between runs — declare once, override per-run. +- Skipping the config check — `setHeadlessWhen(CI)` / env-driven URLs explain many "works locally fails in CI". +- Hiding failures with `retries` instead of fixing the cause. + +## Related skills + +- `codeceptjs-fundamentals` — effects, plugins-from-CLI, waiting rules +- `codeceptjs-exploration` — WebElement inspection, broad-XPath disambiguation +- `codeceptjs-run-analysis` — trace.md walking, error clustering, rerun comparison +- `codeceptjs-auth` — auth failure patterns diff --git a/plugins/codeceptjs/skills/migrate-codeceptjs-4/SKILL.md b/plugins/codeceptjs/skills/migrate-codeceptjs-4/SKILL.md index 87a86d4..d44ad56 100644 --- a/plugins/codeceptjs/skills/migrate-codeceptjs-4/SKILL.md +++ b/plugins/codeceptjs/skills/migrate-codeceptjs-4/SKILL.md @@ -1,9 +1,8 @@ --- name: migrate-codeceptjs-4 -description: "Migrate a CodeceptJS '3.x' project to '4.x'. Trigger when 'package.json' pins 'codeceptjs' at '3.x' or is missing '\"type\": \"module\"', when test files still use CommonJS ('require()' / 'module.exports') against CodeceptJS APIs, when config references removed helpers ('Nightmare', 'Protractor', 'TestCafe', 'AI', 'SoftExpectHelper', 'Mochawesome') or removed plugins ('autoLogin', 'tryTo', 'retryTo', 'eachElement', 'commentStep', 'fakerTransform', 'enhancedRetryFailedStep', 'allure', 'htmlReporter', 'wdio', 'selenoid', 'screenshotOnFail', 'pauseOnFail', 'stepByStepReport'), or when '3.x' APIs are in use ('ai.request' function, Joi schemas in 'seeResponseMatchesJsonSchema', 'restart: 'browser'', 'I.retry()', 'I.limitTime()', Playwright 'customLocators'). Walks the project through Node + package upgrade, ESM conversion, helper/plugin replacements, AI/Zod/effects API changes, 'noGlobals: true' adoption, dependency bumps, and the post-upgrade verify pass." +description: "Migrate a CodeceptJS 3.x project to 4.x. Trigger when `package.json` pins `codeceptjs` at 3.x or is missing `\"type\": \"module\"`, when test files still use CommonJS (`require()` / `module.exports`) against CodeceptJS APIs, when config references removed helpers (`Nightmare`, `Protractor`, `TestCafe`, `AI`, `SoftExpectHelper`, `Mochawesome`) or removed plugins (`autoLogin`, `tryTo`, `retryTo`, `eachElement`, `commentStep`, `fakerTransform`, `enhancedRetryFailedStep`, `allure`, `htmlReporter`, `wdio`, `selenoid`, `screenshotOnFail`, `pauseOnFail`, `stepByStepReport`), or when 3.x APIs are in use (`ai.request` function, Joi schemas in `seeResponseMatchesJsonSchema`, `restart: 'browser'`, `I.retry()`, `I.limitTime()`, Playwright `customLocators`)." --- - # Migrate CodeceptJS 3.x → 4.x CodeceptJS 4 is **ESM-only and TypeScript-first**. There is no compatibility shim for CommonJS — every helper, page object, custom step, and config file must be ESM. This skill drives a project through the upgrade end-to-end. @@ -103,8 +102,8 @@ Run, in order: 5. Grep the repo for `tryTo(`, `retryTo(`, `eachElement(`, `commentStep(`, `softExpect`, `I.softExpect`, `Joi.`, `restart: 'browser'`, `I.retry(`, `I.limitTime(`, and (unless the user chose to keep it) `Mochawesome` / `--reporter mochawesome` — none should remain. `step.retry(` / `step.timeout(` passed as a step argument is the expected replacement, not a leftover. 6. If the project used `autoLogin`: confirm the `auth` plugin restores sessions and roles. -## Pointers +## Related skills -- `node_modules/codeceptjs/docs/migration-4.md` — full reference (this skill is a workflow over it) - `codeceptjs-auth` — replacement for the removed `autoLogin` plugin -- `codeceptjs-fundamentals` — run **after** migration to confirm the new setup is wired correctly +- `codeceptjs-fundamentals` — run **after** migration to confirm wiring +- Full reference: `node_modules/codeceptjs/docs/migration-4.md` diff --git a/plugins/codeceptjs/skills/migrate-cypress-to-codeceptjs/SKILL.md b/plugins/codeceptjs/skills/migrate-cypress-to-codeceptjs/SKILL.md index 86c230a..ad1da1c 100644 --- a/plugins/codeceptjs/skills/migrate-cypress-to-codeceptjs/SKILL.md +++ b/plugins/codeceptjs/skills/migrate-cypress-to-codeceptjs/SKILL.md @@ -1,6 +1,6 @@ --- name: migrate-cypress-to-codeceptjs -description: "Port a Cypress test suite to CodeceptJS 4. Trigger when the project contains `cypress.config.{js,ts,mjs}`, a `cypress/` directory (`cypress/e2e/**/*.cy.{js,ts}`, `cypress/support/{commands,e2e}.{js,ts}`, `cypress/fixtures/`, `cypress/plugins/`, `cypress/component/`), `cypress` in `devDependencies`, or test code that calls `cy.*` (`cy.visit`, `cy.get`, `cy.contains`, `cy.session`, `cy.intercept`, `cy.request`, `cy.task`, `cy.fixture`, `cy.origin`, `cy.mount`), `Cypress.Commands.add(...)`, or `Cypress.env(...)`. Walks the port end-to-end: inventory shared logic (custom commands, ad-hoc page-object modules, shared selectors, fixtures, hooks), install CodeceptJS with the Playwright helper alongside Cypress, port the config, split `Cypress.Commands.add` into two custom helpers — `WebExtra` for browser-driven commands (Playwright `page` / `browserContext`) and `ApiExtras` for HTTP commands (REST / GraphQL helper, never `browserContext.request.*`) — port page-object-style modules to real page objects without inventing wrapper or assertion methods, convert spec files (handing off to `writing-codeceptjs-tests`), replace `cy.session` with the `auth` plugin, swap `cy.fixture` / `cy.request` / `cy.task` / `cy.intercept` for ES imports / REST helper / `ApiExtras` / `I.mockRoute`, then decommission Cypress." +description: "Port a Cypress test suite to CodeceptJS 4. Trigger when the project contains `cypress.config.{js,ts,mjs}`, a `cypress/` directory (`cypress/e2e/`, `cypress/support/{commands,e2e}.*`, `cypress/fixtures/`), `cypress` in `devDependencies`, or test code calling `cy.*` (`cy.visit`, `cy.get`, `cy.contains`, `cy.session`, `cy.intercept`, `cy.request`, `cy.task`, `cy.fixture`, `cy.origin`, `cy.mount`), `Cypress.Commands.add(...)`, or `Cypress.env(...)`." --- # Migrate Cypress → CodeceptJS 4 @@ -11,7 +11,7 @@ Cypress and CodeceptJS share a goal — browser end-to-end testing — but diffe 2. **Helpers, not a bundled browser.** `I.*` dispatches to a configured helper. Cypress is single-browser by design; CodeceptJS lets you pick **Playwright** (recommended for Cypress migrators — Chromium parity plus cross-browser), Puppeteer, or WebDriver, and the test code stays the same. 3. **First-class abstractions.** Page objects, multi-user `session(...)`, the `auth` plugin, and custom helpers are built in. Cypress projects accumulate ad-hoc versions of these; the migration consolidates them onto the framework's idioms. -Authoritative references: `node_modules/codeceptjs/docs/basics.md`, `locators.md`, `playwright.md`, `custom-helpers.md`, `pageobjects.md`. +Authoritative reference: `node_modules/codeceptjs/docs/` (basics, locators, playwright, custom-helpers, pageobjects). ## When to trigger @@ -211,32 +211,30 @@ for (const row of await I.grabWebElements('.row')) { } ``` -**Dry-run as you go.** After each batch of converted specs, run: +**Per batch**: `npx codeceptjs dry-run --steps -c ` — loads every Scenario, resolves every `I.*` call, no browser. Surfaces typos, missing imports, page objects not under `include`, and nonexistent verbs in seconds. Fix before anything real. -```bash -npx codeceptjs dry-run --steps -c -``` - -It loads every scenario, resolves every `I.*` call against the configured helpers, and prints the step list — all without launching a browser. Typos, missing imports, page objects not registered under `include`, and `I.*` verbs that don't exist on `WebExtra` / `ApiExtras` all surface here in seconds. Fix anything that fails before running a real test. +Then run the batch: `npx codeceptjs run --steps -c `. -**Then run the whole batch for real.** Dry-run proves specs parse and resolve — not that they pass. As soon as a batch is dry-run clean, run it against the browser: +- First real runs almost always fail — locator drift, timing the source framework hid behind its own retry, auth/session differences, data assumptions. **Expected; fixing it is part of the migration.** +- Every failure → invoke `debugging-codeceptjs-tests` and fix on the fly (breakpoint, live-page inspection, verified fix). No blind rewrites, no `retry` masking. +- A batch is done when it runs green, not when it dry-runs clean. -```bash -npx codeceptjs run --steps -c -``` +### 6. Locators -First real runs after a migration almost always have failures — locator drift, timing the source framework hid behind its own retry, auth/session differences, data assumptions. **This is expected; fixing it is part of the migration, not a follow-up task.** When a test fails, **invoke the `debugging-codeceptjs-tests` skill and fix it on the fly** — it breakpoints the failing step, inspects the live page via MCP, finds the working locator/wait, and commits the verified fix. Do not bulk-rewrite specs blind, and do not mask failures with `retry`. Drive every failure to a real fix before starting the next batch. A batch is "done" when it runs green, not when it dry-runs clean. +**Scope every locator with a context.** The last argument of every action narrows the lookup to a region — `I.click('Save', '.toolbar')`, `I.fillField('Email', 'u@t.com', '#login-form')`, `I.click({ role: 'button', name: 'Delete' }, '.modal')`. A short semantic or ARIA locator plus a context beats one long unscoped locator: it reads like the page, disambiguates duplicate labels without growing, and survives markup churn. Apply this to every row of the tables below — the source framework's chain usually splits cleanly into *region* + *what the user sees*. -### 6. Locators +`cy.get(sel).within(() => ...)` and `cy.get(parent).find(child)` both collapse onto the context argument — that is where a Cypress chain's parent selector belongs. -CodeceptJS priority — pick the highest that fits: +CodeceptJS priority — pick the highest that fits, then add the context: -1. **Semantic strings** — button text, label, placeholder, link text: `I.click('Save')`, `I.fillField('Email', 'u@t.com')`. -2. **ARIA roles** — `I.click({ role: 'button', name: 'Sign In' })`. -3. **`locate()` builder** — `I.click(locate('button').withText('Edit').inside('tr').withText('Acme')))`. -4. **CSS / XPath** — fallback only. +1. **Semantic strings** — button text, label, placeholder, link text: `I.click('Save', '.toolbar')`, `I.fillField('Email', 'u@t.com', '#login-form')`. Replaces most `cy.contains(...)` calls. +A plain string already matches `aria-label`, so an icon-only control with `aria-label="Save"` is `I.click('Save', )` — never `'aria-label=Save'` or `{ css: '[aria-label="Save"]' }`. +2. **ARIA roles** — `I.click({ role: 'button', name: 'Sign In' }, '#login-form')`. +3. **`$name` via the `customLocator` plugin** — Cypress users often default to `[data-cy=...]`. Keep those attributes, but enable the plugin so they read as `I.click('$submit', '.checkout')` instead of `{ css: '[data-cy=submit]' }`. +4. **`locate()` builder** — `I.click(locate('button').withText('Edit').inside('tr').withText('Acme'))`; often better split as `I.click('Edit', locate('tr').withText('Acme'))`. +5. **CSS / XPath** — fallback only. -Cypress users often default to `[data-cy=...]`. Keep those attributes, but enable the `customLocator` plugin so they read as `$submit` instead of `{ css: '[data-cy=submit]' }`. Full guidance in **`writing-codeceptjs-tests`** § Locators. +Full guidance in **`writing-codeceptjs-tests`** § Locators. ### 7. Actions, assertions, grabs @@ -288,19 +286,10 @@ Only after every spec is ported and CI is green: delete `cypress/`, `cypress.con 5. Hand off to **`codeceptjs-run-analysis`** to inspect `output/trace_*/` artifacts (requires the `aiTrace` plugin enabled). 6. `grep -r "cy\." cypress/` — empty before deleting `cypress/`. -## Pointers - -- `node_modules/codeceptjs/docs/basics.md` — `I.*` vocabulary, locators, assertions, the `await` rule -- `node_modules/codeceptjs/docs/playwright.md` — recommended helper; `mockRoute` for `cy.intercept` -- `node_modules/codeceptjs/docs/locators.md` — semantic / ARIA / `locate()` -- `node_modules/codeceptjs/docs/custom-helpers.md` — `WebExtra` / `ApiExtras` patterns (extending `Helper`, reaching `this.helpers['Playwright']` / `this.helpers['REST']`) -- `node_modules/codeceptjs/docs/api.md` — REST / GraphQL configuration, `setSharedCookies()`, `defaultHeaders`, `JSONResponse` assertions, Zod schemas -- `node_modules/codeceptjs/docs/assertions.md` — built-in `see*` assertions, `ExpectHelper`, `codeceptjs/assertions` factories (use these instead of `if (cond) throw new Error(...)`) -- `node_modules/codeceptjs/docs/pageobjects.md` — porting Cypress page-object-style modules -- `node_modules/codeceptjs/docs/data.md` — fixtures, data factories -- `node_modules/codeceptjs/docs/sessions.md`, `auth.md` — multi-user + login reuse -- `node_modules/codeceptjs/docs/effects.md` — `tryTo`, `retryTo`, `within` -- `writing-codeceptjs-tests` — per-spec rewrite playbook (drive via MCP, learn locators, commit verified steps) -- `debugging-codeceptjs-tests` — **use on every failing test from the first full run** (breakpoint, inspect live page via MCP, fix on the fly) -- `codeceptjs-auth` — replace `cy.session()` and programmatic login -- `codeceptjs-fundamentals` — run **after** migration to confirm the new setup is wired correctly +## Related skills + +- `writing-codeceptjs-tests` — per-spec rewrite playbook (MCP-driven, verified steps) +- `debugging-codeceptjs-tests` — use on every failing test from the first full run +- `codeceptjs-auth` — replaces `cy.session()` and programmatic login +- `codeceptjs-fundamentals` — run after migration to confirm wiring +- Reference docs: `node_modules/codeceptjs/docs/` (basics, playwright, locators, custom-helpers, api, assertions, pageobjects, data, sessions, effects) diff --git a/plugins/codeceptjs/skills/migrate-protractor-to-codeceptjs/SKILL.md b/plugins/codeceptjs/skills/migrate-protractor-to-codeceptjs/SKILL.md index effa75e..acfb9dc 100644 --- a/plugins/codeceptjs/skills/migrate-protractor-to-codeceptjs/SKILL.md +++ b/plugins/codeceptjs/skills/migrate-protractor-to-codeceptjs/SKILL.md @@ -1,6 +1,6 @@ --- name: migrate-protractor-to-codeceptjs -description: "Port a Protractor test suite to CodeceptJS 4. Trigger when the project contains `protractor.conf.{js,ts}`, `protractor` in `devDependencies`, `*.e2e-spec.{js,ts}` files, an `e2e/` (or `protractor/`) directory with spec files, `@types/jasmine` / `jasmine-spec-reporter` in dependencies, imports from `protractor` (`browser`, `element`, `by`, `ExpectedConditions`, `ElementFinder`, `ElementArrayFinder`), or code calling `element(by.X(...))`, `element.all(...)`, `by.addLocator(...)`, `browser.get(...)`, `browser.executeScript(...)`, `browser.wait(EC.*)`, `browser.waitForAngular(...)`, `browser.ignoreSynchronization`, `browser.params.*`, or `browser.driver.*`. Walks the port end-to-end: inventory shared logic (page objects — Protractor projects almost always have them, custom locators via `by.addLocator`, shared helpers, `onPrepare` / `onComplete` hooks, Jasmine custom matchers), install CodeceptJS with the Playwright helper alongside Protractor, port the config, split shared helpers into `WebExtra` (browser-driven via Playwright `page`) and `ApiExtras` (HTTP via REST helper, never `browserContext.request.*`), port existing page objects to CodeceptJS page objects without inventing assertion/one-liner wrappers, replace `.then(...)` promise chains with `await` only on grabs, drop `browser.waitForAngular()` / `browser.ignoreSynchronization` (CodeceptJS auto-waits), translate `element(by.X(...))` to semantic strings / ARIA / `locate()` / `{ css }`, register `by.addLocator` strategies via the `customLocator` plugin or `WebExtra`, convert specs (handing off to `writing-codeceptjs-tests`), replace `browser.params` with `process.env`, swap Jasmine `expect()` matchers for `I.see*` / `ExpectHelper` / `codeceptjs/assertions`, then decommission Protractor." +description: "Port a Protractor test suite to CodeceptJS 4. Trigger when the project contains `protractor.conf.{js,ts}`, `protractor` in `devDependencies`, `*.e2e-spec.{js,ts}` files, an `e2e/` (or `protractor/`) directory with spec files, `@types/jasmine` / `jasmine-spec-reporter` in dependencies, imports from `protractor` (`browser`, `element`, `by`, `ExpectedConditions`), or code calling `element(by.X(...))`, `element.all(...)`, `by.addLocator(...)`, `browser.get(...)` / `browser.executeScript(...)` / `browser.wait(EC.*)` / `browser.waitForAngular(...)` / `browser.ignoreSynchronization` / `browser.params.*`." --- # Migrate Protractor → CodeceptJS 4 @@ -13,7 +13,7 @@ Three foundational differences to internalize: 2. **Helpers, not `browser` / `driver`.** `I.*` dispatches to a configured helper. **Playwright recommended**; WebDriver is also available if the suite must keep running against a Selenium Grid — test code is identical either way. 3. **Auto-wait, not Angular-wait.** Drop `browser.waitForAngular()` and `browser.ignoreSynchronization`. The Playwright and WebDriver helpers wait on DOM and element stability, which covers Angular's render cycle without a framework-specific hook. -Authoritative references: `node_modules/codeceptjs/docs/basics.md`, `locators.md`, `playwright.md`, `webdriver.md`, `custom-helpers.md`, `pageobjects.md`. +Authoritative reference: `node_modules/codeceptjs/docs/` (basics, locators, playwright, webdriver, custom-helpers, pageobjects). ## When to trigger @@ -215,30 +215,26 @@ for (const row of await I.grabWebElements('.row')) { } ``` -**Dry-run as you go.** After each batch of converted specs, run: +**Per batch**: `npx codeceptjs dry-run --steps -c ` — loads every Scenario, resolves every `I.*` call, no browser. Surfaces typos, missing imports, page objects not under `include`, and nonexistent verbs in seconds. Fix before anything real. -```bash -npx codeceptjs dry-run --steps -c -``` - -It loads every scenario, resolves every `I.*` call against the configured helpers, and prints the step list — all without launching a browser. Typos, missing imports, page objects not registered under `include`, and `I.*` verbs that don't exist on `WebExtra` / `ApiExtras` all surface here in seconds. Fix anything that fails before running a real test. +Then run the batch: `npx codeceptjs run --steps -c `. -**Then run the whole batch for real.** Dry-run proves specs parse and resolve — not that they pass. As soon as a batch is dry-run clean, run it against the browser: - -```bash -npx codeceptjs run --steps -c -``` - -First real runs after a migration almost always have failures — locator drift, timing the source framework hid behind its own retry, auth/session differences, data assumptions. **This is expected; fixing it is part of the migration, not a follow-up task.** When a test fails, **invoke the `debugging-codeceptjs-tests` skill and fix it on the fly** — it breakpoints the failing step, inspects the live page via MCP, finds the working locator/wait, and commits the verified fix. Do not bulk-rewrite specs blind, and do not mask failures with `retry`. Drive every failure to a real fix before starting the next batch. A batch is "done" when it runs green, not when it dry-runs clean. +- First real runs almost always fail — locator drift, timing the ControlFlow hid behind its own queueing, auth/session differences, data assumptions. **Expected; fixing it is part of the migration.** +- Every failure → invoke `debugging-codeceptjs-tests` and fix on the fly (breakpoint, live-page inspection, verified fix). No blind rewrites, no `retry` masking. +- A batch is done when it runs green, not when it dry-runs clean. ### 6. Locator preference -CodeceptJS priority — pick the highest that fits: +**Scope every locator with a context.** The last argument of every action narrows the lookup to a region — `I.click('Save', '.toolbar')`, `I.fillField('Email', 'u@t.com', '#login-form')`, `I.click({ role: 'button', name: 'Delete' }, '.modal')`. A short semantic or ARIA locator plus a context beats one long unscoped locator: it reads like the page, disambiguates duplicate labels without growing, and survives markup churn. Apply this to every row of the tables below — the source framework's chain usually splits cleanly into *region* + *what the user sees*. + +CodeceptJS priority — pick the highest that fits, then add the context: -1. **Semantic strings** — button text, label, placeholder, link text: `I.click('Save')`, `I.fillField('Email', 'u@t.com')`. Covers Protractor's `by.linkText`, `by.buttonText`, `by.partialButtonText`, `by.partialLinkText` cleanly. -2. **ARIA roles** — `I.click({ role: 'button', name: 'Sign In' })`. Strong default for Angular apps that ship Material / ARIA-correct components. -3. **`locate()` builder** — `I.click(locate('.row').withText('Acme').inside('table'))`. Direct equivalent of `by.cssContainingText` + element traversal chains. -4. **CSS / XPath / attribute objects** — `{ id: 'foo' }`, `{ name: 'email' }`, `{ css: '[ng-model="user.email"]' }`, `{ xpath: '//div[@id="x"]' }`. The fallback for Angular directive attributes. +1. **Semantic strings** — button text, label, placeholder, link text: `I.click('Save', '.toolbar')`, `I.fillField('Email', 'u@t.com', '#login-form')`. Covers Protractor's `by.linkText`, `by.buttonText`, `by.partialButtonText`, `by.partialLinkText` cleanly. +A plain string already matches `aria-label`, so an icon-only control with `aria-label="Save"` is `I.click('Save', )` — never `'aria-label=Save'` or `{ css: '[aria-label="Save"]' }`. +2. **ARIA roles** — `I.click({ role: 'button', name: 'Sign In' }, '#login-form')`. Strong default for Angular apps that ship Material / ARIA-correct components. +3. **`$name` via the `customLocator` plugin** — when the app tags elements with `data-qa` / `data-test`; beats repeating the attribute selector at every call site. +4. **`locate()` builder** — `I.click(locate('.row').withText('Acme').inside('table'))`. Direct equivalent of `by.cssContainingText` + element traversal chains — though `I.click('Edit', locate('tr').withText('Acme'))` is usually the better split. +5. **CSS / XPath / attribute objects** — `{ id: 'foo' }`, `{ name: 'email' }`, `{ css: '[ng-model="user.email"]' }`, `{ xpath: '//div[@id="x"]' }`. The fallback for Angular directive attributes. | Protractor locator | CodeceptJS 4 | |---|---| @@ -334,19 +330,10 @@ Only after every spec is ported and CI is green: delete `e2e/` (or whichever dir 5. Hand off to **`codeceptjs-run-analysis`** to inspect `output/trace_*/` artifacts (requires the `aiTrace` plugin enabled). 6. `grep -rE "\\bbrowser\\.|\\bby\\.|\\.then\\(|waitForAngular" e2e/` — empty before deleting `e2e/`. -## Pointers - -- `node_modules/codeceptjs/docs/basics.md` — `I.*` vocabulary, locators, assertions, the `await` rule -- `node_modules/codeceptjs/docs/playwright.md` — recommended helper; `mockRoute` for any network mocking; `evaluate` for `executeScript` ports -- `node_modules/codeceptjs/docs/webdriver.md` — alternative helper if the suite stays on Selenium Grid -- `node_modules/codeceptjs/docs/locators.md` — semantic / ARIA / `locate()`, `customLocator` plugin for `by.addLocator` replacements -- `node_modules/codeceptjs/docs/custom-helpers.md` — `WebExtra` / `ApiExtras` patterns (extending `Helper`, reaching `this.helpers['Playwright']` / `this.helpers['REST']`) -- `node_modules/codeceptjs/docs/api.md` — REST / GraphQL configuration, `setSharedCookies()`, `defaultHeaders`, `JSONResponse` assertions, Zod schemas -- `node_modules/codeceptjs/docs/assertions.md` — built-in `see*` assertions, `ExpectHelper`, `codeceptjs/assertions` factories (use these instead of `if (cond) throw new Error(...)`) -- `node_modules/codeceptjs/docs/pageobjects.md` — porting Protractor page objects -- `node_modules/codeceptjs/docs/sessions.md`, `auth.md` — multi-user + login reuse -- `node_modules/codeceptjs/docs/effects.md` — `tryTo`, `retryTo`, `within` -- `writing-codeceptjs-tests` — per-spec rewrite playbook (drive via MCP, learn locators, commit verified steps) -- `debugging-codeceptjs-tests` — **use on every failing test from the first full run** (breakpoint, inspect live page via MCP, fix on the fly) -- `codeceptjs-auth` — replace UI re-login in every `beforeEach` -- `codeceptjs-fundamentals` — run **after** migration to confirm the new setup is wired correctly +## Related skills + +- `writing-codeceptjs-tests` — per-spec rewrite playbook (MCP-driven, verified steps) +- `debugging-codeceptjs-tests` — use on every failing test from the first full run +- `codeceptjs-auth` — replaces UI re-login in every `beforeEach` / manual cookie injection +- `codeceptjs-fundamentals` — run after migration to confirm wiring +- Reference docs: `node_modules/codeceptjs/docs/` (basics, playwright, webdriver, locators, custom-helpers, api, assertions, pageobjects, sessions, effects) diff --git a/plugins/codeceptjs/skills/migrate-selenium-java-to-codeceptjs/SKILL.md b/plugins/codeceptjs/skills/migrate-selenium-java-to-codeceptjs/SKILL.md index eb907c4..5cad472 100644 --- a/plugins/codeceptjs/skills/migrate-selenium-java-to-codeceptjs/SKILL.md +++ b/plugins/codeceptjs/skills/migrate-selenium-java-to-codeceptjs/SKILL.md @@ -1,6 +1,6 @@ --- name: migrate-selenium-java-to-codeceptjs -description: "Port a Selenium WebDriver or Selenide (Java) test suite to CodeceptJS 4. Trigger when the project contains `pom.xml` declaring `selenium-java` / `selenide` / `webdrivermanager`, `build.gradle` / `build.gradle.kts` with `org.seleniumhq.selenium:selenium-java` or `com.codeborne:selenide`, a `src/test/java/` tree with `*Test.java` / `*IT.java` / `*Tests.java` / `*Steps.java`, imports from `org.openqa.selenium.*` (`WebDriver`, `WebElement`, `By`, `WebDriverWait`, `ExpectedConditions`, `Actions`, `JavascriptExecutor`, `Select`, `Keys`) or `com.codeborne.selenide.*` (`Selenide`, `SelenideElement`, `Configuration`, `Condition`, `ElementsCollection`), JUnit 5 / 4 or TestNG annotations (`@Test`, `@BeforeEach`, `@AfterEach`, `@BeforeAll`, `@BeforeMethod`, `@BeforeClass`, `@DataProvider`, `@ParameterizedTest`, `@FindBy`, `@FindAll`), `PageFactory.initElements(...)`, `WebDriverManager.*.setup()`, `new ChromeDriver(...)` / `new RemoteWebDriver(...)`, `Selenide.open(...)`, `$(...).shouldBe(...)` / `$$(...).filter(...)` chains, or Cucumber-JVM step defs (`@Given` / `@When` / `@Then` from `io.cucumber.java.en.*`). Walks the port end-to-end: inventory page objects (almost always present, often `@FindBy`-driven), shared `*Helper` / `*Manager` classes, JUnit/TestNG hooks and listeners, data providers, Cucumber step defs; install CodeceptJS in a parallel directory with the **WebDriver helper as the default** (most native target — same W3C protocol the Java suite already speaks); no driver setup is needed because WebdriverIO v9 auto-starts the matching browser driver, with **Docker Selenium (Selenoid / `selenium/standalone-chrome`) only as a fallback** for parallel/CI/Grid runs per `codeceptjs-fundamentals` and `node_modules/codeceptjs/docs/webdriver.md § \"Selenium in Docker (Selenoid)\"`; port the config (Maven/Gradle deps → `package.json`, Selenide `Configuration.*` / Selenium capabilities → `helpers.WebDriver.*`); split shared Java helpers into `WebExtra` (browser-driven — `JavascriptExecutor` bodies become `browser.execute(...)` via `this.helpers['WebDriver'].browser`) and `ApiExtras` (RestAssured / Apache HttpClient → REST helper, never via the browser helper); port `@FindBy`-driven page objects to CodeceptJS page objects without inventing assertion/one-liner wrappers; **drop explicit wait code** (`WebDriverWait`, `ExpectedConditions.*`, Selenide `.shouldBe(visible)` / `.shouldHave(text(...))` chains) — CodeceptJS auto-waits via `smartWait`; translate `By.X(...)` / `@FindBy(...)` locators to semantic strings / ARIA / `locate()` / `{ css }`; convert specs (handing off to `writing-codeceptjs-tests`), mapping JUnit 5 / JUnit 4 / TestNG annotations to `Feature` / `Scenario` / `Before` / `BeforeSuite` / `After` / `AfterSuite` and `@DataProvider` / `@ParameterizedTest` to `Data(...).Scenario(...)`; swap any custom `LoginHelper` / cookie-based session reuse for the `auth` plugin; keep WireMock as a sidecar for client-side stubs while on WebDriver (WebDriver has no native network interception); for Cucumber-JVM, keep `.feature` files and rewrite step defs in JS via CodeceptJS BDD; then decommission the Java suite. Once the WebDriver suite is green, propose the **optional Playwright swap** (faster, cross-browser from one config, native `I.mockRoute`) — mechanical config change, test code unchanged." +description: "Port a Selenium WebDriver or Selenide (Java) test suite to CodeceptJS 4. Trigger when the project contains `pom.xml` declaring `selenium-java` / `selenide` / `webdrivermanager` / `junit-jupiter` / `testng`, a `build.gradle{,.kts}` with `org.seleniumhq.selenium` / `com.codeborne:selenide`, a `src/test/java/` tree with `*Test.java` / `*IT.java` / `*Steps.java`, imports from `org.openqa.selenium.*` (`WebDriver`, `By`, `WebDriverWait`, `ExpectedConditions`, `Actions`) or `com.codeborne.selenide.*` (`Selenide`, `SelenideElement`, `Condition`), JUnit/TestNG annotations (`@Test`, `@BeforeEach`, `@DataProvider`, `@FindBy`, `@ParameterizedTest`), `PageFactory.initElements(...)`, `new ChromeDriver(...)` / `Selenide.open(...)`, or Cucumber-JVM step defs (`@Given` / `@When` / `@Then`)." --- # Migrate Selenium / Selenide (Java) → CodeceptJS 4 @@ -13,7 +13,7 @@ Three foundational differences to internalize: 2. **Helpers, not `WebDriver` / `driver`.** `I.*` dispatches to a configured helper. **Default to the WebDriver helper** — it speaks the same W3C WebDriver protocol your Java suite already speaks, so the migration is most native: same Selenium server, same browser drivers, same capabilities, same Grid if you have one. Once the suite is green on WebDriver, you can swap in the **Playwright** helper (modern, faster, less flaky, native multi-browser via one config) by changing one helper block — the test code is identical either way. 3. **First-class abstractions.** Page objects, the `auth` plugin, multi-user `session(...)`, custom helpers, and the `customLocator` plugin are built in. Java suites already centralise these (`PageFactory`, `LoginHelper`, `DriverManager`); the migration consolidates them onto the framework's idioms instead of carrying the Java-specific glue forward. -Authoritative references: `node_modules/codeceptjs/docs/basics.md`, `locators.md`, `playwright.md`, `webdriver.md`, `custom-helpers.md`, `pageobjects.md`. +Authoritative reference: `node_modules/codeceptjs/docs/` (basics, locators, playwright, webdriver, custom-helpers, pageobjects). ## When to trigger @@ -317,30 +317,26 @@ for (const sort of testSort) { } ``` -**Dry-run as you go.** After each batch of converted specs, run: +**Per batch**: `npx codeceptjs dry-run --steps -c ` — loads every Scenario, resolves every `I.*` call, no browser. Surfaces typos, missing imports, page objects not under `include`, and nonexistent verbs in seconds. Fix before anything real. -```bash -npx codeceptjs dry-run --steps -c -``` - -It loads every scenario, resolves every `I.*` call against the configured helpers, and prints the step list — all without launching a browser. Typos, missing imports, page objects not registered under `include`, and `I.*` verbs that don't exist on `WebExtra` / `ApiExtras` all surface here in seconds. Fix anything that fails before running a real test. +Then run the batch: `npx codeceptjs run --steps -c `. -**Then run the whole batch for real.** Dry-run proves specs parse and resolve — not that they pass. As soon as a batch is dry-run clean, run it against the browser: - -```bash -npx codeceptjs run --steps -c -``` - -First real runs after a migration almost always have failures — locator drift, timing the explicit `WebDriverWait` code was masking, auth/session differences, data assumptions. **This is expected; fixing it is part of the migration, not a follow-up task.** When a test fails, **invoke the `debugging-codeceptjs-tests` skill and fix it on the fly** — it breakpoints the failing step, inspects the live page via MCP, finds the working locator/wait, and commits the verified fix. Do not bulk-rewrite specs blind, and do not mask failures with `retry`. Drive every failure to a real fix before starting the next batch. A batch is "done" when it runs green, not when it dry-runs clean. +- First real runs almost always fail — locator drift, timing the explicit `WebDriverWait` code was masking, auth/session differences, data assumptions. **Expected; fixing it is part of the migration.** +- Every failure → invoke `debugging-codeceptjs-tests` and fix on the fly (breakpoint, live-page inspection, verified fix). No blind rewrites, no `retry` masking. +- A batch is done when it runs green, not when it dry-runs clean. ### 6. Locators -CodeceptJS priority — pick the highest that fits: +**Scope every locator with a context.** The last argument of every action narrows the lookup to a region — `I.click('Save', '.toolbar')`, `I.fillField('Email', 'u@t.com', '#login-form')`, `I.click({ role: 'button', name: 'Delete' }, '.modal')`. A short semantic or ARIA locator plus a context beats one long unscoped locator: it reads like the page, disambiguates duplicate labels without growing, and survives markup churn. Apply this to every row of the tables below — the source framework's chain usually splits cleanly into *region* + *what the user sees*. -1. **Semantic strings** — button text, label, placeholder, link text: `I.click('Save')`, `I.fillField('Email', 'u@t.com')`. -2. **ARIA roles** — `I.click({ role: 'button', name: 'Sign In' })`. -3. **`locate()` builder** — `I.click(locate('button').withText('Edit').inside('tr').withText('Acme'))`. -4. **CSS / XPath** — fallback only. +CodeceptJS priority — pick the highest that fits, then add the context: + +1. **Semantic strings** — button text, label, placeholder, link text: `I.click('Save', '.toolbar')`, `I.fillField('Email', 'u@t.com', '#login-form')`. +A plain string already matches `aria-label`, so an icon-only control with `aria-label="Save"` is `I.click('Save', )` — never `'aria-label=Save'` or `{ css: '[aria-label="Save"]' }`. +2. **ARIA roles** — `I.click({ role: 'button', name: 'Sign In' }, '#login-form')`. +3. **`$name` via the `customLocator` plugin** — when the suite leans on `data-test` / `data-qa` attributes. +4. **`locate()` builder** — `I.click(locate('button').withText('Edit').inside('tr').withText('Acme'))`; often better split as `I.click('Edit', locate('tr').withText('Acme'))`. +5. **CSS / XPath** — fallback only. Java suites lean on `By.id`, `By.cssSelector`, `By.xpath`, `By.linkText`, and `@FindBy(...)` heavily. Translation: @@ -454,19 +450,10 @@ Once the migration is complete and WebDriver runs are stable in CI, consider swa You gain: cross-browser coverage (Chromium / Firefox / WebKit) from one config, faster runs, native `I.mockRoute(...)` network mocking, richer ARIA snapshots via the MCP loop. -## Pointers - -- `node_modules/codeceptjs/docs/basics.md` — `I.*` vocabulary, locators, assertions, the `await` rule -- `node_modules/codeceptjs/docs/webdriver.md` — default helper for this migration; § "Selenium in Docker (Selenoid)" for the container setup -- `node_modules/codeceptjs/docs/playwright.md` — target for the optional follow-up swap; `mockRoute` for WireMock client-side stubs -- `node_modules/codeceptjs/docs/locators.md` — semantic / ARIA / `locate()`, `customLocator` plugin -- `node_modules/codeceptjs/docs/pageobjects.md` — for ported `@FindBy` POMs -- `node_modules/codeceptjs/docs/custom-helpers.md` — `WebExtra` / `ApiExtras` patterns (see § "WebDriver Example" for the `browser` access pattern) -- `node_modules/codeceptjs/docs/api.md` — REST helper for RestAssured ports -- `node_modules/codeceptjs/docs/auth.md` — replace `LoginHelper` + cookie reuse -- `node_modules/codeceptjs/docs/bdd.md` — CodeceptJS BDD for Cucumber-JVM ports -- `node_modules/codeceptjs/docs/effects.md` — `session`, `tryTo`, `retryTo`, `within` -- **`codeceptjs-fundamentals`** — Docker-fallback Selenium setup convention; run **after** migration to confirm wiring -- **`writing-codeceptjs-tests`** — per-spec rewrite playbook (drive via MCP, learn locators, commit verified steps) -- **`debugging-codeceptjs-tests`** — **use on every failing test from the first full run** (breakpoint, inspect live page via MCP, fix on the fly) -- **`codeceptjs-auth`** — replace `LoginHelper` / cookie-based session reuse +## Related skills + +- `writing-codeceptjs-tests` — per-spec rewrite playbook (MCP-driven, verified steps) +- `debugging-codeceptjs-tests` — use on every failing test from the first full run +- `codeceptjs-auth` — replaces `LoginHelper` / cookie-based session reuse +- `codeceptjs-fundamentals` — Docker-fallback Selenium convention; run after migration to confirm wiring +- Reference docs: `node_modules/codeceptjs/docs/` (basics, webdriver, playwright, locators, custom-helpers, api, auth, bdd, effects) diff --git a/plugins/codeceptjs/skills/migrate-testcafe-to-codeceptjs/SKILL.md b/plugins/codeceptjs/skills/migrate-testcafe-to-codeceptjs/SKILL.md index 13f4024..9863548 100644 --- a/plugins/codeceptjs/skills/migrate-testcafe-to-codeceptjs/SKILL.md +++ b/plugins/codeceptjs/skills/migrate-testcafe-to-codeceptjs/SKILL.md @@ -1,6 +1,6 @@ --- name: migrate-testcafe-to-codeceptjs -description: "Port a TestCafe test suite to CodeceptJS 4. Trigger when the project contains `.testcaferc.{json,js,ts,cjs}`, `testcafe` in `devDependencies`, test files that import from `testcafe` (`Selector`, `ClientFunction`, `Role`, `RequestMock`, `RequestHook`, `RequestLogger`, `t` from a test signature), top-level `fixture('X').page(...)` + `test('y', async t => { ... })` blocks, `Selector(...)` chains (`.withText`, `.withAttribute`, `.nth`, `.find`, `.filter`, `.parent`, `.child`, `.sibling`), `await t.click(...)` / `t.typeText(...)` patterns, `t.useRole(...)`, `t.addRequestHooks(...)`, `t.eval(...)`, `ClientFunction(...)`, TestCafe Studio recordings, or a `tests/` directory matching `*.test.{js,ts}` whose contents start with `fixture(...)`. Walks the port end-to-end: inventory shared logic (TestCafe page-object-style modules, `Role` definitions, `ClientFunction` factories, `RequestMock` factories, custom Test Controller actions, fixture hooks), install CodeceptJS with the Playwright helper alongside TestCafe, port the config, split shared helpers into `WebExtra` (browser-driven via Playwright `page` — `ClientFunction` / `t.eval` ports here as `page.evaluate`) and `ApiExtras` (HTTP via REST helper, never `browserContext.request.*`), port TestCafe page objects to CodeceptJS page objects without inventing assertion/one-liner wrappers, replace `Selector(...)` chains with semantic strings / ARIA / `locate()` / `{ css }`, **strip excess `await` from every action call** (CodeceptJS auto-queues — `await` is only for grabs), convert specs (handing off to `writing-codeceptjs-tests`), replace `Role` + `t.useRole` with the `auth` plugin, swap `RequestMock` for `I.mockRoute`, fold `t.expect(sel.X).Y(...)` chains into `I.see*` / `ExpectHelper` / `codeceptjs/assertions`, then decommission TestCafe." +description: "Port a TestCafe test suite to CodeceptJS 4. Trigger when the project contains `.testcaferc.{json,js,ts,cjs}`, `testcafe` in `devDependencies`, test files importing from `testcafe` (`Selector`, `ClientFunction`, `Role`, `RequestMock`, `RequestHook`, `RequestLogger`), top-level `fixture('X').page(...)` + `test('y', async t => { ... })` blocks, `Selector(...)` chains (`.withText`, `.nth`, `.find`, `.filter`), `await t.click(...)` patterns, `t.useRole(...)`, `t.eval(...)` / `ClientFunction(...)`, or TestCafe Studio recordings." --- # Migrate TestCafe → CodeceptJS 4 @@ -11,7 +11,7 @@ TestCafe and CodeceptJS share a lot at the surface — both expose a single test 2. **Helpers, not a bundled proxy.** TestCafe runs as an HTTP/HTTPS proxy that injects automation into pages; CodeceptJS dispatches `I.*` to a configured helper. **Playwright recommended** — closest feel, fastest, supports all three engines (Chromium / Firefox / WebKit) the same way TestCafe did. 3. **First-class abstractions.** Page objects, multi-user `session(...)`, the `auth` plugin, custom helpers, and the `customLocator` plugin are built in. TestCafe projects accumulate ad-hoc versions of these (Selector-property classes, `Role` factories, `ClientFunction` factories) — the migration consolidates them onto framework idioms. -Authoritative references: `node_modules/codeceptjs/docs/basics.md`, `locators.md`, `playwright.md`, `custom-helpers.md`, `pageobjects.md`. +Authoritative reference: `node_modules/codeceptjs/docs/` (basics, locators, playwright, custom-helpers, pageobjects). ## When to trigger @@ -215,30 +215,26 @@ for (const row of await I.grabWebElements('.row')) { } ``` -**Dry-run as you go.** After each batch of converted specs, run: +**Per batch**: `npx codeceptjs dry-run --steps -c ` — loads every Scenario, resolves every `I.*` call, no browser. Surfaces typos, missing imports, page objects not under `include`, and nonexistent verbs in seconds. Fix before anything real. -```bash -npx codeceptjs dry-run --steps -c -``` - -It loads every scenario, resolves every `I.*` call against the configured helpers, and prints the step list — all without launching a browser. Typos, missing imports, page objects not registered under `include`, and `I.*` verbs that don't exist on `WebExtra` / `ApiExtras` all surface here in seconds. Fix anything that fails before running a real test. +Then run the batch: `npx codeceptjs run --steps -c `. -**Then run the whole batch for real.** Dry-run proves specs parse and resolve — not that they pass. As soon as a batch is dry-run clean, run it against the browser: - -```bash -npx codeceptjs run --steps -c -``` - -First real runs after a migration almost always have failures — locator drift, timing the source framework hid behind its own retry, auth/session differences, data assumptions. **This is expected; fixing it is part of the migration, not a follow-up task.** When a test fails, **invoke the `debugging-codeceptjs-tests` skill and fix it on the fly** — it breakpoints the failing step, inspects the live page via MCP, finds the working locator/wait, and commits the verified fix. Do not bulk-rewrite specs blind, and do not mask failures with `retry`. Drive every failure to a real fix before starting the next batch. A batch is "done" when it runs green, not when it dry-runs clean. +- First real runs almost always fail — locator drift, timing the source framework hid behind its own retry, auth/session differences, data assumptions. **Expected; fixing it is part of the migration.** +- Every failure → invoke `debugging-codeceptjs-tests` and fix on the fly (breakpoint, live-page inspection, verified fix). No blind rewrites, no `retry` masking. +- A batch is done when it runs green, not when it dry-runs clean. ### 6. Locator preference -CodeceptJS priority — pick the highest that fits. TestCafe's lazy chainable `Selector` lines up well with CodeceptJS's `locate()` builder, but most chains shrink considerably because semantic strings cover what `.withText` was doing. +**Scope every locator with a context.** The last argument of every action narrows the lookup to a region — `I.click('Save', '.toolbar')`, `I.fillField('Email', 'u@t.com', '#login-form')`, `I.click({ role: 'button', name: 'Delete' }, '.modal')`. A short semantic or ARIA locator plus a context beats one long unscoped locator: it reads like the page, disambiguates duplicate labels without growing, and survives markup churn. Apply this to every row of the tables below — the source framework's chain usually splits cleanly into *region* + *what the user sees*. + +CodeceptJS priority — pick the highest that fits, then add the context. TestCafe's lazy chainable `Selector` lines up well with CodeceptJS's `locate()` builder, but most chains shrink considerably because a semantic string plus a context covers what `.withText` + `.find` were doing — `Selector('.row').withText('Acme').find('.btn')` becomes `I.click('Edit', locate('.row').withText('Acme'))`. -1. **Semantic strings** — button text, label, placeholder, link text: `I.click('Save')`, `I.fillField('Email', 'u@t.com')`. Covers `Selector('button').withText('Save')` cleanly. -2. **ARIA roles** — `I.click({ role: 'button', name: 'Sign In' })`. Strong default for modern apps. -3. **`locate()` builder** — `I.click(locate('.row').withText('Acme').inside('table'))`. Direct equivalent of TestCafe `Selector('.row').withText('Acme').find('.btn')` style chains. -4. **CSS / XPath / attribute objects** — `{ id: 'foo' }`, `{ name: 'email' }`, `{ css: '[data-test=submit]' }`, `{ xpath: '//div[@id="x"]' }`. Fallback. +1. **Semantic strings** — button text, label, placeholder, link text: `I.click('Save', '.toolbar')`, `I.fillField('Email', 'u@t.com', '#login-form')`. Covers `Selector('button').withText('Save')` cleanly. +A plain string already matches `aria-label`, so an icon-only control with `aria-label="Save"` is `I.click('Save', )` — never `'aria-label=Save'` or `{ css: '[aria-label="Save"]' }`. +2. **ARIA roles** — `I.click({ role: 'button', name: 'Sign In' }, '#login-form')`. Strong default for modern apps. +3. **`$name` via the `customLocator` plugin** — when the suite uses `data-test` / `data-qa` attributes. +4. **`locate()` builder** — `I.click(locate('.row').withText('Acme').inside('table'))`. Direct equivalent of TestCafe `Selector(...)` chains that don't reduce to locator + context. +5. **CSS / XPath / attribute objects** — `{ id: 'foo' }`, `{ name: 'email' }`, `{ css: '[data-test=submit]' }`, `{ xpath: '//div[@id="x"]' }`. Fallback. | TestCafe Selector chain | CodeceptJS 4 | |---|---| @@ -333,18 +329,10 @@ Only after every spec is ported and CI is green: delete `.testcaferc.{json,js,ts 5. Hand off to **`codeceptjs-run-analysis`** to inspect `output/trace_*/` artifacts (requires the `aiTrace` plugin enabled). 6. `grep -rE "\\bfixture\\(|\\btest\\(|Selector\\(|ClientFunction\\(|\\bRole\\(|t\\.click\\(|t\\.typeText\\(" tests/` — empty before deleting the original TestCafe directory. -## Pointers - -- `node_modules/codeceptjs/docs/basics.md` — `I.*` vocabulary, locators, assertions, the `await` rule (the rule TestCafe users have to *un*-learn) -- `node_modules/codeceptjs/docs/playwright.md` — recommended helper; `mockRoute` for `RequestMock`; `evaluate` for `ClientFunction` / `t.eval` ports -- `node_modules/codeceptjs/docs/locators.md` — semantic / ARIA / `locate()` builder (replaces TestCafe Selector chains) -- `node_modules/codeceptjs/docs/custom-helpers.md` — `WebExtra` / `ApiExtras` patterns (extending `Helper`, reaching `this.helpers['Playwright']` / `this.helpers['REST']`) -- `node_modules/codeceptjs/docs/api.md` — REST / GraphQL configuration, `setSharedCookies()`, `defaultHeaders`, `JSONResponse` assertions, Zod schemas -- `node_modules/codeceptjs/docs/assertions.md` — built-in `see*` assertions, `ExpectHelper`, `codeceptjs/assertions` factories (use these instead of `if (cond) throw new Error(...)`) -- `node_modules/codeceptjs/docs/pageobjects.md` — porting TestCafe Selector-based page objects -- `node_modules/codeceptjs/docs/sessions.md`, `auth.md` — replaces `Role` + `t.useRole` -- `node_modules/codeceptjs/docs/effects.md` — `tryTo`, `retryTo`, `within` (replaces `t.switchToIframe`) -- `writing-codeceptjs-tests` — per-spec rewrite playbook (drive via MCP, learn locators, commit verified steps) -- `debugging-codeceptjs-tests` — **use on every failing test from the first full run** (breakpoint, inspect live page via MCP, fix on the fly) -- `codeceptjs-auth` — replace `Role` + `t.useRole` -- `codeceptjs-fundamentals` — run **after** migration to confirm the new setup is wired correctly +## Related skills + +- `writing-codeceptjs-tests` — per-spec rewrite playbook (MCP-driven, verified steps); also the path to re-author Studio recordings +- `debugging-codeceptjs-tests` — use on every failing test from the first full run +- `codeceptjs-auth` — replaces `Role` + `t.useRole` +- `codeceptjs-fundamentals` — run after migration to confirm wiring; effects (`tryTo`, `within`) replace `t.switchToIframe` +- Reference docs: `node_modules/codeceptjs/docs/` (basics, playwright, locators, custom-helpers, api, assertions, pageobjects, sessions, auth) diff --git a/plugins/codeceptjs/skills/refactoring-codeceptjs-tests/SKILL.md b/plugins/codeceptjs/skills/refactoring-codeceptjs-tests/SKILL.md index 5e3199f..8857455 100644 --- a/plugins/codeceptjs/skills/refactoring-codeceptjs-tests/SKILL.md +++ b/plugins/codeceptjs/skills/refactoring-codeceptjs-tests/SKILL.md @@ -1,53 +1,42 @@ --- name: refactoring-codeceptjs-tests -description: Use when refactoring CodeceptJS 4 tests — cleaning up duplication, extracting page objects, taming long locators, moving raw JS into custom helpers. Works targeted (one test or file) or global (whole tests directory). Always proposes changes before applying. Trigger on phrases like "refactor my tests", "clean up", "extract page object", "this test is too long", "deduplicate", or when reviewing test files for quality. +description: > + Use when cleaning up existing CodeceptJS 4 tests — deduplication, extracting + page objects, taming long locators, moving raw JS into custom helpers. + Targeted (one file) or global (whole tests directory); always proposes before + applying. Trigger on "refactor my tests", "clean up", "extract page object", + "this test is too long", "deduplicate", or when reviewing test files for + quality. --- # Refactoring CodeceptJS 4 Tests -Test suites rot in three predictable ways: duplicate UI flows copy-pasted across files, fat locators repeated everywhere, and raw JS escaping into Scenarios. The fix for each is moving the pattern to its proper home — page object, custom helper, or the `auth` plugin — without changing test behaviour. +Suites rot in three predictable ways: duplicated UI flows copy-pasted across files, fat locators repeated everywhere, raw JS escaping into Scenarios. The fix for each is moving the pattern to its proper home without changing behaviour. -This skill works **targeted** (refactor one test or file the user named) or **global** (sweep the whole tests directory). Either way it **proposes changes first** and applies after approval, in reviewable batches. +Works **targeted** (a file or Scenario the user named) or **global** (the whole configured `tests` glob). Either way: **propose first**, apply in reviewable batches after approval. ## Workflow -### 1. Read the project (fundamentals) -Run the **codeceptjs-fundamentals** skill. You need: registered page objects (keys under `include`), the actor file (`custom_steps.js` or similar), any custom helpers already configured, and the `auth` plugin's user list. Without this you don't know where extractions land. - -### 2. Pick scope -- **Targeted** — a file or Scenario the user named. Read it once, propose, apply. -- **Global** — the configured `tests` glob. Treat as a multi-pass cycle: inventory → propose grouped → approve → apply a batch → re-run affected Scenarios → next batch. Never one big edit. - -### 3. Inventory the target -Read the test(s). Note: -- Repeated `I.*` sequences across 2+ Scenarios. -- Locator strings used in 2+ places, or single locators that span multiple lines / deep XPath / nested CSS. -- `I.executeScript` blocks beyond one statement. -- `usePlaywrightTo` / `usePuppeteerTo` / `useWebDriverTo` blocks doing business work (not inspection). -- Hardcoded credentials, URLs, magic strings. -- `I.wait(N)` raw-seconds calls and `await` on plain action steps — fix while you're there. - -### 4. Duplicate UI flows → page object methods -Sequences like login form fill, modal-confirm, search-then-click, table-row-edit. Extract to a method named for the user's intent: `loginPage.signIn(email, pass)`, `searchBar.searchFor(term)`, `cartPage.removeItem(name)`. Add to the existing PO when one fits the area; otherwise create a new PO and register it under `include` in the config. Apply the **rule of three** — wait for the second occurrence before extracting; one-off code shouldn't be abstracted. - -### 5. Fat locators → `locate()` in page objects -Multi-line XPath, deeply nested CSS, or repeated locator strings. Convert to the `locate(...)` builder, leaning on the new 4.x DSL methods (`withClass`, `withoutClass`, `and`, `andNot`, plus `withText`, `inside`, `withDescendant`). Used once → keep `locate()` inline at the call site. Used 2+ times → store as a named page-object field with `.as('description')` so failures point at a meaningful name. - -### 6. Raw JS → custom helper -Big `I.executeScript(...)` blocks, `usePlaywrightTo` doing real DOM walking or business logic, hand-rolled fetches inside a Scenario. Move into a method on a custom helper. Register the helper under `helpers` in the config alongside built-ins. **Critical rule from fundamentals**: inside a custom helper, `I` does **not** exist — compose with another helper via `this.helpers['']` (e.g. `this.helpers['Playwright'].page.evaluate(...)`, `this.helpers['REST'].sendGetRequest(...)`). Tests stay in the `I.*` vocabulary; helpers expose the new capability. - -### 7. Special case — login duplication -If the duplicated UI flow is login, **don't build a `loginPage.signIn` method**. Hand off to the **codeceptjs-auth** skill — that's what the `auth` plugin exists for. Page objects for login flows are a common smell when the project should be using session reuse. - -### 8. Cross-page flows (optional) -Site-wide actions used across many Features — `I.acceptCookies()`, `I.goToBilling()`, `I.dismissNotification()` — belong in the actor (`custom_steps.js`), not a single page's PO. The actor is the right home when no single page owns the action. - -### 9. Propose, then apply -Group proposals by destination file (`pages/loginPage.js`, `helpers/DbHelper.js`, `tests/checkout_test.js`). Show the user the list before editing. In global mode, get explicit approval and work in batches — three to five files at a time. After each batch, run the affected Scenarios: -```bash -npx codeceptjs run --grep '' --steps -``` -Hand the result to **codeceptjs-run-analysis** — confirm every affected scenario still passes and no failure shifted to a new step. Refactors that don't run aren't refactors — they're guesses. +1. **Fundamentals** — run `codeceptjs-fundamentals`. You need: page objects under `include`, the actor file, custom helpers, auth roles. Without this you don't know where extractions land. +2. **Pick scope**: + - Targeted — read once, propose, apply. + - Global — multi-pass cycle: inventory → propose grouped → approve → apply a batch → re-run affected Scenarios → next batch. Never one big edit. +3. **Inventory** — flag: + - Repeated `I.*` sequences across 2+ Scenarios + - Locators used in 2+ places; multi-line XPath / deeply nested CSS + - Unscoped locators carrying their own region (`'.sidebar nav a.settings'`), `'aria-label=X'` spellings, raw `[data-testid=...]` at call sites + - Multi-statement `I.executeScript`; `usePlaywrightTo`/`usePuppeteerTo`/`useWebDriverTo` doing business work + - Hardcoded credentials, URLs, magic strings + - `I.wait(N)` and stray `await` on plain actions — fix while you're there +4. **Duplicate flows → page-object methods** named for user intent (`loginPage.signIn(email, pass)`, `cartPage.removeItem(name)`). Rule of three: extract at the second occurrence; one-off code stays put. +5. **Fat locators** — split before reaching for the builder: structural half becomes the context argument, rest stays a semantic string. + - `I.click('.sidebar nav a.settings')` → `I.click('Settings', '.sidebar')` + - `I.click({ css: '[aria-label="Save"]' })` → `I.click('Save', '.toolbar')` + - `I.click({ css: '[data-qa=submit]' })` → `I.click('$submit', '.checkout')` with `customLocator` + Only what survives the split needs `locate()`: used once → inline at call site; used 2+ times → named PO field with `.as('description')` so failures point at a meaningful name. +6. **Raw JS → custom helper** — `I.executeScript` blocks, `usePlaywrightTo` doing DOM walking or business logic, hand-rolled fetches inside Scenarios. Remember the fundamentals rule: **`I` does not exist inside a helper** — compose via `this.helpers['']`. Tests stay in the `I.*` vocabulary. +7. **Login duplication** — don't build a `loginPage.signIn` method. Hand off to `codeceptjs-auth`: that's what the `auth` plugin exists for. Login POs are a smell when session reuse applies. +8. **Site-wide actions** (`I.acceptCookies()`, `I.goToBilling()`) — actor file (`custom_steps.js`), not any single page's PO. Right home when no single page owns the action. ## Decision tree @@ -56,26 +45,34 @@ Hand the result to **codeceptjs-run-analysis** — confirm every affected scenar | UI flow on one page | Method on that page's PO | | UI flow spanning pages, site-wide | Actor (`custom_steps.js`) | | Login | `auth` plugin (see `codeceptjs-auth`) | +| Long locator carrying its own region | Short semantic locator + context arg | | Long locator used once | `locate()` inline | -| Long locator used 2+ times | `locate()` stored as a PO field | +| Long locator used 2+ times | `locate()` stored as PO field | | Raw browser API / file system / DB / mail | Custom helper | | Test-data setup hitting an API | REST helper + Data Object | +## Propose, then apply + +Group proposals by destination file (`pages/loginPage.js`, `helpers/DbHelper.js`, `tests/checkout_test.js`). Show the list before editing. In global mode: explicit approval, batches of three to five files, re-run after each batch: + +```bash +npx codeceptjs run --grep '' --steps +``` + +Hand output to `codeceptjs-run-analysis` — confirm every affected Scenario still passes and no failure shifted to a new step. Refactors that don't run aren't refactors. + ## Things to avoid - Refactoring without re-running the affected Scenarios afterwards. -- Extracting a flow that's only used once — wait for the second occurrence. -- Renaming PO methods or fields without grepping for every caller first. -- Moving code to a custom helper when a page object would do. +- Extracting a flow used only once — wait for the second occurrence. +- Renaming PO methods/fields without grepping every caller. +- A custom helper where a page object would do. - Squashing distinct flows into one method (`loginPage.do(thing)` is a smell). -- Touching Scenario names or tags — CI pipelines and `--grep` filters reference them. -- Mass-applying changes in global mode without batching and re-running. +- Touching Scenario names or tags — CI and `--grep` reference them. +- Mass-applying in global mode without batching and re-running. -## Pointers +## Related skills -- `node_modules/codeceptjs/docs/pageobjects.md` — class-based POs, `inject()`, lifecycle hooks -- `node_modules/codeceptjs/docs/custom-helpers.md` — extending `Helper`, the `this.helpers[...]` rule -- `node_modules/codeceptjs/docs/locators.md` — `locate()` builder + 4.x DSL methods -- `node_modules/codeceptjs/docs/best.md` — broader test-organisation guidance -- `codeceptjs-fundamentals` skill — DI, `inject()`, the `I`-unreachable-from-helpers rule -- `codeceptjs-auth` skill — for login deduplication +- `codeceptjs-fundamentals` — Main rule, Where things go, Architecture (`I`-unreachable rule) +- `codeceptjs-auth` — login deduplication +- `codeceptjs-run-analysis` — post-refactor verification diff --git a/plugins/codeceptjs/skills/writing-codeceptjs-tests/SKILL.md b/plugins/codeceptjs/skills/writing-codeceptjs-tests/SKILL.md index 419e850..b883599 100644 --- a/plugins/codeceptjs/skills/writing-codeceptjs-tests/SKILL.md +++ b/plugins/codeceptjs/skills/writing-codeceptjs-tests/SKILL.md @@ -1,206 +1,102 @@ --- name: writing-codeceptjs-tests -description: Use when writing a new CodeceptJS 4 test, extending an existing Scenario, or porting a manual test plan to code. Builds tests live — opens the real page through the CodeceptJS MCP server, queries ARIA/HTML to learn locators, runs each step incrementally to verify it works, then commits the verified sequence to a test file. Two authoring modes — Mode A (incremental `run_code`) for known flows, Mode B (scaffold-and-pause — write `I.amOnPage(...); pause();`, run via MCP, drive the live browser, replace `pause()` with the verified sequence) for greenfield / unknown flows. Never invents locators or flows from imagination; drives the actual browser. Trigger on any request to create, write, add, draft, or scaffold a CodeceptJS test, login flow, end-to-end check, or "test from scratch". +description: > + Use when creating a new CodeceptJS 4 test, extending an existing Scenario, or + porting a manual test plan to code. Builds tests against the live browser via + the CodeceptJS MCP server — never from imagined locators or flows. Trigger on + any request to create, write, add, draft, or scaffold a CodeceptJS test, + login flow, end-to-end check, or "test from scratch". --- # Writing CodeceptJS 4 Tests -A test that was never executed during authoring is unreliable by definition. The right way to write a Scenario is to drive the real browser through the CodeceptJS MCP server, query the page to learn locators, and only commit steps that actually pass. This skill is the playbook for that loop. +A test that was never executed during authoring is unreliable. Drive the real browser via the CodeceptJS MCP server, verify every locator against the live page, commit only steps that passed. -Two authoring modes, picked by how much of the flow you already know: +Two modes, picked by how much of the flow you already know: -- **Mode A — incremental `run_code`** — send one or two `I.*` lines per `run_code`, read the response, repeat. Suited to extending an existing test or porting a manual plan with known steps. -- **Mode B — scaffold-and-pause** (recommended for greenfield / unknown flows) — write a stub `Scenario` containing `I.amOnPage('/...'); pause();`, run via MCP `run_test`. The browser opens and yields control at `pause()` on the live container. Drive the page through `run_code` to discover the flow, then edit the test to replace `pause()` with the verified sequence and re-run. +- **Mode A — incremental `run_code`**: send one or two `I.*` lines per call, read response, repeat. For extending existing tests, known flows, porting manual plans. +- **Mode B — scaffold-and-pause** (recommended for greenfield / unknown flows): write a stub Scenario containing `I.amOnPage('/...'); pause();`, run via MCP `run_test`. The browser opens and yields control at `pause()` — drive the live page via `run_code`, then replace `pause()` with the verified sequence. -Both modes share the discovery / locator / commit steps below; the difference is just *where the in-flight exploration happens*. +Both share the same discovery / locator / commit steps; the difference is *where the in-flight exploration happens*. ## Workflow -### 1. Read the project (fundamentals) -Run the **codeceptjs-fundamentals** skill first. You need: active web helper, base URL, plugins (especially `aiTrace`, `auth`), AI provider, page-object names, env vars. - -### 2. Map what's already there -Before adding anything, enumerate: -- `npx codeceptjs check -c ` — verifies the setup loads (config, helpers, plugins, page objects, hooks, tests, defs). Each pass/fail line doubles as an inventory of what the project has wired up. -- `npx codeceptjs list -c ` — every available `I.` for the active helpers, including custom ones. Check before suggesting any method. -- `npx codeceptjs dry-run -c ` — every Scenario the active config would load, with naming conventions and tags. Add `--steps` to also print queued `I.*` calls, `--numbers` for step indices. -- MCP equivalents: `list_actions`, `list_tests`. - -This catches duplication, surfaces patterns the new test should follow, and confirms a custom step / page-object method doesn't already cover the planned flow. - -> ⚠ `dry-run` does **not** initialize plugins. The `auth` plugin's injected `login(...)` (and any other plugin-injected function) is undefined under `dry-run`, so `Before(({ login }) => login('admin'))` raises **"login is not a function"** even though the actual `run` works fine. Use `--steps` to inspect Scenario shape; ignore plugin-inject errors. To verify auth works, do a real `run` (or MCP `run_test`). - -### 3. Decide if auth is needed -If the path under test sits behind login, invoke the **codeceptjs-auth** skill: -- If `auth` is already configured, use the existing role: `Before(({ login }) => login())`. -- If not, the auth skill walks through adding the plugin and env-var credentials. -- Public path? Skip. - -### 4. Look for similar tests and page objects -Before writing anything new, scan: -- existing Scenarios that touch the same feature area -- page objects in the directories registered under `include` -- custom steps in the actor file -- data factories that already create the entities the test needs (user, post, …) - -If a page-object method already encodes the locators for this area, drive through it (`profilePage.openSettings()`) instead of writing raw `I.click` chains. - -### 5. Identify the starting page -The page where the new test does its work, **after** any auth. Get a real URL, not a guess. Look for similar tests and page objects. If you don't have that data, ask user where to start. CodeceptJS always use relative urls. The host must be set inside config (helpers.Playwright.url or helpers.WebDriver.url or helpers.Puppeteer.url) - -### 6. Make sure aiTrace + MCP are wired -- **aiTrace under MCP** — auto-enabled. The MCP server forces `plugins.aiTrace = { on: 'step', enabled: true }` for every session (`bin/mcp-server.js`), so you do **not** need to add it to the project config when authoring through MCP. The user can override with `start_browser({ plugins: { aiTrace: { enabled: false } } })`. -- **aiTrace for CLI runs** — *not* auto-enabled. Either add `plugins: { aiTrace: { enabled: true } }` to the active config, or pass `-p aiTrace` on the runner. Without it, the verification run in step 10 produces no `output/trace_*/` artifacts and `codeceptjs-run-analysis` has nothing to read. -- **MCP** — confirm the AI client points at `node_modules/codeceptjs/bin/mcp-server.js` with `CODECEPTJS_CONFIG` and `CODECEPTJS_PROJECT_DIR` set. See `node_modules/codeceptjs/docs/mcp.md` if it isn't. -- **Headless** — run tests headlessly by default. Either rely on `setHeadlessWhen(CI)` (export `CI=1` for the session) or set `show: false` in the helper config. - -### 7. Open a live session via MCP - -Pick a mode: - -**Mode A — incremental `run_code`** (existing test extension, known flow): -Send a minimal scaffold to MCP `run_code` — `login()` if auth is needed, then `I.amOnPage()`. The response includes URL, ARIA snapshot, screenshot, and console logs. This is the ground truth for everything that follows. - -**Mode B — scaffold-and-pause** (greenfield / unknown flow): -Write a draft test stub directly in the test file — minimal but real. For a public page: - -```js -Scenario('draft - feature exploration', ({ I }) => { - I.amOnPage('/') - pause() -}) -``` - -…or with auth: - -```js -Before(({ login }) => login('admin')) - -Scenario('draft - feature exploration', ({ I }) => { - I.amOnPage('/dashboard') - pause() -}) -``` - -Run it via MCP `run_test`. The browser opens, navigates, and yields control at `pause()` — the response carries `{ status: 'paused', pausedAfter, page, suggestions }`. The same `I` / browser the test is using is now driven by `run_code` against the live page. - -### 8. Learn the page and pick locators -Hand off to the **codeceptjs-exploration** skill: read the ARIA snapshot first, fall back to HTML when needed, and use `I.grabWebElement` / `I.grabWebElements` (with permissive XPaths when the obvious locator misses) to enumerate and disambiguate candidates. Commit a locator only after verifying it matches exactly one element via MCP `run_code`. In Mode B, this exploration happens *during the pause window* — the page is sitting there waiting for you. - -Translate devtools-style strict locators to readable form **before** they hit the test file — see the **Locators** section below. - -### 9. Build the Scenario via MCP - -**Mode A** — for each user goal (fill a field, click a button, see a confirmation), run one or two CodeceptJS commands through MCP `run_code`, then read the response. - -**Mode B** — work the live page from the pause: try one or two `I.*` lines via `run_code`, read the response, navigate the next step. Each successful command goes into a scratchpad you'll paste back into the test file in step 10. When you've reached the end of the flow you wanted, the scratchpad is your verified sequence. - -After every command (either mode) ask: -- Did the URL or ARIA change the way you expected? -- Any new errors in the console logs? -- Does a `grab*` value match what was expected? - -If a step fails — try a different locator, add a `waitFor*`, or reconsider the flow. **Stop and ask the user** when something is genuinely ambiguous (two "Save" buttons; an unclear empty state; a feature flag that might not be enabled). Don't push through. - -### 10. Commit the verified sequence -When every step has worked once in isolation, paste them into a test file: -- Match existing file naming and the **one-Feature-per-file** rule. -- Use a page-object method or custom step wherever one fits — don't duplicate selectors. -- **Translate every locator to its readable form** (see the **Locators** section). No `{ css: 'input[placeholder*="…"]' }`, no `{ xpath: '//tr[contains(.,…)]//…' }` in committed code. Strict locators are a code-review red flag unless nothing semantic, ARIA, or `locate()`-shaped fits. -- Wrap secrets with `secret(...)`; pull credentials from env vars only. -- Add the tag the suite already uses (`{ tag: '@smoke' }`) when relevant. -- **Mode B specific** — replace the `pause()` line with the verified sequence above it. Remove the `draft` Scenario name and rename to its real intent. Don't leave `pause()` in a committed test. -- Run the file end-to-end with `aiTrace` enabled: `npx codeceptjs run --grep '' --steps`. Hand the result to **codeceptjs-run-analysis** — it reads `output/trace_*/` artifacts via bash tools so you can confirm the flow ran clean. Only declare done when the scenario passes there. - -## Locators — readable, semantic, scoped - -Pick the highest-level form that fits. Priority, top wins: - -1. **Dedicated test attributes** — when the page consistently exposes `data-testid` / `data-qa` / similar. Stable by design. - ```js - I.click('[data-testid="submit-order"]') - ``` -2. **Semantic + context** — plain string (label / button text / placeholder / `aria-label`). - ```js - I.click('Save', '.toolbar') - I.fillField('Email', 'u@t.com', '#login-form') - ``` -3. **ARIA role** — survives markup churn; doubles as accessibility check. - ```js - I.click({ role: 'button', name: 'Sign In' }) - ``` -4. **`locate()` builder** — for structural conditions, repeated row/cell targeting, anything CSS can't express in one line. - ```js - I.click(locate('button').withText('Edit').inside(locate('tr').withText('Acme Corp'))) - ``` -5. **Strict `{ id }` / `{ name }` / `{ css }`** — when nothing above fits. - ```js - I.fillField({ id: 'email' }, 'u@t.com') - I.seeElement({ css: '.invoice-row.paid' }) - ``` -6. **`{ xpath }`** — last resort, for axes (`ancestor`, `following-sibling`) or text predicates the builder doesn't cover. - ```js - I.click({ xpath: '//button[@aria-pressed="true"]' }) - ``` - -If item 1 applies broadly across the app, enable the **`customLocator` plugin** so `$name` resolves to the configured attribute. Once it's on: - -```js -// before // after, with customLocator: { attribute: 'data-qa' } -I.click({ css: '[data-qa=submit]' }) I.click('$submit') -I.fillField({ css: '[data-qa=email]' }, ...) I.fillField('$email', ...) -``` - -See `node_modules/codeceptjs/docs/plugins.md` § *customLocator*. - -### Context - -Most actions accept a **context** as the last argument — any locator type. The lookup runs only inside that region, which disambiguates duplicate labels and keeps semantic strings usable. When several actions target the same region, bind the locator once and reuse it; do not repeat the chain inline. - -Prefer stable structural regions — they survive UI rewrites better than the markup they wrap: - -- Landmark elements — `nav`, `main`, `header`, `footer`, `aside`, `{ role: 'dialog' }`, `{ role: 'navigation' }`. -- App-shell containers — `.main-app`, `.app-content`, `.sidebar`, `.toolbar`, `.modal`, `.drawer`. -- A row, card, or list item identified by its data, expressed via `locate(...)`. - -**`I.see` and `I.dontSee` require a context.** Their first argument is plain text matched across the page, so without a context the assertion can resolve against navigation, a footer, or an unrelated component and produce a false pass. Other assertions (`I.seeElement`, `I.seeNumberOfElements`, etc.) take an explicit locator, which scopes them on its own. - -### Picking a specific match - -When a locator matches several elements, CodeceptJS uses the first by default. To target another match without writing a more specific locator, pass `elementIndex` via `step.opts()` (`import step from 'codeceptjs/steps'`) — accepts a 1-based number, a negative index, or `'first'` / `'last'`. Same path for `step.opts({ exact: true })` to make a single step throw on ambiguity. See `node_modules/codeceptjs/docs/locators.md` § *Picking a specific element*. - -Full reference: `node_modules/codeceptjs/docs/locators.md`. - -## Waiting - -CodeceptJS auto-waits before each action, but explicit waits are still needed when: -- a **loader / spinner / skeleton** must hide before the next step → `I.waitForInvisible('.spinner')`, `I.waitForDetached('.skeleton')` -- a **modal / drawer / panel / section** hasn't rendered yet → `I.waitForVisible('.modal')`, `I.waitForElement({ role: 'dialog' })` -- **data must finish loading** — list rows, cards, charts, async text → `I.waitForElement('.user-row', 10)`, `I.waitForText('Loaded', 10, '.status')` - -Detect what to wait for by reading the page HTML / ARIA between MCP steps. If the next element is gated by a spinner overlay or rendered after a fetch, scroll the markup until you find the gating element, pick a stable selector, and wait for the right state (visible, invisible, detached, text-present). - -`I.wait(N)` (raw seconds) is OK during **authoring** to confirm a timing hypothesis — if a 5-second sleep makes the step pass, the cause is timing. **Replace it with a specific `I.waitFor*` before committing.** Hardcoded sleeps are slow on fast machines, flaky on slow ones, and hide the real sync point. +1. **Fundamentals first** — run `codeceptjs-fundamentals`. You need: active web helper, base URL, plugins (`aiTrace`, `auth`), AI provider, page objects, env vars. +2. **Map what's already there** — `check` / `list` / `dry-run` (see fundamentals § Discover). Catches duplication and confirms no custom step or page-object method already covers the planned flow. + - ⚠ `dry-run` does **not** initialize plugins: `Before(({ login }) => login('admin'))` raises "login is not a function" under dry-run even though a real run works. Inspect shape with `--steps`; ignore plugin-inject errors; verify auth with a real run. +3. **Auth** — path behind login → invoke the `codeceptjs-auth` skill. Existing role configured: `Before(({ login }) => login())`. Not configured: auth skill walks adding it. Public path: skip. +4. **Similar tests & page objects** — scan Scenarios in the feature area, POs under `include`, actor-file custom steps, data factories. If a PO method already encodes this area's locators, drive through it instead of raw `I.click` chains. +5. **Starting page** — a real URL *after* any auth, not a guess; ask the user if unknown. Relative URLs only — host lives in the config (`helpers.Playwright.url` etc.). +6. **aiTrace + MCP wiring**: + - Under MCP: `aiTrace` is forced on by the MCP server — nothing to configure. + - CLI runs: *not* auto-enabled — declare in config or pass `-p aiTrace`, or step 11 produces no `output/trace_*/` artifacts for run-analysis. + - Run headless by default (`setHeadlessWhen(CI)` with `CI=1` exported, or `show: false`). + - Confirm MCP client points at `node_modules/codeceptjs/bin/mcp-server.js` with `CODECEPTJS_CONFIG` and `CODECEPTJS_PROJECT_DIR` set. +7. **Open a live session** (pick mode): + - Mode A: `run_code` scaffold — `login()` if needed, then `I.amOnPage()`. The response (URL, ARIA snapshot, screenshot, console) is ground truth for everything after. + - Mode B: write a minimal-but-real draft in the test file: + ```js + Before(({ login }) => login('admin')) // if needed + + Scenario('draft - feature exploration', ({ I }) => { + I.amOnPage('/dashboard') + pause() + }) + ``` + Run via MCP `run_test` → `{ status: 'paused', pausedAfter, page, suggestions }`. The test's own `I` / browser is now driven by `run_code`. +8. **Learn the page** — hand off to `codeceptjs-exploration`: ARIA snapshot first, HTML fallback, enumerate and disambiguate candidates, commit a locator only after verifying it matches exactly one element via `run_code`. In Mode B this happens inside the pause window. +9. **Build the Scenario** — one or two commands at a time via `run_code` into a scratchpad. After each ask: did URL/ARIA change as expected? New console errors? Do grabbed values match expectations? + - Step failed → try a different locator, add a specific `waitFor*`, or reconsider the flow. + - Genuinely ambiguous (two Save buttons, unclear empty state, possible feature flag) → **stop and ask the user**. + - Optional UI elements (cookie banners) → `await tryTo(...)` instead of `if` (fundamentals § Effects) — keeps scenarios linear. +10. **Commit the verified sequence**: + - Match existing naming; one `Feature` per file. + - Use page-object methods / custom steps where they fit — don't duplicate selectors. + - Translate every locator to readable form (priority below). Strict `{ css }` / `{ xpath }` in committed code are a review red flag unless nothing else fits. + - Credentials from env vars only, wrapped in `secret(...)`. + - Mode B: replace the `pause()` line with the sequence; rename `draft - ...` to the real intent. +11. **Final verification**: `npx codeceptjs run --grep '' --steps` with aiTrace enabled → hand output to `codeceptjs-run-analysis` to confirm the flow ran clean in `output/trace_*/`. Done only when it passes there. + +## Locator priority (writing time) + +Always pass context — see `codeceptjs-fundamentals` § Locators for rationale. Top wins: + +1. **Semantic string** — visible text, label, placeholder, `name`, `aria-label`: `I.click('Save', '.toolbar')` +2. **ARIA role** — text ambiguous within context ("Delete" link *and* button), or role part of the assertion: `I.click({ role: 'button', name: 'Sign In' }, '#login-form')` +3. **`$name` via `customLocator`** — app exposes `data-testid`/`data-qa` broadly +4. **`locate()` builder** — structural conditions; often the structural half belongs in the context: `I.click('Edit', locate('tr').withText('Acme Corp'))` +5. **Strict `{ id }` / `{ name }` / `{ css }`** — last resorts above exhausted +6. **`{ xpath }`** — axes / text predicates the builder can't express + +Writing-time specifics beyond fundamentals: + +- Plain strings already match `aria-label` — never write `'aria-label=Save'` or `{ css: '[aria-label="Save"]' }`. +- Prefer stable contexts: landmarks (`nav`, `main`, `{ role: 'dialog' }`), app-shell containers (`.sidebar`, `.modal`), rows/cards identified by their data. +- **`I.see` / `I.dontSee` require a context** — their first arg is plain text matched across the whole page; unscoped they can false-pass on nav/footer content. +- Several matches → `step.opts({ elementIndex: N })` (1-based, negative, `'first'`/`'last'`) or `step.opts({ exact: true })`; import `step` from `codeceptjs/steps`. + +## Waiting while authoring + +- Auto-waiting covers interactions; detect gating elements (spinner overlay, post-fetch render) from the live HTML/ARIA between MCP steps → pick the stable selector and a specific `waitFor*`. +- `I.wait(N)` is acceptable **during authoring** to confirm a timing hypothesis — if a sleep makes the step pass, timing is the cause. Replace with the specific `waitFor*` before committing. ## Things to avoid -- Writing tests from imagined locators or imagined routes. -- Defaulting to strict `{ css }` / `{ xpath }` locators when a semantic string, ARIA role, or `locate()` chain would do. See the **Locators** section. -- Hardcoding credentials anywhere — env vars + `secret()` only. -- Skipping the page-object scan. -- Adding `await` to plain action steps (see fundamentals' `await` rule). -- Adding `waitFor*` speculatively before checking whether auto-waiting already handles it. -- Leaving `I.wait(N)` (raw seconds) in committed tests — replace with a specific `waitFor*`. -- Leaving the `pause()` from Mode B's stub in the committed test — it must be replaced with the verified sequence before the file is done. -- Using Mode A when the flow is genuinely unknown — round-tripping `run_code` for every step is slower than Mode B and easier to lose track of state in. -- Declaring the test done without running the committed file end-to-end with `aiTrace` enabled. - -## Pointers - -- `node_modules/codeceptjs/docs/basics.md` — locators, assertions, waits, hooks -- `node_modules/codeceptjs/docs/test-structure.md` — Feature/Scenario syntax -- `node_modules/codeceptjs/docs/locators.md`, `docs/element-selection.md` -- `node_modules/codeceptjs/docs/pageobjects.md`, `docs/sessions.md`, `docs/within.md` -- `node_modules/codeceptjs/docs/data.md` — REST helper, Data Objects -- `node_modules/codeceptjs/docs/mcp.md` — MCP tool list and client config -- `node_modules/codeceptjs/docs/secrets.md` — `secret()` wrapper +- Writing tests from imagined locators or routes — everything from the live page. +- Strict locators where semantic / ARIA / `locate()` fits. +- Long unscoped locators instead of short locator + context. +- Spelling out accessible names or repeating `[data-testid=...]` at call sites. +- Hardcoded credentials anywhere. +- Skipping the similar-test / page-object scan. +- `await` on plain action steps (fundamentals await rule); speculative `waitFor*` before checking auto-waiting. +- Leaving `I.wait(N)` or `pause()` in committed tests. +- Using Mode A for genuinely unknown flows — slower than Mode B, easier to lose state. +- Declaring done without the end-to-end aiTrace run. + +## Related skills + +- `codeceptjs-fundamentals` — rules, effects, discovery (run first) +- `codeceptjs-exploration` — page inspection, WebElement API +- `codeceptjs-auth` — login/session reuse +- `codeceptjs-run-analysis` — trace verification +- `debugging-codeceptjs-tests` — when the committed test misbehaves