Skip to content

Rewrite CodeceptJS skills: bullets over prose, doc-verified content, cross-references #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DavertMik merged 2 commits into from
Aug 23, 2026
Merged

Rewrite CodeceptJS skills: bullets over prose, doc-verified content, cross-references #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6daf472
Rewrite CodeceptJS skills: bullets over prose, doc-verified content, …
Aug 21, 2026
512b7e3
Trim skill descriptions to invocation triggers
Aug 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions plugins/codeceptjs/skills/ci-fix-tests/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: ci-fix-tests
description: "Use on CI in non-interactive mode when a CodeceptJS run failed — automatically attempt safe fixes (locator drift, missing waits), rerun only the failing scenarios, compare against the baseline, roll back any edit that didn't help, and always write a markdown report at `output/ci-fix.md` for the CI step to consume. Conservative by design: no refactors, no config edits, no auth fixes, no flake-masking retries. Trigger on \"ci fix tests\", \"auto-fix failing tests\", \"attempt repair on CI\", or as a CI step after a failed run."
description: "Use on CI in non-interactive mode after a failed CodeceptJS run — attempts safe fixes only (locator drift, missing waits), reruns failing scenarios, rolls back edits that didn't help, reports at `output/ci-fix.md`. No refactors, no config edits, no flake-masking. Trigger on \"ci fix tests\", \"auto-fix failing tests\", \"attempt repair on CI\", or as a CI step after a failed run."
---

# Auto-fix CodeceptJS Tests on CI
Expand All @@ -25,7 +25,7 @@ Save this as the **baseline** — `count`, `failing_set`, `clusters`.
### 2. Pick safe fix candidates
Only attempt fixes from this allowed list:

- **Locator drift** — the failed locator no longer matches anything but a similar element exists. Use the **codeceptjs-exploration** skill (headless) to find candidates; pick one with high semantic stability (ARIA `{ role, name }` → visible text → `data-testid` → composed CSS). Replace the locator at the failing step only.
- **Locator drift** — the failed locator no longer matches anything but a similar element exists. Use the **codeceptjs-exploration** skill (headless) to find candidates; pick a short locator scoped to a stable region — `I.click('Save', '.toolbar')` — in this order: visible text / accessible name → ARIA `{ role, name }` → `$name` (`customLocator`) → composed CSS. Replace the locator at the failing step only.
- **Missing wait for a spinner / loader / modal** — the failed step's ARIA snapshot shows a spinner or skeleton present, or the target element appears later. Add a single matching `I.waitFor*` immediately before the failing step.
- **`I.wait(N)` replacement** — when a hardcoded sleep is the only thing between a failing assertion and a passing one and the gating element is identifiable, replace the sleep with a specific `waitFor*`.

Expand Down Expand Up @@ -81,7 +81,7 @@ Failing scenarios: N
- ...

## Attempted fixes
- `tests/foo_test.js:42` — locator drift: `'Save'` → `{ role: 'button', name: 'Save' }`
- `tests/foo_test.js:42` — locator drift: `I.click('Save')` → `I.click('Save', '.toolbar')`
- `tests/bar_test.js:15` — added `I.waitForInvisible('.spinner')` before checkout click
- `tests/baz_test.js:7` — replaced `I.wait(3)` with `I.waitForVisible('.confirmation-dialog', 10)`

Expand Down Expand Up @@ -112,9 +112,8 @@ The first line of `Status:` is the machine-parseable signal. The rest is for the
- Writing a partial or missing `output/ci-fix.md`. CI depends on it; the absence of a report is itself a failure mode.
- Running the **whole** suite for the verification step — only the originally-failing scenarios, to keep CI time bounded.

## Pointers
## Related skills

- `codeceptjs-run-analysis` — read trace artifacts, cluster failures, build the baseline set.
- `codeceptjs-exploration` — find replacement locators when one has drifted.
- `codeceptjs-fundamentals` — confirm helper, config, which env the CI run used.
- `node_modules/codeceptjs/docs/aitrace.md` — trace format.
- `codeceptjs-run-analysis` — baseline, clustering, post-fix comparison
- `codeceptjs-exploration` — replacement locators for drift
- `codeceptjs-fundamentals` — config, helper, CI environment
111 changes: 47 additions & 64 deletions plugins/codeceptjs/skills/codeceptjs-auth/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,51 +1,45 @@
---
name: codeceptjs-auth
description: Use when a CodeceptJS test needs login, when different user roles are involved, or when the writing-codeceptjs-tests skill identifies authorization is required. Configures the `auth` plugin for session reuse, derives the login flow from the actual login page HTML (not guesses), keeps the real flow inside `steps_file.js` so `I.login*()` is callable directly and the conf stays small, loads credentials from a `.env` file via the modern Node `process.loadEnvFile()` API (no `dotenv` dependency), and supports multiple roles. Trigger on mentions of login, sign-in, sign-up, authentication, sessions, "logged in", admin/editor/user roles, or auth-related test failures.
description: >
Use when a CodeceptJS test needs login, user roles, or session reuse — or
when another CodeceptJS skill identifies authorization is required. Also on
auth-related test failures. Trigger on login, sign-in, sign-up,
authentication, sessions, "logged in", admin/editor/user roles.
---

# CodeceptJS Auth Plugin

The `auth` plugin logs each user in once, captures cookies (or local-storage / token via overrides), and restores the session for subsequent tests. Stale sessions trigger a fresh login automatically.
The `auth` plugin logs each user in once, captures cookies (or localStorage/token via overrides), and restores the session for subsequent tests. Stale sessions trigger a fresh login automatically.

Source of truth: `node_modules/codeceptjs/lib/plugin/auth.js` — JSDoc lists every recipe. Reference doc: `node_modules/codeceptjs/docs/auth.md`.

## When to add it

Suggested if tests have repeatable authentication and session is needed to be persisted accross tests. Also optimizes start time of tests by saving previous sessision cookies in files.

Suggested as performance optimization, refactoring measure.

If `auth` plugin already exists in the project (check fundamentals' output), reuse the existing `inject` name and user keys.
If the project already has `auth` configured (fundamentals' discovery output), **reuse** its existing inject name and user keys — don't reconfigure.

## Decide first — ask the user

Four answers shape the plugin. Don't guess any. If the project or test plan doesn't make them obvious, ASK.
Four answers shape the plugin. Don't guess any.

1. **Is a session needed at all?** Public flows (landing, signup) don't need one — skip the plugin.
2. **One user or many?** Default to one. Add more only when more than one is actually exercised.
3. **If many — what splits them?** Don't assume "admin / editor / viewer". Real systems split users by role, workspace / tenant, plan tier, sign-in provider (Google vs SSO vs password — same person), or per-test fixture. ASK the user; use the answer to name the `users.<key>` entries.
4. **What's the auth type?** Form login is the default; others need a different `login()` body:
- **Form** — `fillField` → `click Sign in`. Canonical shape below.
- **OAuth / SSO** — click provider button, drive the IdP page (often a separate origin).
- **Magic link / passwordless** — UI flow is rarely worth automating. Prefer a backdoor — API mint, or read the link from a test mailbox.
- **API token** — skip the form; `I.executeScript` to write the token into `localStorage`, or `I.setCookie(...)`.
- **2FA / OTP** — `async login`; fetch the code from a test mailbox / backdoor before submitting.
1. **Is a session needed at all?** Public flows (landing, signup) → skip the plugin.
2. **One user or many?** Default one; add more only when actually exercised.
3. **If many — what splits them?** Role, workspace/tenant, plan tier, sign-in provider, per-test fixture — real systems vary. Ask; use the answer to name `users.<key>` entries.
4. **What's the auth type?**
- **Form** — default; canonical shape below
- **OAuth / SSO** — click provider button, drive the IdP page (often separate origin)
- **Magic link / passwordless** — UI flow rarely worth automating; prefer an API mint or reading the link from a test mailbox
- **API token** — skip the form; write the token into `localStorage` via `executeScript`, or `I.setCookie(...)`
- **2FA / OTP** — async `login`; fetch the code from a test mailbox / backdoor before submitting

## Rules

1. **Login flow must not be written in the conf.** Better to place it nto steps_file.js (if it is includded) or page object. The conf better to reference it like: `login: (I) => I.login()`.
2. **Credentials should not be stored in configs. Use `.env` via `process.loadEnvFile()`** (modern Node, no `dotenv` package). Passwords wrapped with `secret(...)`. No literal credentials anywhere — conf, steps file, test, or git history.
3. **`.env` is gitignored; `.env.example` is committed** with the var names and no values. `output/*_session.json` is gitignored too.
1. **Login flow never lives in the conf.** Put it in `steps_file.js` (if included) or a page object; conf only references it: `login: (I) => I.login()`.
2. **Credentials from env only** — `.env` loaded via `process.loadEnvFile()` (no dotenv dependency). Passwords wrapped with `secret(...)`. No literals anywhere — conf, steps file, test, git history.
3. **`.env` is gitignored; `.env.example` is committed** with names, no values. Gitignore `output/*_session.json` too.

## Canonical shape

```js
// codecept.conf.js — first line of the file
process.loadEnvFile() // or dotenv.load() if this package availble
// ...
process.loadEnvFile()

export const config = {
// ...
include: { I: './steps_file.js' },
plugins: {
auth: {
Expand Down Expand Up @@ -79,56 +73,46 @@ export default function () {
}
```

```
# .env (gitignored) # .env.example (committed)
USER_EMAIL=USER@example.com USER_EMAIL=
USER_PASSWORD=<secret> USER_PASSWORD=
```sh
# .env (gitignored) # .env.example (committed)
USER_EMAIL=... USER_EMAIL=
USER_PASSWORD=<secret> USER_PASSWORD=
```

## Pre-flight (before writing config)

1. **Read the real login page HTML.** Don't guess locators. MCP: `run_code` to login page and inspect the ARIA snapshot. Field labels / `name` / `id` / submit control — from the actual page. It's ok to ask user about authorization if it is not clear how to makeit.
2. **Pick a role-specific post-login marker.** Something the page renders only for *this* user (navbar username, `data-user-role`).
3. **Confirm session storage.** Cookies (default) for server-rendered apps; `localStorage`/`sessionStorage` for SPAs — verify with `I.executeScript(() => Object.keys(localStorage))` after a manual login. Default cookie `fetch`/`restore` silently no-op for token storage.
1. **Read the real login page** — MCP `run_code` to `/login`, inspect the ARIA snapshot (`codeceptjs-exploration`). Field labels / `name` / `id` / submit control from the actual page, not guesses. Unclear authorization mechanism → ask the user.
2. **Pick a role-specific post-login marker** — something rendered only for *this* user (navbar username, `data-user-role`).
3. **Confirm session storage** — cookies (default) for server-rendered apps; localStorage/sessionStorage for SPAs. Verify after a manual login with `I.executeScript(() => Object.keys(localStorage))`. Cookie fetch/restore silently no-op against token storage.

## Verify

Or MCP `run_test` against a one-Scenario file that calls `login(<role>)` then asserts on the post-login marker.

Enable `saveToFile: true` only after the verification round-trip succeeds — a bad saved session masks a broken `login`.

## Refactor

Add to before hooks (if applied to all tests in suite) or to exact tests in syute

`Before(({ login }) => login())`

`login` was declared in auth plugin configuration

Run real, not dry:
Run a one-Scenario file that calls `login(<role>)` then asserts on the post-login marker:

```bash
npx codeceptjs run --grep '<scenario>' --debug
npx codeceptjs run --grep '<scenario>' --debug # real run, not dry — dry-run doesn't init plugins
```

Enable `saveToFile: true` only after this round-trip succeeds — a bad saved session masks a broken `login`.

Then wire into hooks/tests: `Before(({ login }) => login())` for suite-wide, or per-test as needed.

## Multi-session shape
## Multi-role shape

Only after question 3 is answered. Name `users.<key>` after whatever splits them in *this* system; one matching login method per key in `steps_file.js`.
Only after question 3 is answered. Keys named after whatever splits users *in this system*; one matching actor method per key:

```js
// keys named after the dimension (role / workspace / provider / …)
users: {
admin: { login: (I) => I.loginAsAdmin() },
workspaceB: { login: (I) => I.loginToWorkspaceB() },
}
```

Don't parameterise into a single `login(key)` — the plugin keys sessions by name and explicit methods read better. Switching between sessions in one Scenario: `session('<key>')` opens a separate browser context (see `node_modules/codeceptjs/docs/sessions.md`).
Don't parameterise into a single `login(key)` — the plugin keys sessions by name, explicit methods read better. Switch mid-Scenario: `session('<key>')` opens a parallel browser context (fundamentals § Writing tests).

## Token / local-storage auth
## Token / localStorage auth

When sessions live outside cookies, override `fetch` and `restore`:
Override `fetch` / `restore` when sessions live outside cookies:

```js
admin: {
Expand All @@ -142,17 +126,16 @@ admin: {
}
```

`check(I, session)` receives whatever `fetch` returned — throw inside `check` to force a fresh login (e.g., when a `/me` endpoint shows the wrong user).

## Common pitfalls
`check(I, session)` receives whatever `fetch` returned — throw inside `check` to force fresh login (e.g. `/me` endpoint shows wrong user).

- **Credentials inlined in conf or test.** Always env-driven. A code review showing a literal email or password = skill failed.
- **Forgetting to gitignore `.env` and `output/*_session.json`.** Both leak credentials.
## Pitfalls

- Credentials inlined in conf/test — always env-driven + `secret()`.
- Forgetting to gitignore `.env` and `output/*_session.json` — both leak credentials.

## Pointers
## Related skills

- `node_modules/codeceptjs/lib/plugin/auth.js` — JSDoc recipes (cookie / multi-user / local-storage / async / session-validation)
- `node_modules/codeceptjs/docs/auth.md` — full reference
- `node_modules/codeceptjs/docs/sessions.md` — `session()` for multi-user Scenarios
- `node_modules/codeceptjs/docs/secrets.md` — the `secret()` wrapper
- `codeceptjs-fundamentals` — secrets rule, sessions, config mutation trap
- `codeceptjs-exploration` — reading the live login page
- `writing-codeceptjs-tests` / `refactoring-codeceptjs-tests` — invoke this skill when auth is identified
- `debugging-codeceptjs-tests` — auth-related failure patterns
Loading