Run the skill-harness suite in CI; fix pnpm 11 installs

.github/dependabot.yml

@@ -0,0 +1,25 @@
+version: 2
+updates:
+  # Keep the commit-SHA pins in .github/workflows current.
+  - package-ecosystem: github-actions
+    directory: /
+    schedule:
+      interval: weekly
+    groups:
+      actions:
+        patterns: ["*"]
+
+  - package-ecosystem: npm
+    directory: /skill-harness
+    schedule:
+      interval: weekly
+    groups:
+      # One PR per week for all harness dependencies; a failing suite is
+      # the signal that a bump needs a human.
+      harness:
+        patterns: ["*"]
+    ignore:
+      # The harness asserts against a byte-pinned validator bundle built for
+      # this zod major; a major bump is a deliberate re-vendor, not a bot PR.
+      - dependency-name: zod
+        update-types: ["version-update:semver-major"]

.github/workflows/skill-harness.yml

@@ -0,0 +1,80 @@
+# Skill-harness CI (see #33). Offline checks only: the live bench
+# (`pnpm bench`) calls a model API and is deliberately not run here.
+name: skill-harness
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+# Read-only token. No step needs secrets or write access, so fork PRs run
+# with the exact same grants as same-repo PRs.
+permissions:
+  contents: read
+
+# One run per ref. A newer push to a PR cancels that PR's older run; pushes
+# to main are never cancelled, so every main commit keeps its own verdict.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+defaults:
+  run:
+    working-directory: skill-harness
+
+jobs:
+  check:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 10
+
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+
+      # The digest generator imports only node: builtins, so it runs on the
+      # runner's default Node before any toolchain setup. Putting it first
+      # means a stale digest is reported even if the install below fails.
+      - name: Schema digest is in sync
+        working-directory: .
+        run: node scripts/generate-schema-digest.mjs --check
+
+      # pnpm version comes from `packageManager` in skill-harness/package.json.
+      # Do not add a `version:` input here: the action errors when both are
+      # set and disagree, and one source of truth is the point.
+      - uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
+        with:
+          package_json_file: skill-harness/package.json
+
+      # Node version comes from `engines.node` in the same package.json.
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version-file: skill-harness/package.json
+          cache: pnpm
+          cache-dependency-path: skill-harness/pnpm-lock.yaml
+
+      - name: Install (frozen lockfile)
+        id: install
+        run: pnpm install --frozen-lockfile
+
+      # The checks below run independently once install succeeds, so a
+      # single run reports every failure rather than stopping at the first.
+
+      # A frozen install must not want to rewrite the lockfile or the
+      # workspace file; if it does, the committed files are out of date.
+      - name: Lockfile is stable
+        if: ${{ !cancelled() && steps.install.conclusion == 'success' }}
+        working-directory: .
+        run: git diff --exit-code -- skill-harness/pnpm-lock.yaml skill-harness/pnpm-workspace.yaml
+
+      - name: Test
+        if: ${{ !cancelled() && steps.install.conclusion == 'success' }}
+        run: pnpm test
+
+      - name: Biome check
+        if: ${{ !cancelled() && steps.install.conclusion == 'success' }}
+        run: pnpm biome:check
+
+      - name: Typecheck
+        if: ${{ !cancelled() && steps.install.conclusion == 'success' }}
+        run: pnpm typecheck

CLAUDE.md

@@ -22,7 +22,7 @@ If you add a second plugin to the marketplace later, the same rule applies: vers
 
 Production Claude Code caches only each skill's `SKILL.md` in the system prompt; a skill's `references/` directory is **not** auto-loaded for real users. Anything a policy/skill author needs at authoring time must live inline in `SKILL.md`, not in a sidecar file.
 
-The `dtwo-gateway-config` skill's `### Schema Digest` (between `<!-- BEGIN SCHEMA DIGEST -->` / `<!-- END SCHEMA DIGEST -->`) is **generated** from `dtwo/skills/dtwo-gateway-config/schema-reference.json` — do not hand-edit it. Regenerate with `node scripts/generate-schema-digest.mjs`; `--check` fails if it's stale, and the `skill-harness` suite runs exactly that, so `pnpm test` is what enforces it. The vendored schema is a verbatim copy of the artifact the product repo's schema generator emits; refresh it by copying, never by editing in place.
+The `dtwo-gateway-config` skill's `### Schema Digest` (between `<!-- BEGIN SCHEMA DIGEST -->` / `<!-- END SCHEMA DIGEST -->`) is **generated** from `dtwo/skills/dtwo-gateway-config/schema-reference.json` — do not hand-edit it. Regenerate with `node scripts/generate-schema-digest.mjs`; `--check` fails if it's stale; the `skill-harness` suite runs exactly that, and CI (`.github/workflows/skill-harness.yml`) runs both the `--check` and the suite on every push to `main` and every PR. The vendored schema is a verbatim copy of the artifact the product repo's schema generator emits; refresh it by copying, never by editing in place.
 
 ## Test locally before pushing
 
@@ -35,6 +35,8 @@ The `dtwo-gateway-config` skill's `### Schema Digest` (between `<!-- BEGIN SCHEM
 
 Clean up when done: `/plugin uninstall dtwo@dtwo` then `/plugin marketplace remove dtwo`.
 
+The offline checks (tests, biome, tsc, digest drift) also run in CI via `.github/workflows/skill-harness.yml`; the live bench does not.
+
 ## This repo is public (MIT)
 
 The scaffold and skill prose are open source. Don't reference internal-only repos, ADR numbers, build tags, or infrastructure in committed content — keep it to what a customer or external contributor should see.

README.md

@@ -87,7 +87,7 @@ node scripts/generate-schema-digest.mjs
 git diff dtwo/skills/dtwo-gateway-config/SKILL.md
 
 # Guardrail — exits 1 if SKILL.md is stale relative to the vendored schema.
-# `skill-harness` runs this as a test, so `pnpm test` covers it too.
+# `skill-harness` runs this as a test, and CI (.github/workflows/skill-harness.yml) runs it on every PR.
 node scripts/generate-schema-digest.mjs --check
 ```
 

skill-harness/README.md

@@ -4,7 +4,7 @@ Benchmark harness for the `dtwo-gateway-config` Claude skill shipped
 in this plugin repo. Two testing layers, both in this package:
 
 - **Offline rubrics** — deterministic property checks against a raw YAML
-  config. Run in CI, no LLM. Sub-second.
+  config. Run in CI (`.github/workflows/skill-harness.yml`), no LLM. Sub-second.
 - **Live bench** — runs the skill against Claude (`claude-cli` or
   `--provider=anthropic`), samples N times per fixture, scores each output
   against the rubrics, aggregates pass@k + Wilson 95% CIs, optionally
@@ -39,12 +39,18 @@ From the `skill-harness/` directory:
 ```bash
 pnpm test                       # unit tests, incl. the schema-digest drift check
 pnpm biome:check
+pnpm typecheck                  # tsc --noEmit
 pnpm bench --dry-run --tier=required
 ```
 
 The dry-run prints the fixture selection and system-prompt stats without
 issuing any LLM calls.
 
+CI (`.github/workflows/skill-harness.yml`) runs the first three of these —
+`pnpm test`, `pnpm biome:check`, `pnpm typecheck` — plus the root schema-digest
+`--check`, on every PR and every push to `main`. The bench (dry-run or live) is
+**not** run in CI; the live bench needs API credentials.
+
 ### Live bench
 
 Two providers:

skill-harness/package.json

@@ -4,6 +4,10 @@
   "private": true,
   "description": "Benchmark harness for the dtwo-gateway-config Claude skill. Offline rubrics + live bench (claude-cli / Anthropic API) with pass@k + Wilson CI aggregation and a committed regression baseline.",
   "type": "module",
+  "packageManager": "pnpm@11.18.0",
+  "engines": {
+    "node": "24"
+  },
   "exports": {
     "./*": [
       "./src/*"
@@ -13,6 +17,7 @@
     "test": "node --import tsx --test 'src/__tests__/**/*.test.ts'",
     "biome:check": "biome check --error-on-warnings",
     "biome:fix": "biome check --write .",
+    "typecheck": "tsc --noEmit",
     "bench": "tsx src/bin/bench.ts"
   },
   "dependencies": {
@@ -26,10 +31,5 @@
     "@types/node": "25.6.0",
     "tsx": "4.21.0",
     "typescript": "5.9.3"
-  },
-  "pnpm": {
-    "overrides": {
-      "esbuild": ">=0.28.1"
-    }
   }
 }

skill-harness/pnpm-workspace.yaml

@@ -0,0 +1,4 @@
+overrides:
+  esbuild: '>=0.28.1'
+allowBuilds:
+  esbuild: false