ci(conformance): bump harness to 0.2.0-alpha.5 preview

[maxisbey]

Bumps the conformance harness pin to the 0.2.0-alpha.5 preview build so the staging branch can assert the post-#2907 error codes (-32020..-32022) once the server-stateless / header-validation work starts emitting them.

Part of #2891. Originally targeted the v2-2026-07-28 staging branch (#2917), retargeted to main after #2917 merged.

Motivation and Context

#2912 noted that the published @modelcontextprotocol/conformance@0.2.0-alpha.4 predates conformance#353 (the renumber). conformance#357 is the alpha.5 release PR that carries it; this PR pins to its pkg.pr.new preview build at commit 65fcd39 so the staging branch runs against the renumbered harness without waiting for the npm release.

What changed

• conformance.yml: CONFORMANCE_VERSION → CONFORMANCE_PKG (full npx package spec, so the same line accepts either a registry version or a tarball URL). Pinned to https://pkg.pr.new/@modelcontextprotocol/conformance@65fcd39 — the immutable commit-SHA form, not the mutable @357 PR ref. A new "Fetch and verify conformance harness" step downloads the tarball, checks it against CONFORMANCE_PKG_SHA256, and re-points CONFORMANCE_PKG at the verified local file before any npx call (no-op when CONFORMANCE_PKG is a registry spec). The comment carries a TODO to swap to @modelcontextprotocol/conformance@0.2.0-alpha.5 once conformance#357 publishes, dropping the SHA + fetch step.
• run-server.sh + both client steps: npx --yes "$CONFORMANCE_PKG".
• expected-failures.yml: removed auth/scope-step-up from client: — conformance#337 gated the SEP-2350 scope-union check to 2026-07-28+, so it now passes on the 2025 client leg. It stays in expected-failures.2026-07-28.yml (still fails there).
• Both expected-failures*.yml header comments updated to reference CONFORMANCE_PKG.

How Has This Been Tested?

All 5 conformance legs run locally against the 65fcd39 build:

Leg	Exit	Delta
server --suite active	0	none
server --suite draft	0	none
server --suite all --spec-version 2026-07-28	0	none
client --suite all	1 → 0	stale auth/scope-step-up removed
client --suite all --spec-version 2026-07-28	0	none

No new scenarios appeared (the SEP-2663 tasks scenarios from conformance#262 and the server/discover mocks from conformance#347 are not in active/draft/all at this build).

Breaking Changes

None.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Checklist

• I have read the MCP Documentation
• My code follows the repository's style guidelines
• New and existing tests pass locally
• I have added appropriate error handling
• I have added or updated documentation as needed

_{AI Disclaimer}

[claude]

I didn't find any bugs — the fetch-and-verify step, sha256 pin, and expected-failures reconciliation all look consistent — but pinning main's CI to a pkg.pr.new preview tarball (vs. waiting for the alpha.5 npm release) is a maintainer policy call, so leaving this for a human.

Extended reasoning...

Overview

This PR is CI-only: it switches the conformance harness pin in .github/workflows/conformance.yml from a registry version (CONFORMANCE_VERSION = 0.2.0-alpha.4) to a full package spec (CONFORMANCE_PKG) pointing at a pkg.pr.new preview tarball at commit 65fcd39, adds a fetch-and-verify step (sha256 check, then re-points to the local tarball) in both the server and client jobs, updates run-server.sh and the client steps to use $CONFORMANCE_PKG, and removes the now-passing auth/scope-step-up entry from the 2025 client baseline.

Security risks

The main consideration is supply-chain: CI now executes a tarball fetched from pkg.pr.new instead of the npm registry. The PR mitigates this reasonably — the URL uses the immutable commit-SHA form, the tarball is verified against a recorded sha256 before any npx invocation, the workflow keeps permissions: contents: read, and checkout uses persist-credentials: false. The fetch step is duplicated identically in both jobs and correctly no-ops for registry specs. I found no flaws in the implementation, but whether main's CI should depend on an unreleased preview build at all (rather than waiting for the 0.2.0-alpha.5 npm publish, per the in-file TODO) is a project policy decision.

Level of scrutiny

Moderate. The change doesn't touch SDK code, only the conformance CI harness, so worst case is CI noise rather than a shipped defect. Still, changes to what code gets downloaded and executed in CI deserve a human maintainer's sign-off rather than an automated approval.

Other factors

The author reports all five conformance legs passing locally against the pinned build, and the expected-failures edits are consistent with the stated behavior change in conformance#337 (scope-step-up removed only from the 2025 client baseline, retained for the 2026 leg). The header-comment updates in both expected-failures files match the new variable name. No prior reviews or unresolved comments are on the PR.