[Copilot]

On October 19, gpt-5-mini, gpt-5.4, gpt-5.4-mini, gpt-5.5 and grok-4.5 were removed from the Copilot catalogue. Since #70389, Copilot refuses to start when a workflow-declared sub-agent model doesn't resolve. Run 39506475007's catalogue (awf-reflect.json) shows 25 dead pins across the repository's workflows. This PR fixes the 18 from the issue and 6 same-kind pins that a maintainer added to scope. It leaves one: video-analyzer's copilot/gemini-3.7-flash, which is the owner's decision.

Alias definitions, the resolver, the harnesses, AWF configuration and the pin checker are unchanged.

How replacements were chosen

• Aliases where the field and engine resolve them: Copilot and Pi sub-agents, and top-level model: on Pi.
• Concrete live IDs for fallback-models, routing candidates, routed-smoke sub-agents, and engines that don't resolve aliases.
• copilot/claude-haiku-5.5 for every engine path that uses Chat Completions, since no GPT model in this catalogue advertises it. I picked haiku over sonnet so these workflows stay within their existing max-ai-credits limits (5 and 20).
• GPT sub-agent coverage on /responses is kept in every sub-agent smoke.

Sub-agents

Workflow	Sub-agent	Was	Now
smoke-copilot-sub-agents	gpt5-whoami (was mini-whoami)	gpt-5-mini	gpt-5 alias, resolves to gpt-5.6-luna
smoke-copilot-sub-agents	luna-whoami (was mini54-whoami)	gpt-5.4-mini	gpt-5.6-luna (exact)
smoke-copilot-sdk-routed-claude	mini-whoami	gpt-5.4-mini	gpt-5.6-luna
smoke-pi-routed	mini-whoami	copilot/gpt-5.4-mini	copilot/gpt-5.3-codex
smoke-pi-sub-agents	luna-whoami (was mini-whoami)	copilot/gpt-5-mini	copilot/gpt-5.6-luna
daily-model-resolution (production)	run-analyzer	gpt-5.4-mini	mini alias, now runs on claude-haiku-5.5

Notes:

• run-analyzer moves from GPT to Claude. No small GPT model remains in the catalogue, and the built-in gpt-5-mini alias still finds no match.
• Routed smokes use exact IDs. The shared routing assertions match requests to sub-agents by a fixed model name, and their C1 check requires each sub-agent model to differ from the routing candidates. In smoke-pi-routed the only candidate is gpt-5.6-luna, which is why its sub-agent uses gpt-5.3-codex.
• smoke-pi-sub-agents deliberately pins exact IDs, so it stays exact.

Sub-agent assertions

• smoke-copilot-sub-agents: the post-step now reads each sub-agent's model.resolution record. It requires status resolved for the alias child and exact for the others, then checks the served model and endpoint against the resolved model rather than the literal alias:

  const expected = [
    ["haiku-whoami", "claude-haiku-4.5", "exact", "/chat/completions"],
    ["gpt5-whoami", "gpt-5", "resolved", "/responses"],
    ["luna-whoami", "gpt-5.6-luna", "exact", "/responses"],
  ];

• smoke-copilot-sdk-routed-claude: resolutionRoles: ["mini-whoami"], so a successful resolution of the sub-agent is now required.
• Descriptions, replies and the parent's expected-response lists are updated to match.

Top-level copilot/gpt-5.4 pins

Engine path	Now
aider, goose, opencode, deepseek-harness, pydantic-ai, smoke-aider, smoke-goose (Chat Completions, no alias resolution)	copilot/claude-haiku-5.5
crush (uses /responses for GPT models; no alias resolution)	copilot/gpt-5.6-luna
pi (resolves aliases)	copilot/gpt-5

Fallbacks and routing

Each fallback differs from the workflow's main model and from the other fallback.

Workflow	Main model	Fallbacks now
smoke-copilot, smoke-copilot-arm, smoke-pi	default / auto	gpt-5.6-luna, claude-sonnet-5
smoke-copilot-sdk	gpt-5, resolves to gpt-5.6-luna	claude-sonnet-5, gpt-5.3-codex
smoke-service-ports, smoke-work-queue	gpt-5.3-codex	gpt-5.6-luna, claude-haiku-5.5

smoke-copilot-routed: dropped gpt-5.4-mini, so allowed-models is [gpt-5.6-luna, claude-haiku-4.5] (GPT and Claude). Both allowedModels assertion arrays match it.

Regression test (repository-pins-reflection-regression)

• Fixture: actions/setup/js/test_data/awf_reflect/run-39506475007-awf-reflect.json, a byte-for-byte copy of the run's awf-reflect.json. SHA-256 is 3d9fa7af7e07e93100bb614e45eb85abafbd16a3bbb2cb62a687a04cb5fff9b3; it is checked for credentials and its source is recorded in that folder's README.
• Scans the repository's real workflows and asserts on the report's classifications, not the CLI exit code:
  • each of the 24 fixed workflow/field pairs is exact or alias;
  • the 21 affected workflows have no unresolvable pins;
  • the repository's unresolvable pins are exactly {video-analyzer.md, model, copilot/gemini-3.7-flash}.
• Other tests updated because they read the changed workflow text: smoke_copilot_sub_agents.test.cjs, pi_subagent_extension.test.cjs, goose_runtime_test.go, prompts_test.go.

Verification still needed (maintainer dispatch on this branch)

None of these smoke runs has happened on the branch yet.

Workflow	Expected
smoke-copilot-sub-agents (inline_strict or delegated_sequential variant)	Every sub-agent resolution is exact or resolved; served-model and endpoint assertions pass
smoke-copilot-sdk-routed-claude	Same, with the GPT child on /responses
smoke-pi-routed	Same, including the GPT child's /responses request
smoke-pi-sub-agents	All three delegations succeed
engine-conformance-aider, smoke-aider	No model_unavailable and no endpoint 400 on Chat Completions
smoke-copilot-routed	Routing picks from the updated pool; both assertion arrays pass

Known failures on main

• Go lint: three modernize (embedlit) findings in claude_engine.go, codex_engine.go and comment_memory.go. This PR doesn't touch those files.
• smoke-copilot: a separate shell/URL-denial failure, out of scope here.

• Fixes Fix stale Copilot workflow pins exposed by fail-closed sub-agent resolution #70649

[github-actions]

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · sonnet50 · 14.2 AIC · ⌖ 15.1 AIC · ◇ 6.58 AIC · ⊞ 12.6K · ◷

🧪 Test quality analysis by Test Quality Sentinel

[github-actions]

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · gpt56 · 9.84 AIC · ⌖ 6.15 AIC · ⊞ 14.7K · ◷

Generated by Ponytail Reviewer for #70652

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

[github-actions]

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · opus55 · 11.1 AIC · ⌖ 36.4 AIC · ◇ 15.8 AIC · ⊞ 1.9K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

[github-actions]

Verdict: no blocking findings

Reviewed the changed workflow pins, sub-agent evidence assertions, and the new repository pin regression coverage. I found no actionable correctness, security, performance, or maintainability issue in the changed lines. Verification was static only; smoke runs are not available in this review environment.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · 2.11 AIC · ⌖ 0.493 AIC · ⊞ 1.6K
_{Comment /review to run again}

[github-actions]

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd — no actionable issues found; approving.

📋 Key Themes & Highlights

Key Themes

• Root cause correctly addressed: pins are updated to live catalogue IDs/aliases rather than papering over the fail-closed resolution error, matching /diagnosing-bugs guidance to fix the cause (#70389's stricter validation), not the symptom.
• Regression test is exemplary (model_pin_check.test.cjs): the new repository-pins-reflection-regression describe block pins the fixture by SHA-256, asserts on the report's classifications (not CLI exit code), and enumerates both the 21 fixed workflows and the one known remaining failure (video-analyzer.md). This is /tdd-aligned — it prevents silent regression to dead pins and reads as a spec of the expected catalogue state.
• Sub-agent assertion upgrade in smoke-copilot-sub-agents.md's post-step now validates the actual model.resolution event (configured_model, status, resolved_model) instead of only checking the served model string — this closes a gap where an alias could silently resolve to the wrong model while still looking exact by regex alone.
• I traced the mini alias chain (mini → haiku → copilot/haiku) against pkg/workflow/data/model_aliases.json and confirmed it resolves to claude-haiku-5.5 as claimed for daily-model-resolution.md's run-analyzer, and verified the model.resolution status vocabulary (resolved/exact) is consistent with resolve_model_alias.cjs.

Positive Highlights

• ✅ Clear, well-documented mapping table in the PR description of every pin change and why
• ✅ Fixture provenance is tracked in test_data/awf_reflect/README.md with run URL, commit, and shape description
• ✅ Honest scoping: explicitly leaves video-analyzer's dead pin as an owner decision rather than silently fixing it
• ✅ New model.resolution-based sub-agent tests (smoke_copilot_sub_agents.test.cjs) cover failure modes: missing resolution record, failed status, and resolved-model drift

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 79 AIC · ⌖ 14.9 AIC · ⊞ 10.2K
_{Comment /matt to run again}

[Copilot]

🔵 Needs a closer look

Cross-engine replacements and the production model change lack branch runtime verification, and confirmed review issues remain unresolved.

2 open issues

• Correlate GPT child requests with individual invocations · New
• Decode import.meta.url before constructing filesystem paths · New

What changed in this PR

Updates gh-aw workflows to replace retired Copilot model pins while preserving fail-closed sub-agent resolution.

Changes:

• Replace stale main, sub-agent, fallback, and routing models with aliases or compatible live IDs.
• Synchronize smoke prompts, assertions, and generated workflows.
• Add catalogue-backed regression coverage and update affected tests.

File	Description
pkg/​workflow/​prompts_test.go	Expect the production sub-agent’s mini alias.
pkg/​workflow/​goose_runtime_test.go	Update Goose’s expected model.
actions/​setup/​js/​test_data/​awf_reflect/​README.md	Record catalogue fixture provenance.
actions/​setup/​js/​smoke_copilot_sub_agents.test.cjs	Test sub-agent resolution evidence.
actions/​setup/​js/​pi_subagent_extension.test.cjs	Update expected GPT identity.
actions/​setup/​js/​model_pin_check.test.cjs	Add repository pin regression coverage.
.github/​workflows/​smoke-work-queue.md	Replace retired fallback models.
.github/​workflows/​smoke-service-ports.md	Replace retired fallback models.
.github/​workflows/​smoke-pi.md	Replace the retired GPT fallback.
.github/​workflows/​smoke-pi-sub-agents.md	Update GPT child model and replies.
.github/​workflows/​smoke-pi-sub-agents.lock.yml	Synchronize generated sub-agent metadata.
.github/​workflows/​smoke-pi-routed.md	Update GPT child and routing assertions.
.github/​workflows/​smoke-pi-routed.lock.yml	Synchronize generated assertions and metadata.
.github/​workflows/​smoke-goose.md	Select Chat Completions-compatible Haiku.
.github/​workflows/​smoke-copilot.md	Replace the retired GPT fallback.
.github/​workflows/​smoke-copilot-sub-agents.md	Update children and verify resolved models.
.github/​workflows/​smoke-copilot-sub-agents.lock.yml	Synchronize generated resolution assertions.
.github/​workflows/​smoke-copilot-sdk.md	Replace the retired SDK fallback.
.github/​workflows/​smoke-copilot-sdk-routed-claude.md	Update GPT child and require resolution evidence.
.github/​workflows/​smoke-copilot-sdk-routed-claude.lock.yml	Synchronize generated child assertions.
.github/​workflows/​smoke-copilot-routed.md	Remove the retired routing candidate.
.github/​workflows/​smoke-copilot-routed.lock.yml	Synchronize routing configuration and assertions.
.github/​workflows/​smoke-copilot-arm.md	Replace the retired GPT fallback.
.github/​workflows/​smoke-aider.md	Select Chat Completions-compatible Haiku.
.github/​workflows/​engine-conformance-pydantic-ai.md	Switch conformance model to Haiku.
.github/​workflows/​engine-conformance-pydantic-ai.lock.yml	Synchronize generated model and costs.
.github/​workflows/​engine-conformance-pi.md	Use the GPT-5 alias.
.github/​workflows/​engine-conformance-opencode.md	Switch conformance model to Haiku.
.github/​workflows/​engine-conformance-opencode.lock.yml	Synchronize generated model and costs.
.github/​workflows/​engine-conformance-goose.md	Switch conformance model to Haiku.
.github/​workflows/​engine-conformance-goose.lock.yml	Synchronize generated model and costs.
.github/​workflows/​engine-conformance-deepseek-harness.md	Switch conformance model to Haiku.
.github/​workflows/​engine-conformance-deepseek-harness.lock.yml	Synchronize generated model and costs.
.github/​workflows/​engine-conformance-crush.md	Select a live GPT Responses model.
.github/​workflows/​engine-conformance-crush.lock.yml	Synchronize generated model and costs.
.github/​workflows/​engine-conformance-aider.md	Switch conformance model to Haiku.
.github/​workflows/​engine-conformance-aider.lock.yml	Synchronize generated model and costs.
.github/​workflows/​daily-model-resolution.md	Move run-analyzer to the mini alias.
.github/​workflows/​daily-model-resolution.lock.yml	Synchronize production sub-agent metadata.

_{Review effort: Max}

[SivaKesava1]

End-to-end results at 547a7345a8, and two review items to fix

CI: all 80 checks pass. Locally: the 55 JS tests in the three changed suites (including repository-pins-reflection-regression) and the Go goose and prompts tests pass. A full gh aw compile of every workflow with a build of this head gives no lock-file drift.

End to end (dispatched on this branch)

Run	Result
smoke-copilot-sdk-routed-claude 39516056325	✅ mini-whoami → gpt-5.6-luna, and S1–S4 pass (gpt-5.6-luna /responses 200 under a routed claude-haiku-4.5 main model). R1–R4, M1, C1 and U1 pass.
smoke-pi-routed 39516064879	✅ mini-whoami → gpt-5.3-codex /responses 200; haiku-whoami → claude-haiku-4.5 /v1/messages 200. R1–R4 and C1 pass.
smoke-pi-sub-agents 39516073462	✅ success
smoke-aider 39516082508	✅ 5 requests, all claude-haiku-5.5 /chat/completions 200. No model_unavailable and no endpoint 400.
smoke-copilot-routed 39516101702	✅ selected gpt-5.6-luna from [gpt-5.6-luna, claude-haiku-4.5]. R1–R4 and D1–D3 pass.
smoke-copilot-sub-agents 39516038378 (single_agent_control)	✅ success. All three sub-agents resolve before start: gpt-5 → gpt-5.6-luna (resolved), claude-haiku-4.5 and gpt-5.6-luna (exact).
smoke-copilot-sub-agents 39516047181 (inline_strict)	❌ haiku-whoami must actually run exactly once. Not caused by this PR, see below.
engine-conformance-aider (dispatched as 39516091848)	Not testable by dispatch: it's a work-queue worker and stops at activation with work_queue_assignment_required. smoke-aider covers the same Chat Completions path.

The inline_strict failure is the main agent not delegating, not a model problem.

• All four model.resolution records are exact or resolved, as expected.
• The Copilot session shows the main model (gpt-5.3-codex) making no task call at all: it went straight to safeoutputs-create_issue. There are 0 subagent.started events.
• This smoke failed on main before the catalogue change too. 39179665680 (19 Oct) failed with the same assertion. 38985134241 (17 Oct) failed the other way: "Control variant must not invoke sub-agents".
• The delegation reliability of this smoke is a separate issue.

Changes reviewed

• The PR touches only the 21 workflows, their lock files, the pin-check test with its saved-reflection fixture, and four tests that read workflow text.
• Production daily-model-resolution's run-analyzer moves to the mini alias, which is claude-haiku-5.5 today (no small GPT model remains).
• Chat Completions engines move to copilot/claude-haiku-5.5, to stay within their max-ai-credits.
• Fallbacks differ from each workflow's main model and from each other.
• The only remaining dead pin is video-analyzer's copilot/gemini-3.7-flash, which is an owner decision.

Ready to merge.