Setup-only dependency upgrades pass activation but fail safe outputs: GH_AW_JOB_CHECK_RUN_ID #70273

Problem

A Dependabot update that changes only github/gh-aw-actions/setup can leave the generated workflow incompatible with the publishing runtime. Activation accepts the old compiler version, the agent completes its work, and add_comment then fails with:

Missing or invalid safe-output execution provenance: GH_AW_JOB_CHECK_RUN_ID

We observed this with a workflow compiled by gh-aw v0.88.7: it published successfully with setup v0.91.9, then failed at Process Safe Outputs after a setup-only update to v0.91.11. The activation log explicitly reported Version check passed: v0.88.7.

Cause and existing work

#68075 and merged PR #68077 introduced the execution-provenance requirement in v0.91.10. The PR explicitly says existing workflows require recompilation. The provenance requirement itself is intentional; the gap is accepting an incompatible compiler/runtime combination until after the agent has run.

• add_comment.cjs at v0.91.11 calls appendExecutionProvenance before publishing.
• execution_provenance.cjs requires GH_AW_JOB_CHECK_RUN_ID.
• The older generated publishing step does not supply that environment variable. The updated compiler supplies it at step scope using ${{ job.check_run_id }}.
• The existing activation version check checks a remotely fetched compiler blocklist/minimum rather than this compiler/runtime requirement.

We also reproduced the exact exception locally using the provenance modules from both v0.91.11 and v0.92.4, with valid synthetic runtime metadata except for the missing variable. Supplying a positive synthetic check-run ID made the isolated function pass. This was a function-level test, not an end-to-end Actions run on v0.92.4.

Reproduction

1. Use an existing workflow compiled with v0.88.7, with an add-comment safe output and no GH_AW_JOB_CHECK_RUN_ID in its generated publishing step.
2. Upgrade only the setup action from v0.91.9 to v0.91.11, as a Dependabot dependency PR does; do not recompile.
3. Run the workflow. Activation accepts the compiler version and the agent produces an add_comment message, but publication fails with the error above.

Expected: detect the incompatible compiler/runtime combination during activation, before agent execution, and explain which compiler upgrade and recompilation are required.

Proposed implementation plan

1. Extend the activation compatibility validation in actions/setup/js/check_version_updates.cjs (or a shared helper invoked there) to enforce publishing requirements associated with the executing setup runtime. The check must also apply to already-generated older workflows. Avoid globally blocking older compiler versions that still work with their pinned older runtime.
2. Produce an actionable error and job summary identifying the mismatch and directing users to update the compiler and run gh aw compile, then commit the regenerated workflow. Make this a local runtime requirement rather than depending exclusively on a successful remote compatibility-config fetch.
3. Preserve the strict provenance validation in actions/setup/js/execution_provenance.cjs; do not substitute a fabricated identifier or publish without provenance.
4. Add regression coverage in actions/setup/js/check_version_updates.test.cjs and the existing provenance tests: old compiler/compatible runtime remains accepted, old compiler/new incompatible runtime fails before agent execution, and a regenerated workflow supplies the required check-run context. Cover unavailable remote compatibility configuration.
5. Document that setup-only dependency upgrades can require recompilation, and include this migration in the user-facing upgrade guidance.

Current remediation for affected consumers: update to a compatible compiler and regenerate the workflow. This report requests earlier detection and clearer upgrade guidance.