diff --git a/.github/workflows/notify-main-failure.yml b/.github/workflows/notify-main-failure.yml index 739d20d..01f9a97 100644 --- a/.github/workflows/notify-main-failure.yml +++ b/.github/workflows/notify-main-failure.yml @@ -16,15 +16,24 @@ # every merge. This is derived from the GitHub API rather than cross-run # state, so it is self-healing (no cache to go stale). # -# `freeze-scoped: true` adds a third outcome for STAGING callers. A staging -# failure is only release-blocking once the freeze window is open; mid-week it is -# an ordinary integration-branch red, and paging the channel for it is what -# teaches people to ignore the channel. So a freeze-scoped failure posts the red -# alert while the `staging-freeze` ruleset is active, and outside the freeze -# posts a muted grey notice instead (`outside-freeze: silent` drops it entirely, -# at the cost of a broken staging branch being able to sit red unnoticed until -# the next freeze opens). Prod and freeze/unfreeze callers leave this alone — a -# failure there is always worth interrupting for. +# `freeze-scoped: true` scopes a STAGING caller to the release window. The policy +# this implements: +# +# failure on main -> alert +# failure on staging, freeze OPEN -> alert (staging is the release candidate) +# failure on staging, freeze NOT open -> nothing (ordinary integration-branch red) +# +# Mid-week staging is the integration branch, and paging the channel for it is +# what teaches people to ignore the channel. `outside-freeze: notice` posts a +# muted grey message instead of nothing, for a caller that wants the trail; the +# default is silence. Either way the run is still red in the Actions tab, and +# pipeline-watchdog.yml still covers a staging branch that stops building at all. +# +# Prod and freeze/unfreeze callers leave this alone: a failure there is always +# worth interrupting for. A workflow triggered on BOTH main and staging from one +# notify job cannot hardcode this — pass an expression instead, or a main failure +# gets silenced too: +# freeze-scoped: ${{ github.ref_name == 'staging' }} # # It posts its own message rather than the deploy-notification composite, whose # copy is deploy-specific ("has failed deploying to ...") and reads wrong for @@ -79,9 +88,9 @@ on: type: boolean default: false outside-freeze: - description: "What a freeze-scoped failure does while staging is NOT frozen: 'notice' (muted, no panic styling) or 'silent' (post nothing)." + description: "What a freeze-scoped failure does while staging is NOT frozen: 'silent' (post nothing) or 'notice' (muted, no panic styling)." type: string - default: notice + default: silent runs-on: description: "Runner label for the notify job" type: string diff --git a/README.md b/README.md index 3e3c926..a09bf6f 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,13 @@ missing recovery message is the symptom to look for. A staging failure is not the same event all week. Once the freeze window is open, `staging` is the release candidate and a red pipeline blocks the release. Before it, `staging` is the integration branch and the same red is routine, so paging the channel for it is what teaches people to scroll past the channel. -`freeze-scoped: true` on a **staging** caller keeps the red `:rotating_light:` alert while the `staging-freeze` ruleset is `active`, and drops it to a muted grey `:warning:` notice outside the window: +`freeze-scoped: true` on a **staging** caller implements that policy: + +| Where it failed | Freeze window | Result | +|---|---|---| +| `main` | n/a | red alert | +| `staging` | open | red alert | +| `staging` | not open | nothing posted | ```yaml with: @@ -97,7 +103,13 @@ A staging failure is not the same event all week. Once the freeze window is open freeze-scoped: true ``` -Add `outside-freeze: silent` to post nothing at all outside the window instead. That trades the noise for a blind spot: a staging pipeline can then sit red unnoticed until the next freeze opens on a branch that no longer builds, so `notice` is the default. +Add `outside-freeze: notice` to post a muted grey `:warning:` message instead of nothing, for a caller that wants the trail. Silence is the default; the run is still red in the Actions tab either way, and `pipeline-watchdog.yml` still covers a staging branch that stops building at all. + +A workflow triggered on **both** main and staging from one notify job must not hardcode `true`, or main failures get silenced too. Pass an expression: + +```yaml + freeze-scoped: ${{ github.ref_name == 'staging' }} +``` Freeze state is read from the `staging-freeze` ruleset itself, not inferred from workflow history — a freeze that skipped itself because staging had nothing unreleased still concludes `success`, and history cannot tell that apart from a real freeze. Reading rulesets needs admin, so this reuses the same App that toggles them (`vars.RELEASE_APP_CLIENT_ID` + `secrets.RELEASE_APP_PRIVATE_KEY`); no extra `permissions:` on the caller. If the App token or the ruleset lookup fails, it escalates to the red alert rather than downgrading, so a lookup problem can never silence a real release-blocking failure.