Semver CI should compare against last released API, not main

[kosiew]

Summary

The current breaking-change detector compares pull requests against the base branch (typically main), not against the latest released API surface. This creates semver failures for changes that only modify unreleased APIs and do not break any public, shipped contract.

This issue proposes changing the blocking semver baseline to released versions, with an optional non-blocking advisory comparison to main.

Current behavior

The semver check is wired to compare against the PR base branch:

1. Workflow runs on pull requests targeting main:
   • .github/workflows/breaking_changes_detector.yml
2. Changed crates are computed against apache/${BASE_REF}:
   • .github/workflows/breaking_changes_detector.yml
3. Semver check is invoked with apache/${BASE_REF}:
   • .github/workflows/breaking_changes_detector.yml
4. Script passes that value directly to cargo semver-checks via --baseline-rev:
   • ci/scripts/changed_crates.sh

The comment workflow also states the comparison is against base branch:

• .github/workflows/breaking_changes_detector_comment.yml

Problem

Comparing against main can flag API breaks that are only breaks relative to unreleased commits on main, not breaks relative to the latest release users depend on.

This lowers signal quality for a blocking check and can delay valid PRs that simply refine unreleased APIs.

Concrete example

In #22106, semver flagged removed members on HigherOrderSignature.

Those members were introduced after release 53.1.0 and later changed before release. So the PR is incompatible with current main state, but not incompatible with the last public release API.

Why this matters

1. Blocking semver should model user-facing released compatibility, not in-progress branch churn.
2. False-positive blocking results increase maintainer and reviewer overhead.
3. Contributors get conflicting signals: bot says break while release contract is still intact.
4. Trust in semver automation drops when failures are not release-relevant.

Proposed behavior

Use released baselines for blocking semver checks

[kumarUjjawal]

I think this is the right approach. What do you think @rluvaton ?

[gstvg]

I believe that comparing only against the last release would flag all breaking changes from every PR since the release. Perhaps we can keep a updated file like changes_since_last_release.txt which is the cargo-semver-check output of last_release..main, then in the CI we check the PR against main the last release branch, diff it with changes_since_last_release.txt, and show only added trunks? I checked cargo-semver-checks docs and didn't find anything that could help. In the mean time we could update the check comment to clarify that it may contains false positives

[rluvaton]

The reason for base branch is due to not have all breaking changes but only actual ones

And also, think about back porting PRs