Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
361 changes: 361 additions & 0 deletions .github/scripts/conformance-case-body-drift.sh

Large diffs are not rendered by default.

703 changes: 703 additions & 0 deletions .github/scripts/conformance-case-body-drift.test.sh

Large diffs are not rendered by default.

85 changes: 85 additions & 0 deletions .github/scripts/conformance-registered-case-ids.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
#!/usr/bin/env bash
#
# Print the conformance case ids THIS SDK registers, one per line.
#
# This is the repo-specific half of the case-body drift check: the id source
# depends on how this repo's harness records registrations, so it lives here and
# conformance-case-body-drift.sh stays generic.
#
# For this repo the ids come out of conformance-report.json, which the
# conformance suite writes when the JUnit run closes (ConformanceRunState) into
# the directory named by the `conformance.report.dir` system property — set in
# the root pom to the reactor root, so the file lands beside CHANGELOG / README.
# That report is the harness's own record of what registered, so it cannot
# disagree with what the suite actually did — the reason for reading it rather
# than grepping @ConformanceCase("...") annotations out of the test sources,
# which would be a second, weaker id extractor that reports what it matched and
# stays silent about what it missed.
#
# The report lists one entry per CATALOG case, so presence in it is not
# registration. `test_id` is: ConformanceExtension records it when a test
# carrying @ConformanceCase runs, and the placeholder entries the report
# synthesizes for uncovered catalog cases carry only `case_id` and `status`.
# Reading `test_id` rather than `status` matters — a registered case whose test
# failed or was skipped is still registered, and still needs its body watched,
# but its status is not "passed".
#
# Only `.cases` is read. `.uncatalogued_tests` alongside it also carries
# `test_id` entries, but those are tests with no @ConformanceCase mapping at
# all — they have no case id to contribute.
#
# Requires the suite to have run, so the report on disk belongs to this commit.
#
# Inputs (environment):
# CONFORMANCE_REPORT path to conformance-report.json
# (default: $GITHUB_WORKSPACE/conformance-report.json)
#
# Exit status:
# 0 ids printed on stdout
# 1 the report is missing, unreadable, or holds no registered case

set -euo pipefail

REPORT="${CONFORMANCE_REPORT:-${GITHUB_WORKSPACE:-.}/conformance-report.json}"

fail() {
echo "::error::$1" >&2
exit 1
}

if ! command -v jq > /dev/null 2>&1; then
fail "registered case ids: jq is not available, so the conformance report cannot be read."
fi

if [[ ! -f "$REPORT" ]]; then
fail "registered case ids: '$REPORT' does not exist. The conformance suite writes it when the JUnit run closes, so either the suite did not run or it failed before the report was written."
fi

if ! jq -e . "$REPORT" > /dev/null 2>&1; then
fail "registered case ids: '$REPORT' is not valid JSON."
fi

if ! jq -e '(.cases | type) == "array" and (.cases | length) > 0' "$REPORT" > /dev/null 2>&1; then
fail "registered case ids: '$REPORT' has no non-empty .cases array."
fi

# An entry with a case_id that is not a string, or empty, would silently drop
# out of the filter below and take a real registration with it.
if ! jq -e 'all(.cases[]; (.case_id | type) == "string" and (.case_id | length) > 0)' "$REPORT" > /dev/null 2>&1; then
fail "registered case ids: '$REPORT' holds a case entry with a missing or non-string case_id."
fi

# A test_id that is present but not a non-empty string is a shape this filter
# has no reading for: absent means "not registered", and anything else would be
# treated as a registration on the strength of a value nothing can name.
if ! jq -e 'all(.cases[]; (has("test_id") | not) or ((.test_id | type) == "string" and (.test_id | length) > 0))' "$REPORT" > /dev/null 2>&1; then
fail "registered case ids: '$REPORT' holds a case entry whose test_id is present but is not a non-empty string."
fi

ids="$(jq -r '.cases[] | select(has("test_id")) | .case_id' "$REPORT")"

if [[ -z "$ids" ]]; then
fail "registered case ids: '$REPORT' records no case with a test_id, so no @ConformanceCase mapping registered. Any check restricted to this list would be vacuously green."
fi

printf '%s\n' "$ids"
76 changes: 76 additions & 0 deletions .github/scripts/fetch-conformance-catalog.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#!/usr/bin/env bash
#
# Check out the shared conformance catalog at the revision this repo pins.
#
# Copies of this script exist outside this repo. They are not byte-identical —
# each one describes its own build — but the guards are meant to stay in step: a
# guard tightened in one copy and not the rest is how the copies drift apart.
#
# The catalog lives in github.com/AuthPlane/conformance and is updated
# independently of this repo, so cloning its default branch would let a catalog
# change turn an unrelated PR red here. The ref is pinned instead, single-sourced
# from the tracked .conformance-catalog-ref at the repo root — bump it there when
# adopting new catalog cases, together with the SDK-side coverage for them, so a
# catalog change can never break CI on its own.
#
# Uses the runner's built-in git rather than actions/checkout: equivalent trust
# for a public repo, no third-party action surface to SHA-pin.
#
# This script exists because the read/guard/fetch sequence is needed by more than
# one workflow (ci.yml, release.yml and conformance-catalog-drift.yml). Keeping it
# inline in each meant the guard could be tightened in one and not the others; the
# pin was single-sourced but the logic reading it was not.
#
# Checks out into $RUNNER_TEMP — outside $GITHUB_WORKSPACE — so the catalog stays
# out of the working tree: it must never be picked up as a module or a resource by
# the reactor, and `git add -A` in the release commit must never stage it as an
# embedded gitlink.
#
# Requires: GITHUB_WORKSPACE, RUNNER_TEMP.
#
# Optional: CONFORMANCE_CATALOG_DEST overrides the checkout directory. The drift
# workflow needs the pinned catalog and the catalog tip side by side in the same
# job to compare case bodies, so it cannot let both land on the default path.
# Every other caller leaves it unset and gets $RUNNER_TEMP/conformance.

set -euo pipefail

: "${GITHUB_WORKSPACE:?GITHUB_WORKSPACE must be set}"
: "${RUNNER_TEMP:?RUNNER_TEMP must be set}"

REF_FILE="$GITHUB_WORKSPACE/.conformance-catalog-ref"
DEST="${CONFORMANCE_CATALOG_DEST:-$RUNNER_TEMP/conformance}"
CATALOG_REPO="https://github.com/AuthPlane/conformance.git"
CATALOG_FILE="oauth-sdk-conformance-catalog.yaml"

if [[ ! -f "$REF_FILE" ]]; then
echo "::error::$REF_FILE is missing; the conformance catalog revision is unpinned"
exit 1
fi

CONFORMANCE_CATALOG_REF="$(tr -d '[:space:]' < "$REF_FILE")"

# Guard against un-pinning: the ref must be a full commit SHA, not a branch or
# tag name, either of which would silently track a moving target.
if ! grep -Eq '^[0-9a-f]{40}$' <<< "$CONFORMANCE_CATALOG_REF"; then
echo "::error::.conformance-catalog-ref must be a 40-hex commit SHA, got '$CONFORMANCE_CATALOG_REF'"
exit 1
fi

git init -q "$DEST"
if ! git -C "$DEST" fetch --depth=1 "$CATALOG_REPO" "$CONFORMANCE_CATALOG_REF"; then
echo "::error::Pinned conformance catalog ref $CONFORMANCE_CATALOG_REF is unreachable"
exit 1
fi
git -C "$DEST" checkout -q FETCH_HEAD

# The alignment assertion hard-fails when CONFORMANCE_CATALOG_PATH points at a
# missing file, but it reports that as a harness problem rather than drift. Fail
# here instead, where the cause is unambiguous: the fetch succeeded and the
# catalog still is not where every caller expects it.
if [[ ! -f "$DEST/$CATALOG_FILE" ]]; then
echo "::error::$CATALOG_FILE is not in the catalog at $CONFORMANCE_CATALOG_REF; the fetch succeeded but produced no catalog in $DEST"
exit 1
fi

echo "Conformance catalog checked out at $CONFORMANCE_CATALOG_REF in $DEST"
33 changes: 12 additions & 21 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,29 +23,20 @@ jobs:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

# Uses the runner's built-in git instead of actions/checkout: equivalent
# trust for a public repo (AuthPlane/conformance), no third-party action
# surface to SHA-pin. Keeps the catalog outside the workspace so
# Conformance catalog pinned by SHA (was: clone of the latest default
# branch). The single source of truth for the ref is the tracked
# `.conformance-catalog-ref` file at the repo root — bump it when adopting
# new catalog cases, together with the SDK-side conformance coverage, so a
# catalog change can never break CI on its own. The Checkout step above
# must precede this, which reads that file out of the workspace.
#
# The read/guard/fetch sequence lives in the script rather than inline
# here: more than one workflow needs it, and inline copies meant the
# 40-hex-SHA guard could be tightened in one and not the others. It checks
# the catalog out to $RUNNER_TEMP/conformance, outside the workspace, so
# release.yml's `git add -A` cannot stage it as an embedded gitlink.
- name: Check out shared conformance catalog (outside workspace)
# Conformance catalog pinned by SHA (was: clone of the latest default
# branch). The single source of truth for the ref is the tracked
# `.conformance-catalog-ref` file at the repo root — bump it when
# adopting new catalog cases, together with the SDK-side conformance
# coverage, so a catalog change can never break CI on its own. The
# Checkout step above must precede this read. Source:
# github.com/AuthPlane/conformance.
run: |
CONFORMANCE_CATALOG_REF="$(cat "$GITHUB_WORKSPACE/.conformance-catalog-ref")"
# Guard the pin: a non-SHA value would silently un-pin CI to whatever
# ref resolves at fetch time.
grep -Eq '^[0-9a-f]{40}$' <<<"$CONFORMANCE_CATALOG_REF" \
|| { echo "::error::.conformance-catalog-ref must be a 40-hex commit SHA"; exit 1; }
git init -q "${{ runner.temp }}/conformance"
git -C "${{ runner.temp }}/conformance" \
fetch --depth=1 https://github.com/AuthPlane/conformance.git "$CONFORMANCE_CATALOG_REF" \
|| { echo "::error::Pinned conformance catalog ref $CONFORMANCE_CATALOG_REF is unreachable"; exit 1; }
git -C "${{ runner.temp }}/conformance" checkout -q FETCH_HEAD
run: .github/scripts/fetch-conformance-catalog.sh

- name: Setup Java
uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4.8.0
Expand Down
Loading
Loading