INTEROP-9265: Add ACM operator upgrade step for OPP product upgrades - #82641
INTEROP-9265: Add ACM operator upgrade step for OPP product upgrades#82641amp-rh wants to merge 2 commits into
Conversation
Add step registry entry at interop/opp/product-upgrade/acm/ that upgrades ACM via OLM subscription channel change and validates the operator reaches Succeeded phase. Includes MCE co-upgrade validation and hub health checks (MCH phase, policy propagator, managed clusters). This step gates downstream product upgrades (ACS, ODF, Quay) in the OPP coordinated product upgrade workflow (INTEROP-8941).
|
@amp-rh: This pull request references INTEROP-9265 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci-robot
added
the
jira/valid-reference
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughAdds a CI step-registry entry that upgrades the ACM operator through an OLM subscription channel change, approves InstallPlans, waits for completion, validates MCE and hub health, and collects diagnostics and summary artifacts. ChangesACM operator upgrade
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CI
participant OLM
participant ACM
participant Cluster
CI->>OLM: Resolve and patch ACM subscription channel
OLM-->>CI: Create InstallPlan
CI->>OLM: Approve manual InstallPlan
OLM->>ACM: Install upgraded CSV
CI->>ACM: Wait for CSV Succeeded
CI->>Cluster: Validate MCE and hub health
Cluster-->>CI: Return health status
Suggested reviewers: 🚥 Pre-merge checks | ✅ 14 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (14 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: amp-rh The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
openshift-merge-bot
Bot
added
the
rehearsals-ack
Code Review: ACM Operator Upgrade StepOverall this is a well-structured step with clean function decomposition, proper artifact collection, defensive error handling in diagnostics, and thorough post-upgrade validation (MCE + hub health). The ref YAML documentation with exit codes is excellent. A few issues and suggestions below. Bug:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh (1)
79-97: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winEmpty
spec.sourceNamespacedegrades into a misleading error.If the Subscription omits
sourceNamespace, thepackagemanifestquery runs against the current context namespace and returns nothing, surfacing as "No channels found" rather than the real cause. Default it toopenshift-marketplaceand validatepackage_nametoo.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh` around lines 79 - 97, Update the subscription lookup flow to default an empty catalog_namespace from spec.sourceNamespace to openshift-marketplace before querying packagemanifest. Also validate package_name immediately after retrieving it and return a clear error when it is empty, before executing the channels lookup; preserve the existing no-channels handling for valid package manifests.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh`:
- Around line 303-312: Update the same-channel logic in the target_channel check
to verify that the referenced InstallPlan represents a pending upgrade, rather
than treating any installPlanRef.name as pending. Inspect the referenced
InstallPlan’s spec.clusterServiceVersionNames and status.phase, or compare the
subscription’s currentCSV and installedCSV, and exit 0 only when no upgrade is
actually available; preserve the existing upgrade flow when one is pending.
- Around line 36-40: Update get_current_csv and the corresponding subscription
lookups around the callers near lines 290-297 to explicitly handle oc failures.
Ensure command substitution failures are caught without triggering set -e, then
return an empty result so the existing “No ACM subscription found” handling
executes and exits with status 3.
- Around line 180-193: Update parse_timeout to reject inputs that match none of
its supported formats instead of echoing 0. After the existing parsing branches,
return a nonzero status and emit a clear error for invalid values such as
“1h30m” or “45min”; preserve the current seconds conversion for valid minute,
second, hour, and numeric inputs.
- Around line 143-178: Update wait_for_csv_succeeded to ignore the pre-upgrade
CSV by requiring get_current_csv to differ from the previously installed
current_csv before accepting a Succeeded phase. Preserve the existing timeout,
failure handling, and polling behavior, and only return success after the
replacement CSV reports Succeeded.
- Around line 323-351: Update the InstallPlan lookup in the retry loop to read
`.status.installPlanRef.name`, matching the earlier subscription lookup. Scope
the fallback `oc get installplan` query to InstallPlans whose owner reference
name matches `${ACM_SUBSCRIPTION_NAME}` before selecting the newest result, so
the approval logic only targets the ACM Subscription’s InstallPlan.
---
Nitpick comments:
In
`@ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh`:
- Around line 79-97: Update the subscription lookup flow to default an empty
catalog_namespace from spec.sourceNamespace to openshift-marketplace before
querying packagemanifest. Also validate package_name immediately after
retrieving it and return a clear error when it is empty, before executing the
channels lookup; preserve the existing no-channels handling for valid package
manifests.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: e4f7e528-1fa1-4acd-9866-6113eb749e67
📒 Files selected for processing (5)
ci-operator/step-registry/interop/opp/product-upgrade/OWNERSci-operator/step-registry/interop/opp/product-upgrade/acm/OWNERSci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.shci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-ref.metadata.jsonci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-ref.yaml
| wait_for_csv_succeeded() { | ||
| local timeout_seconds | ||
| timeout_seconds="$(parse_timeout "${ACM_UPGRADE_TIMEOUT}")" | ||
| local start_time elapsed new_csv phase | ||
| start_time="$(date +%s)" | ||
|
|
||
| while true; do | ||
| elapsed="$(( $(date +%s) - start_time ))" | ||
| if (( elapsed > timeout_seconds )); then | ||
| echo >&2 "ERROR: Timeout (${ACM_UPGRADE_TIMEOUT}) waiting for CSV upgrade" | ||
| return 2 | ||
| fi | ||
|
|
||
| new_csv="$(get_current_csv)" | ||
| if [[ -z "${new_csv}" ]]; then | ||
| sleep 10 | ||
| continue | ||
| fi | ||
|
|
||
| phase="$(get_csv_phase "${new_csv}")" | ||
| echo " CSV: ${new_csv} Phase: ${phase} (${elapsed}s elapsed)" | ||
|
|
||
| case "${phase}" in | ||
| Succeeded) | ||
| return 0 | ||
| ;; | ||
| Failed) | ||
| echo >&2 "ERROR: CSV ${new_csv} entered Failed phase" | ||
| return 1 | ||
| ;; | ||
| *) | ||
| sleep 15 | ||
| ;; | ||
| esac | ||
| done | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
wait_for_csv_succeeded can return immediately on the pre-upgrade CSV.
Right after the channel patch, status.currentCSV still holds ${current_csv}, which is already Succeeded, so the loop can exit on the first iteration and the step reports a successful upgrade that never happened. Gate on the CSV name changing (or on spec.version differing) before accepting Succeeded.
🐛 Suggested fix
wait_for_csv_succeeded() {
+ local previous_csv="$1"
local timeout_seconds
@@
new_csv="$(get_current_csv)"
- if [[ -z "${new_csv}" ]]; then
+ if [[ -z "${new_csv}" || "${new_csv}" == "${previous_csv}" ]]; then
sleep 10
continue
fi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh`
around lines 143 - 178, Update wait_for_csv_succeeded to ignore the pre-upgrade
CSV by requiring get_current_csv to differ from the previously installed
current_csv before accepting a Succeeded phase. Preserve the existing timeout,
failure handling, and polling behavior, and only return success after the
replacement CSV reports Succeeded.
| parse_timeout() { | ||
| local input="$1" | ||
| local minutes=0 seconds=0 | ||
| if [[ "${input}" =~ ^([0-9]+)m$ ]]; then | ||
| minutes="${BASH_REMATCH[1]}" | ||
| elif [[ "${input}" =~ ^([0-9]+)s$ ]]; then | ||
| seconds="${BASH_REMATCH[1]}" | ||
| elif [[ "${input}" =~ ^([0-9]+)h$ ]]; then | ||
| minutes="$(( BASH_REMATCH[1] * 60 ))" | ||
| elif [[ "${input}" =~ ^([0-9]+)$ ]]; then | ||
| minutes="${input}" | ||
| fi | ||
| echo "$(( minutes * 60 + seconds ))" | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
parse_timeout silently yields 0 on unparsable input.
An unexpected ACM_UPGRADE_TIMEOUT (e.g. 1h30m, 45min) falls through all branches and returns 0, causing an immediate timeout on the first loop iteration. Fail loudly instead.
♻️ Suggested change
elif [[ "${input}" =~ ^([0-9]+)$ ]]; then
minutes="${input}"
+ else
+ echo >&2 "ERROR: Unsupported timeout format: ${input}"
+ return 3
fi📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| parse_timeout() { | |
| local input="$1" | |
| local minutes=0 seconds=0 | |
| if [[ "${input}" =~ ^([0-9]+)m$ ]]; then | |
| minutes="${BASH_REMATCH[1]}" | |
| elif [[ "${input}" =~ ^([0-9]+)s$ ]]; then | |
| seconds="${BASH_REMATCH[1]}" | |
| elif [[ "${input}" =~ ^([0-9]+)h$ ]]; then | |
| minutes="$(( BASH_REMATCH[1] * 60 ))" | |
| elif [[ "${input}" =~ ^([0-9]+)$ ]]; then | |
| minutes="${input}" | |
| fi | |
| echo "$(( minutes * 60 + seconds ))" | |
| } | |
| parse_timeout() { | |
| local input="$1" | |
| local minutes=0 seconds=0 | |
| if [[ "${input}" =~ ^([0-9]+)m$ ]]; then | |
| minutes="${BASH_REMATCH[1]}" | |
| elif [[ "${input}" =~ ^([0-9]+)s$ ]]; then | |
| seconds="${BASH_REMATCH[1]}" | |
| elif [[ "${input}" =~ ^([0-9]+)h$ ]]; then | |
| minutes="$(( BASH_REMATCH[1] * 60 ))" | |
| elif [[ "${input}" =~ ^([0-9]+)$ ]]; then | |
| minutes="${input}" | |
| else | |
| echo >&2 "ERROR: Unsupported timeout format: ${input}" | |
| return 3 | |
| fi | |
| echo "$(( minutes * 60 + seconds ))" | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh`
around lines 180 - 193, Update parse_timeout to reject inputs that match none of
its supported formats instead of echoing 0. After the existing parsing branches,
return a nonzero status and emit a clear error for invalid values such as
“1h30m” or “45min”; preserve the current seconds conversion for valid minute,
second, hour, and numeric inputs.
| if [[ "${target_channel}" == "${current_channel}" ]]; then | ||
| echo "Already on target channel ${target_channel}; checking if upgrade is available..." | ||
| install_plan="$(oc get subscription "${ACM_SUBSCRIPTION_NAME}" \ | ||
| -n "${ACM_SUBSCRIPTION_NAMESPACE}" \ | ||
| -o jsonpath='{.status.installPlanRef.name}' 2>/dev/null || true)" | ||
| if [[ -z "${install_plan}" ]]; then | ||
| echo "No pending upgrade on current channel; nothing to do" | ||
| exit 0 | ||
| fi | ||
| fi |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
"Pending upgrade" detection is unreliable.
status.installPlanRef.name remains populated for the already-applied InstallPlan, so this branch almost never takes the exit 0 path; instead it falls through to a no-op channel patch and then to wait_for_csv_succeeded, which reports success without upgrading. Check the referenced InstallPlan's spec.clusterServiceVersionNames / status.phase (or compare status.currentCSV with status.installedCSV) instead of mere presence.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh`
around lines 303 - 312, Update the same-channel logic in the target_channel
check to verify that the referenced InstallPlan represents a pending upgrade,
rather than treating any installPlanRef.name as pending. Inspect the referenced
InstallPlan’s spec.clusterServiceVersionNames and status.phase, or compare the
subscription’s currentCSV and installedCSV, and exit 0 only when no upgrade is
actually available; preserve the existing upgrade flow when one is pending.
| install_plan="" | ||
| for _ in $(seq 1 12); do | ||
| install_plan="$(oc get subscription "${ACM_SUBSCRIPTION_NAME}" \ | ||
| -n "${ACM_SUBSCRIPTION_NAMESPACE}" \ | ||
| -o jsonpath='{.status.installplan.name}' 2>/dev/null || true)" | ||
| if [[ -z "${install_plan}" ]]; then | ||
| install_plan="$(oc get installplan -n "${ACM_SUBSCRIPTION_NAMESPACE}" \ | ||
| --sort-by=.metadata.creationTimestamp \ | ||
| -o jsonpath='{.items[-1:].metadata.name}' 2>/dev/null || true)" | ||
| fi | ||
| if [[ -n "${install_plan}" ]]; then | ||
| break | ||
| fi | ||
| sleep 10 | ||
| done | ||
|
|
||
| if [[ -n "${install_plan}" ]]; then | ||
| echo "InstallPlan: ${install_plan}" | ||
| local_approval="$(oc get installplan "${install_plan}" \ | ||
| -n "${ACM_SUBSCRIPTION_NAMESPACE}" \ | ||
| -o jsonpath='{.spec.approval}' 2>/dev/null || true)" | ||
| if [[ "${local_approval}" == "Manual" ]]; then | ||
| echo "Approving manual InstallPlan..." | ||
| oc patch installplan "${install_plan}" \ | ||
| -n "${ACM_SUBSCRIPTION_NAMESPACE}" \ | ||
| --type merge \ | ||
| -p '{"spec":{"approved":true}}' | ||
| fi | ||
| fi |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# How other steps resolve a Subscription's InstallPlan
rg -n --glob 'ci-operator/step-registry/**/*.sh' -e 'installPlanRef' -e 'status\.installplan' -C2 | head -60Repository: openshift/release
Length of output: 155
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file="ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh"
printf '--- file exists and line count ---\n'
wc -l "$file" || true
printf '--- relevant sections ---\n'
sed -n '280,360p' "$file" | nl -ba -v280
printf '--- all installplan references in file ---\n'
rg -n -i 'installplan|installPlan|subscription|ACM_SUBSCRIPTION' "$file" -C2
printf '--- repo-wide manual approvals of InstallPlans ---\n'
rg -n --glob '*.sh' 'oc patch installplan|approved.*true|approval|installPlanRef|installplan\.name|installPlan\.name' -C3 | head -200Repository: openshift/release
Length of output: 246
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Check whether git is available ==="
git --version || true
echo "=== Locate target file(s) ==="
git ls-files 'ci-operator/step-registry/**/*-commands.sh' | rg 'interop-opp-product-upgrade-acm-commands\.sh|acm.*product-upgrade|product-upgrade-acm' || true
echo "=== Run targeted search if target file path is known ==="
if [ -f ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh ]; then
sed -n '280,360p' ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh | nl -ba -v280
fiRepository: openshift/release
Length of output: 443
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file="ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh"
tmp="$(mktemp)"
line=1
while IFS= read -r lineno content; do
case "$content" in
*installplan*|*installPlan*|*installPlanRef*|*subscription*|*ACM_SUBSCRIPTION*)
start=$((lineno-2)); [ "$start" -lt 1 ] && start=1
end=$((lineno+3)); echo "--- lines $start-$end around installplan/subscription refs ---"
sed -n "${start},${end}p" "$file" - | awk -v S="$start" '{printf "%d\t%s\n", S+$0-1, $0}'
;;
esac
done < <(cat -n "$file" | awk '{gsub(/^\s+[0-9]+\t/, "", $0); print NR"\t"$0}')
printf '\n--- repo-wide InstallPlan approval/owner refs ---\n'
rg -n --glob '*.sh' 'oc patch installplan|approved.*true|approval|installPlanRef|status\.installplan\.name|metadata\.ownerReferences|ownerReferences' -C2 | head -240
printf '\n--- behavioral probe for the owner-agnostic fallback and duplicate field expression ---\n'
python3 - <<'PY'
installplan_name_expr = ".installplan.name"
installPlanRef_expr = ".installPlanRef.name"
fallback_all_namespaces = "Newest InstallPlan in namespace regardless of owner"
print("Field expression:", installplan_name_expr)
print("Consistent expression elsewhere:", installPlanRef_expr)
print("Falls back to:", fallback_all_namespaces)
PYRepository: openshift/release
Length of output: 208
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file="ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh"
echo "--- targeted lines 290-360 ---"
awk '{printf "%d\t%s\n", NR, $0}' "$file" | sed -n '290,360p'
echo
echo "--- all matches in file ---"
grep -InEi 'installplan|installPlan|installPlanRef|subscription|ACM_SUBSCRIPTION' "$file" || true
echo
echo "--- repo-wide InstallPlan/owner references in SH files ---"
rg -n --glob '*.sh' 'oc patch installplan|approved.*true|approval|installPlanRef|status\.installplan\.name|metadata\.ownerReferences|ownerReferences' -C2 | head -240 || trueRepository: openshift/release
Length of output: 5727
Use installPlanRef and scope the InstallPlan fallback to the ACM Subscription.
Lines 323-331 still read .status.installplan.name, while the same value is read as .status.installPlanRef.name at line 305. The fallback also selects the newest InstallPlan in the namespace; in namespaces with multiple operators, patch that InstallPlan can approve an unrelated manual InstallPlan. Filter the fallback by metadata.ownerReferences[?(@.name=="${ACM_SUBSCRIPTION_NAME}")] before taking the latest one.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.sh`
around lines 323 - 351, Update the InstallPlan lookup in the retry loop to read
`.status.installPlanRef.name`, matching the earlier subscription lookup. Scope
the fallback `oc get installplan` query to InstallPlans whose owner reference
name matches `${ACM_SUBSCRIPTION_NAME}` before selecting the newest result, so
the approval logic only targets the ACM Subscription’s InstallPlan.
…t default - Add || true to all oc get helpers to prevent inherit_errexit from killing callers before empty-string checks can run - Use consistent .status.installPlanRef.name (not deprecated .installplan) - Add fallback default (30m) for unrecognized timeout formats in parse_timeout - Write acm-upgraded-version and acm-upgraded-channel to SHARED_DIR for downstream step consumption
|
[REHEARSALNOTIFIER] Note: If this PR includes changes to step registry files ( Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
@amp-rh: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
3 participants
Summary
Add step registry entry at
interop/opp/product-upgrade/acm/that upgrades ACM via OLM subscription channel change and validates the operator reaches Succeeded phase.What it does:
This step gates downstream product upgrades (ACS, ODF, Quay) in the OPP coordinated product upgrade workflow (INTEROP-8941).
Jira
Files
ci-operator/step-registry/interop/opp/product-upgrade/OWNERSci-operator/step-registry/interop/opp/product-upgrade/acm/OWNERSci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-commands.shci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-ref.yamlci-operator/step-registry/interop/opp/product-upgrade/acm/interop-opp-product-upgrade-acm-ref.metadata.jsonTest plan
ci/prow/ci-operator-registrypasses (step ref validation)ci/prow/step-registry-shellcheckpassesci/prow/ownerspasses (OWNERS file validation)Summary by CodeRabbit
Adds a new ACM operator upgrade step to the CI step-registry interop
interop/opp/product-upgradeso OPP-coordinated product upgrades can reliably advance ACM/OLM in a controlled, validated way.interop-opp-product-upgrade-acmstep definition (command, 45m execution timeout, env vars, resource requests, and documented exit codes).interop-opp-product-upgrade-acm-commands.shupgrade script that:ACM_TARGET_CHANNEL, or auto-selects the next higher semver-based channel from the catalog.spec.approval=Manual.Succeeded(with support form|s|h|minutestimeout formats and a default of 30m).acm-upgrade-summary.txt; also optionally records upgraded version/channel intoSHARED_DIRfor downstream gating of ACS/ODF/Quay upgrades.