Add CUDN pod churn and CUDN churn periodic tests for 5.0#80485
Add CUDN pod churn and CUDN churn periodic tests for 5.0#80485mohit-sheth wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughThis PR adds two new CUDN churn performance test jobs to the OpenShift perfscale CI suite, updating the benchmarking script to use a forked e2e-benchmarking repository pinned to a specific branch, and configuring the new test workloads to run pod and CUDN churn scenarios on 24-node AWS clusters with 250-iteration overrides. ChangesCUDN Churn Testing Jobs
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes Suggested labels
🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: mohit-sheth The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/pj-rehearse periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-cudn-pod-churn-250-24nodes |
|
@mohit-sheth: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@mohit-sheth: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/openshift-qe/cudn-density/openshift-qe-cudn-density-commands.sh`:
- Around line 38-39: Hardcoded temporary fork settings (REPO_URL and TAG_OPTION)
lack a revert tracker; update the file to add a TODO comment adjacent to the
REPO_URL and TAG_OPTION declarations that includes the upstream PR/issue URL (or
CI ticket) and a short "revert when merged" note, or reference a scheduled
task/milestone to perform the revert—mention the exact upstream PR number/link
and the intended revert action so future maintainers can find and remove the
fork override.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 532e70f5-63f4-4e29-97ac-bb105a1eb576
⛔ Files ignored due to path filters (1)
ci-operator/jobs/openshift-eng/ocp-qe-perfscale-ci/openshift-eng-ocp-qe-perfscale-ci-main-periodics.yamlis excluded by!ci-operator/jobs/**
📒 Files selected for processing (2)
ci-operator/config/openshift-eng/ocp-qe-perfscale-ci/openshift-eng-ocp-qe-perfscale-ci-main__aws-5.0-nightly-x86.yamlci-operator/step-registry/openshift-qe/cudn-density/openshift-qe-cudn-density-commands.sh
| REPO_URL="https://github.com/mohit-sheth/e2e-benchmarking"; | ||
| TAG_OPTION="--branch use-fork-kube-burner-ocp"; |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚖️ Poor tradeoff
Temporary fork URL lacks tracking mechanism for revert.
Lines 38-39 hardcode the fork URL and branch as explicitly temporary (per PR description: "Revert e2e-benchmarking fork URL after upstream kube-burner-ocp PR merges"). Without a TODO comment, issue reference, or other tracking, this revert could be forgotten, leaving the repository diverged from upstream.
Recommendation: Add a TODO comment with the upstream PR reference or GitHub issue link to track the revert, or ensure the revert is tied to a scheduled task/milestone.
📝 Suggested fix to add tracking
+# TODO: Revert to cloud-bulldozer after upstream kube-burner-ocp PR merges
+# See: https://github.com/cloud-bulldozer/kube-burner-ocp/pull/458
REPO_URL="https://github.com/mohit-sheth/e2e-benchmarking";
TAG_OPTION="--branch use-fork-kube-burner-ocp";📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| REPO_URL="https://github.com/mohit-sheth/e2e-benchmarking"; | |
| TAG_OPTION="--branch use-fork-kube-burner-ocp"; | |
| # TODO: Revert to cloud-bulldozer after upstream kube-burner-ocp PR merges | |
| # See: https://github.com/cloud-bulldozer/kube-burner-ocp/pull/458 | |
| REPO_URL="https://github.com/mohit-sheth/e2e-benchmarking"; | |
| TAG_OPTION="--branch use-fork-kube-burner-ocp"; |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/step-registry/openshift-qe/cudn-density/openshift-qe-cudn-density-commands.sh`
around lines 38 - 39, Hardcoded temporary fork settings (REPO_URL and
TAG_OPTION) lack a revert tracker; update the file to add a TODO comment
adjacent to the REPO_URL and TAG_OPTION declarations that includes the upstream
PR/issue URL (or CI ticket) and a short "revert when merged" note, or reference
a scheduled task/milestone to perform the revert—mention the exact upstream PR
number/link and the intended revert action so future maintainers can find and
remove the fork override.
|
/pj-rehearse periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-cudn-churn-250-24nodes |
|
@mohit-sheth: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-cudn-pod-churn-250-24nodes pj-rehearse periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-cudn-churn-250-24nodes |
|
@mohit-sheth: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@mohit-sheth: job(s): pj-rehearse either don't exist or were not found to be affected, and cannot be rehearsed |
|
/pj-rehearse periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-cudn-pod-churn-250-24nodes pj-rehearse periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-cudn-churn-250-24nodes |
|
@mohit-sheth: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@mohit-sheth, Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
@mohit-sheth, |
Add two new 24-node periodic tests to validate kube-burner-ocp CUDN churn support (kube-burner/kube-burner-ocp#458): - cudn-pod-churn-250-24nodes: 50% pod churn with 30m duration - cudn-churn-250-24nodes: 10% CUDN group churn with 3 cycles Temporarily points e2e-benchmarking to mohit-sheth fork with use-fork-kube-burner-ocp branch for pj-rehearse validation. Signed-off-by: Mohit Sheth <msheth@redhat.com>
mohit-sheth
force-pushed
the
cudn-churn-tests
branch
from
f3a4720 to
db182a0
Compare
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
@mohit-sheth: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
1 participant
Summary
cudn-pod-churn-250-24nodes: 50% pod churn with 30m duration, 1m delaycudn-churn-250-24nodes: 10% CUDN group churn with 3 cycles, 10m durationuse-fork-kube-burner-ocpbranch) for pj-rehearse validationTest plan
Summary by CodeRabbit
This PR extends the OpenShift CI perfscale AWS 5.0 nightly periodic suite by adding two new 24-node periodic jobs to validate kube-burner-ocp CUDN churn behavior via the existing
openshift-qe-cudn-densitytest chain.cudn-pod-churn-250-24nodes(cron0 6 * * *) runs onaws-perfscale-qewith 21 additional worker nodes and sets pod churn parameters: 50% churn, 30m duration, 1m delay (withOVERRIDE_ITERATIONS=250).cudn-churn-250-24nodes(cron0 8 * * *) runs on the same cluster profile and sets CUDN group churn parameters: 10% churn, 3 cycles, 10m per cycle (also withOVERRIDE_ITERATIONS=250).Both jobs use the same compute/infra instance type configuration (m6a.2xlarge for compute/control-plane and r5.2xlarge for infra) and execute the CUDN density chain with the usual pre/post step wiring (
openshift-qe-workers-scale→ipi-aws-pre+ ingress/monitoring registry creation, thenipi-aws-post).To enable validation while dependent upstream changes are in-flight, the CUDN density step script temporarily points e2e-benchmarking to a maintained fork: it now clones
https://github.com/mohit-sheth/e2e-benchmarkingand checks out theuse-fork-kube-burner-ocpbranch (instead of the prior upstream repo/tag logic). The fork reference is intended to be reverted back to the upstream repository after the corresponding kube-burner-ocp PR is merged.The PR includes (and the author has been exercising) pj-rehearse runs for both new periodic jobs to confirm the periodic job definitions execute correctly in the rehearse environment.