fix(cilium): limit concurrent agent and Envoy rollouts - #4065
Conversation
devantler
left a comment
There was a problem hiding this comment.
🤖 Generated by the Agentic Engineer
Exact-head review of aaa8511d6db480aadd64d8c8eb9e23261e13502b: the Cilium 1.20.2 chart exposes both updateStrategy and envoy.updateStrategy, each defaulting to maxUnavailable: 2; the production render now sets both to RollingUpdate/1. The existing CI-registered activation test failed against the baseline and passed after the change. Its assertions still reject the old Helm disableWait handoff, and the separate autoscaler-gate test passed. Shellcheck and git diff --check passed. No RBAC, policy, Coroot suppression, or image change appears in this diff.
No blocking code findings. This is a concurrency reduction, not proof of zero packet loss: the two DaemonSets can still overlap on different nodes, so post-deployment readiness and Coroot readback remain required.
devantler
left a comment
There was a problem hiding this comment.
🤖 Generated by the Agentic Engineer
Exact-head review of f53276dc1c9f6a08eff8f484763e04fa8919de88 against bad3342cdaaaaa70510847ae942c2d70b9a9cdf0: the authored manifest changes only the agent and Envoy DaemonSet rollout values (each RollingUpdate/maxUnavailable=1); the test checks both fields in the production render and still rejects the old disableWait handoff. The first CI run's authorization gate correctly failed on the changed HelmRelease. Its approved renderer measured aggregate 33eb2b3a1201e8c6c304eab0bb65feed702d8244ed96ebb315b90f3901d95792 and diagnosed only HelmRelease kube-system/cilium as changed; the new commit records that digest and the exact prior digest 65fecc82a527f6b5fc9c2154b58d2268d4eb4cfa49bb2ec9dfdfd65789621250. Local authorization Go tests now pass with the new value. No grant-bearing object, service account, binding, policy, or AWS identity changed.
No blocking code findings. Await the rerun of the required CI gate and production readiness/readback before treating the risk reduction as delivered. The two DaemonSets can still roll on different nodes simultaneously; this does not establish zero packet loss.
@coderabbitai review |
|
✅ Action performedReview finished.
|
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (3)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details🔇 Additional comments (3)
📝 WalkthroughWalkthroughThe Cilium HelmRelease now sets Priority: ➖ Normal Severity of issue fixed: Medium Merge Risk: ⚪ Minimal · up to Both Cilium DaemonSets are configured for bounded rollouts, and no actionable issue remains before normal merge checks. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Why
During the Cilium 1.20.2 rollout, production observability jobs hit DNS and service timeouts and left failed-job warnings. The chart currently permits two networking agents and two Envoy pods to be unavailable at once. That concurrency increases the disruption risk during planned upgrades, even though the exact packet path of each timeout is not proven.
What
Keep normal Helm-managed rolling upgrades, but limit each Cilium agent and Envoy DaemonSet to one unavailable pod at a time. Replace the old test that required the chart default with a production-render assertion for the new bounds. The old temporary rollout handoff and
disableWaitstay absent; no Coroot error checks are weakened.The production authorization gate correctly detected the changed Cilium HelmRelease. Its approved CI renderer reported exactly that one changed resource and no added or removed resource; the aggregate fingerprint is re-approved with the previous digest recorded. No identity or grant changes.
Fixes #4064