Skip to content

helm: configurable liveness, readiness and startup probes + named container ports - #2172

Open
DrFaust92 wants to merge 4 commits into
robusta-dev:masterfrom
DrFaust92:helm-configurable-probes
Open

DrFaust92 wants to merge 4 commits into
robusta-dev:masterfrom
DrFaust92:helm-configurable-probes

Conversation

@DrFaust92

@DrFaust92 DrFaust92 commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

What

Adds livenessProbe, readinessProbe and startupProbe values for the runner and the forwarder, rendered only when set, and names the container ports so probes can reference them by name (http/5000 on the runner, metrics/2112 on the forwarder).

New docs page at docs/setup-robusta/health-probes.rst covering which endpoints each component actually serves, which to point probes at, and why to reference ports by name.

The only change to a default render is the two ports: blocks, so existing installs get one pod restart on upgrade and no other behavior change.

Why

Follows the endpoint/history writeup and proposal in #1158, which @adamantal 👍'd.

  • /healthz reflects sink health, so a liveness probe on it restarts the runner whenever a sink flaps — nothing is wrong with the runner. That is what the 2023 probes did (add liveness probe to the runner (check discovery health) #825, add startup probe for long boot time #845) before 2b2292c5 reverted them, and chore(helm): define pod probes #1960 stalled after re-adding pod probes. Liveness on /metrics restarts only a genuinely wedged process; /healthz is documented as a readiness option for anyone who wants the sink-aware signal.
  • Forwarder has no /healthz — only /metrics on :2112 — so that is what the docs point at.
  • Default-off keeps this out of the way of anyone who doesn't want probes, while unblocking platforms that require them.
  • Named ports mean a probe reads port: http instead of repeating 5000, survives a listen-port change, and resolves in NetworkPolicy and Prometheus Operator configs too. A name that no container declares also fails with an explicit port "..." not found rather than a generic connection refused.

Service targetPort deliberately stays numeric. A named targetPort resolves per pod, so upgrading from a chart version whose pods had no named port leaves the old pod Ready with its endpoint published with no port at all — I checked on kind and the EndpointSlice comes back ports: null — and the Service black-holes traffic until the rollout finishes. Numeric resolves either way. It can move to the name in a later version once no supported upgrade path starts from unnamed pods.

Note on the original report: automountServiceAccountToken: false no longer produces a running-but-unready pod. On 0.49.0 the runner exits on its first API call and the forwarder exits fatal on the missing kubeconfig, so both land in CrashLoopBackOff — Kubernetes already reports that. Probes here are for wedged processes, slow starts, and rollout/Service gating; the docs say so rather than implying they detect misconfiguration.

Testing

Installed on kind twice — once with numeric probe ports, once with all probes referencing the names (port: http, port: metrics). Both pods reached 1/1 Ready with 0 restarts, and EndpointSlices kept real ports. Confirmed helm template is byte-identical to master apart from the two ports: blocks, and rendered clean with runner.hardenedFs=true and grafanaRenderer.enableContainer=true.

Runner and forwarder probe blocks are rendered only when set, so the default
install is byte-identical to before.

Liveness is documented on /metrics rather than /healthz. /healthz reflects
sink health, so a transient sink outage on a liveness probe restarts the runner
while nothing is wrong with it - the reason the 2023 probes were reverted in
2b2292c. /healthz is documented as a readiness option instead.

Closes robusta-dev#1158
@coderabbitai

coderabbitai Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6d961580-e25a-4ace-a8fc-055549fadb5e
📥 Commits

Reviewing files that changed from the base of the PR and between 3cc2b20 and e96c518.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2ac2a812-74f7-46fb-8a90-55eaf660e19a
📥 Commits

Reviewing files that changed from the base of the PR and between 8b0b149 and 3cc2b20.

📒 Files selected for processing: 4
  • docs/setup-robusta/index.rst
  • helm/robusta/templates/forwarder.yaml
  • helm/robusta/templates/runner.yaml
  • helm/robusta/values.yaml
🚧 Files skipped from review as they are similar to previous changes: 1
  • docs/setup-robusta/index.rst

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Walkthrough

The Helm chart adds optional startup, liveness, and readiness probe settings for the runner and forwarder. The templates declare named container ports and render configured probes. Setup documentation describes endpoints, configuration examples, and endpoint selection.

Changes

Health probe configuration

Layer / File(s) Summary
Probe settings and setup guidance
helm/robusta/values.yaml, docs/setup-robusta/health-probes.rst, docs/setup-robusta/index.rst
Adds optional probe settings for both workloads. Documents endpoints, named ports, examples, startup behavior, and Service port guidance.
Container ports and probe rendering
helm/robusta/templates/forwarder.yaml, helm/robusta/templates/runner.yaml
Adds named container ports and conditionally renders configured startupProbe, livenessProbe, and readinessProbe blocks.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Suggested reviewers: avi-robusta

Merge Risk: ⚪ Minimal · up to 3cc2b

The chart adds opt-in probes while leaving default probe behavior unchanged. The documented endpoints and ports match the runner and forwarder, and no actionable merge risk remains.

Pre-merge checks: Passed 5
Check name Status Explanation
Description check Passed The description clearly explains the configurable probes, named ports, documentation, rationale, upgrade behavior, and testing for this changeset.
Title check Passed The title clearly summarizes the main Helm changes: configurable health probes and named container ports.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (4 skipped: 4 unsupported.)

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

Declares http/5000 on the runner and metrics/2112 on the forwarder so probes
can reference the port by name, and documents why that is preferable.

Service targetPort stays numeric on purpose. A named targetPort resolves per
pod, so upgrading from a chart version whose pods had no named port leaves the
old pod Ready with its endpoint published with no port at all - verified on
kind, the EndpointSlice comes back with ports: null - and the Service
black-holes traffic until the rollout completes.
@DrFaust92 DrFaust92 changed the title helm: configurable liveness, readiness and startup probes helm: configurable liveness, readiness and startup probes + named container ports Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant