Repository navigation
Conversation
Signed-off-by: Mofesola Babalola <me@mofesola.com>
mofesola
requested a deployment
to
external-contributors
November 4, 2026 06:52 — with
GitHub Actions
Waiting
|
Hi @mofesola, welcome and thank you for your contribution. We will try to review your Pull Request as quickly as possible. In the meantime, please take a look at the contribution guidelines if you have not done so already. |
|
CC @envoyproxy/api-shepherds: Your approval is needed for changes made to |
Author
|
The Istio side that turns this on for InferencePool clusters is up as a draft: istio/istio#62620 |
This branch is waiting to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commit Message: override_host: match endpoints behind an internal listener
Additional Description:
An endpoint picker (Gateway API Inference Extension) returns the pod it picked as
IP:Port, and override_host looks that string up in the cluster's host map. When the cluster's endpoints are internal addresses, the lookup can never match. Istio ambient does this: a gateway reaches ambient pods through theconnect_originateinternal listener, so the endpoints areenvoy://connect_originate/IP:Port. Every request then silently goes to the fallback policy and the picker is ignored. Istio issue with the repro and numbers: istio/istio#62618This adds
internal_listener_namesto OverrideHost. If an override host isn't found as given, it's looked up again asenvoy://<name>/<override host>for each name, in order. When a host is matched that way,selected_host_keygets the override host as the picker gave it (IP:Port), so the picker sees the same value it sent. Off by default, so nothing changes unless it's set.I went with a list of listener names and an exact lookup instead of matching any internal address by
endpoint_id, to keep the lookup O(1) and avoid matching an internal address the control plane didn't intend. Happy to change that if you'd prefer a different shape.I used an AI coding assistant to help write this change and the tests. I've reviewed all of it and can take changes.
Risk Level: Low. New optional field, off by default.
Testing: Unit tests for the new field: match through an internal listener and the value written to
selected_host_key, no match when the field is unset, a plain address still matched first, and listener names tried in order. All//test/extensions/load_balancing_policies/override_host/...targets pass. On a real cluster (Istio 1.31.3 ambient, llm-d router v0.11.0) I forced the same lookup string from a Lua filter: the picker's choice was followed for 12/12 requests with the pool in ambient, against 3/3/3/3 round robin without it.Docs Changes: API docs on the new field.
Release Notes: Added in
changelogs/current/new_features.Platform Specific Features: N/A
[Optional API Considerations:] New repeated string field on
envoy.extensions.load_balancing_policies.override_host.v3.OverrideHost.