Skip to content

override_host: match endpoints behind an internal listener - #48996

Open
mofesola wants to merge 1 commit into
envoyproxy:mainfrom
mofesola:override-host-internal-listener-names
Open

mofesola wants to merge 1 commit into
envoyproxy:mainfrom
mofesola:override-host-internal-listener-names

Conversation

@mofesola

@mofesola mofesola commented Nov 4, 2026

Copy link
Copy Markdown

Commit Message: override_host: match endpoints behind an internal listener

Additional Description:
An endpoint picker (Gateway API Inference Extension) returns the pod it picked as IP:Port, and override_host looks that string up in the cluster's host map. When the cluster's endpoints are internal addresses, the lookup can never match. Istio ambient does this: a gateway reaches ambient pods through the connect_originate internal listener, so the endpoints are envoy://connect_originate/IP:Port. Every request then silently goes to the fallback policy and the picker is ignored. Istio issue with the repro and numbers: istio/istio#62618

This adds internal_listener_names to OverrideHost. If an override host isn't found as given, it's looked up again as envoy://<name>/<override host> for each name, in order. When a host is matched that way, selected_host_key gets the override host as the picker gave it (IP:Port), so the picker sees the same value it sent. Off by default, so nothing changes unless it's set.

I went with a list of listener names and an exact lookup instead of matching any internal address by endpoint_id, to keep the lookup O(1) and avoid matching an internal address the control plane didn't intend. Happy to change that if you'd prefer a different shape.

I used an AI coding assistant to help write this change and the tests. I've reviewed all of it and can take changes.

Risk Level: Low. New optional field, off by default.

Testing: Unit tests for the new field: match through an internal listener and the value written to selected_host_key, no match when the field is unset, a plain address still matched first, and listener names tried in order. All //test/extensions/load_balancing_policies/override_host/... targets pass. On a real cluster (Istio 1.31.3 ambient, llm-d router v0.11.0) I forced the same lookup string from a Lua filter: the picker's choice was followed for 12/12 requests with the pool in ambient, against 3/3/3/3 round robin without it.

Docs Changes: API docs on the new field.

Release Notes: Added in changelogs/current/new_features.

Platform Specific Features: N/A

[Optional API Considerations:] New repeated string field on envoy.extensions.load_balancing_policies.override_host.v3.OverrideHost.

Signed-off-by: Mofesola Babalola <me@mofesola.com>
@repokitteh-read-only

Copy link
Copy Markdown

Hi @mofesola, welcome and thank you for your contribution.

We will try to review your Pull Request as quickly as possible.

In the meantime, please take a look at the contribution guidelines if you have not done so already.

🐱

Caused by: #48996 was opened by mofesola.

see: more, trace.

@repokitteh-read-only

Copy link
Copy Markdown

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to (api/envoy/|docs/root/api-docs/).
envoyproxy/api-shepherds assignee is @markdroth
CC @envoyproxy/api-watchers: FYI only for changes made to (api/envoy/|docs/root/api-docs/).

🐱

Caused by: #48996 was opened by mofesola.

see: more, trace.

@mofesola

mofesola commented Nov 4, 2026

Copy link
Copy Markdown
Author

The Istio side that turns this on for InferencePool clusters is up as a draft: istio/istio#62620

This branch is waiting to be deployed

1 waiting deployment
external-contributors — 7c0049d0 Waiting Nov 4, 2026 by mofesola via authorize #61082
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants