Skip to content

[FLINK-40684][model] Fix tensor name case in Triton models. - #29217

Open
damjad wants to merge 1 commit into
apache:masterfrom
damjad:fix/flink-model-triton-tensor-names
Open

damjad wants to merge 1 commit into
apache:masterfrom
damjad:fix/flink-model-triton-tensor-names

Conversation

@damjad

@damjad damjad commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

Triton Inference Server's KServe V2 protocol treats tensor names as case-sensitive. A model
configured with input / output tensors in its config.pbtxt rejects requests that send
INPUT / OUTPUT, returning HTTP 400 unexpected inference output 'OUTPUT'.

TritonInferenceModelFunction.buildInferenceRequest() unconditionally uppercased both tensor names:

// Before (buggy):
inputNode.put("name", inputName.toUpperCase());
outputNode.put("name", outputName.toUpperCase());

This caused every request to a model with lowercase tensor names to fail at the network layer.

Brief change log

  • Remove .toUpperCase() from the input tensor name assignment in buildInferenceRequest()
  • Remove .toUpperCase() from the output tensor name assignment in buildInferenceRequest()
  • Add TritonTensorNameCasingTest — two MockWebServer-backed integration tests that capture the outgoing HTTP request and assert the name fields in inputs and outputs match the column names declared in the model schema without any case transformation

Verifying this change

This change added tests and can be verified as follows:

  • TritonTensorNameCasingTest#testLowercaseTensorNamesArePreserved — columns named input/output produce "name":"input" / "name":"output" in the JSON body
  • TritonTensorNameCasingTest#testMixedCaseTensorNamesArePreserved — columns named myInput/myOutput are also sent unchanged
  • Tests fail when .toUpperCase() is re-introduced (expected: "input" but was: "INPUT")
  • Manually verified against a live Triton endpoint (torch-model with lowercase tensors): HTTP 400 → HTTP 200

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects how connectors interact with Flink: no
  • The SQL / Table API: no
  • Core ML / Model inference (flink-model-triton): yes

Documentation

No public API or configuration changes. The fix aligns behaviour with the KServe V2 specification,
which defines tensor names as opaque strings whose casing is determined by the model's
config.pbtxt


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 noreply@anthropic.com

@damjad damjad changed the title Fix/flink model triton tensor names [FLINK-40684][model] Fix tensor name case in Triton models. Sep 16, 2026
@flinkbot

flinkbot commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@spuru9

spuru9 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

@damjad One of the commits has Coauthor-by: AI tag can you squash the commits and add disclosure as per AI guidelines.

@damjad
damjad force-pushed the fix/flink-model-triton-tensor-names branch from 77d46e2 to 23f8279 Compare September 16, 2026 20:29
…e requests.

Triton's KServe V2 protocol treats tensor names as case-sensitive. The previous
implementation called inputName.toUpperCase() and outputName.toUpperCase() when
building the inference request, causing HTTP 400 errors for any model whose
config.pbtxt declared lowercase tensor names (e.g. 'input', 'output').

Remove the toUpperCase() calls so tensor names are sent exactly as declared in
the model schema. Add TritonTensorNameCasingTest with two MockWebServer-backed
tests that capture the outgoing HTTP request and assert the name fields in inputs
and outputs preserve their original casing.

Generated-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@damjad
damjad force-pushed the fix/flink-model-triton-tensor-names branch from 23f8279 to eabadbc Compare September 16, 2026 20:32
@damjad

damjad commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

@spuru9 updated.

@damjad

damjad commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

@featzhang tagging you for visibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants