Repository navigation
Conversation
Weijun-H
force-pushed
the
case-first-required-operand-reuse
branch
from
November 10, 2026 02:28
b80a0c8 to
bccefb6
Compare
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Active findings identify incorrect evaluation of structurally equal but distinct volatile instances and a rewrite path that can turn valid CASE evaluation into an array-precondition error.
2 open issues
What changed in this PR
Adds an opt-in API for physical-plan producers to reuse a CASE’s first required WHEN operand as its ELSE value, avoiding duplicate evaluation without relying on logical optimization.
Changes:
- Introduces a caller-supplied reuse policy and prepared CASE evaluation.
- Tracks permission through expression rewrites, equality, hashing, and serialization eligibility.
- Adds tests for reuse, branch selection, projection, rewrites, and error ordering.
| File | Description |
|---|---|
datafusion/physical-expr/src/expressions/mod.rs |
Re-exports RequiredOperandReusePolicy. |
datafusion/physical-expr/src/expressions/case.rs |
Implements operand reuse, permission handling, and regression tests. |
Review effort: Balanced
Comment on lines
+423
to
+427
| let operand = owner | ||
| .body | ||
| .else_expr | ||
| .as_ref() | ||
| .expect("authorized CASE has ELSE"); |
Comment on lines
+1583
to
+1586
| rebuilt = rebuilt.with_required_operand_reuse( | ||
| authorized, | ||
| Arc::clone(&required.policy), | ||
| ); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Which issue does this PR close?
Closes #27280.
Rationale for this change
CASE WHEN f(x) = ... THEN ... ELSE f(x) ENDrepeatsf(x)for ELSE rows. This opt-in API reuses its first result at the original evaluation position, avoiding extra work for expensive operands. Direct physical-plan builders can use it without logical CSE, while preserving evaluation position.What changes are included in this PR?
RequiredOperandReusePolicyandCaseExpr::with_required_operand_reuse. Callers identify an unconditional, exactly-once operand and guarantee that column replacement, projection and a private column preserve expression semantics; unknown leaves decline reuse.Eq/Hash;try_to_protoreturnsOk(None)for a CASE with permission.What is the testing strategy for this PR?
Validated tree
42c33d272af2: 77 CASE tests (10 reuse), whole-workspace clippy, formatting, and extended workspace tests pass (13,632 standard tests, 8 ignored; 547/547 SQLLogicTest files). Both new regressions fail on the prior implementation and pass with the instance-selection and scalar-rewrite fixes.The table compares ON/OFF within the same corrected version, not this version against an unpatched baseline. Physical-expression benchmark: Rust 1.99.1, Apple arm64,
release-nonlto; 8,192 rows. Each OFF/ON mode has three rounds of 30 samples. Times below are medians of three per-round arithmetic means oftime/iterations; percentages use those OFF/ON values. First includes CASE construction, authorization and preparation, with inputs/operand nodes prebuilt. Repeated reuses the prepared plan. Output and evaluated-row checks run outside timing. Two input batches alternate; at ELSE 50% each sends a different contiguous half to ELSE. Each iteration evaluates one 8,192-row batch; times are per batch.Positive percentages mean less time. † Paired rounds disagree in direction; no directional conclusion. The 100-column input adds 99 unused Int32 columns sharing buffers; only widths 1 and 100 were tested. These single-machine microbenchmarks establish neither statistical significance nor end-to-end query speedups.
Are there any user-facing changes?
New opt-in physical-expression API. Conventional SQL planning does not enable it automatically. Callers must verify semantic safety and whether avoided computation outweighs added costs; cheap operands can become slower.