Skip to content

test: characterize correlated NOT IN null-aware behaviour - #25558

Merged
adriangb merged 1 commit into
apache:mainfrom
pydantic:na-tests
Sep 21, 2026
Merged

adriangb merged 1 commit into
apache:mainfrom
pydantic:na-tests

Conversation

@adriangb

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Correlated NOT IN returns wrong results in several shapes, and a few do not plan at all. The fixes span two layers — the hash join executor and decorrelation — and touch different shapes. Landing the coverage first, pinned to what DataFusion does today, makes each fix's diff show exactly which behaviour it changes instead of burying the flips in a large change.

Nothing here changes behaviour. Every expectation is what main produces right now, and the wrong ones carry a note and a link to the issue.

Concretely, this is wrong today (DuckDB 1.5.2 and PostgreSQL 17.11 both return three rows):

CREATE TABLE oc(id INT, g INT) AS VALUES (1,5),(2,5),(3,0),(4,NULL),(NULL,5),(NULL,0);
CREATE TABLE ic(id INT) AS VALUES (1),(NULL);
SELECT id, g FROM oc WHERE oc.id NOT IN (SELECT ic.id FROM ic WHERE oc.g > 0);
-- returns no rows

What changes are included in this PR?

sqllogictest, in null_aware_anti_join.slt and null_aware_mark_join.slt:

  • correlated NOT IN with a non-equality correlation, which stays a residual join filter;
  • a correlation naming only outer columns, so it cannot become an equi-join key;
  • a constant value expression, with and without a correlation;
  • a subquery inside the IN value, both spellings of the outer IN;
  • IS NOT NULL over a subquery predicate, and a comparison between two marks — the contexts that can tell a NULL mark from a FALSE mark;
  • plan pins for the join the planner picks in each case.

Benchmarks:

  • Q09, a correlated non-negated IN. It must not use a null-aware join. Q01–Q08 all cover the direction where null-awareness is required, so a regression that adds it where it is not needed is invisible to them. Q09 passes today.
  • correctness canaries on Q05–Q08, each comparing the NOT IN result against a reference that does not use NOT IN. All four disagree today, so they are pinned to false.

What is the testing strategy for this PR?

This PR is tests. Expected results were verified against DuckDB 1.5.2 and PostgreSQL 17.11. The whole suite is green on main, including all nine benchmarks.

Are there any user-facing changes?

No.

🤖 Generated with Claude Code

Adds sqllogictest and benchmark coverage for correlated `NOT IN`, pinned to
what DataFusion does today. Several of these expectations are wrong, and a few
shapes do not plan at all; each such block carries a note and a link to
apache#25336. The fixes flip them, so the
flip is visible in those diffs rather than buried in a large change.

sqllogictest, in `null_aware_anti_join.slt` and `null_aware_mark_join.slt`:
- correlated `NOT IN` with a non-equality correlation, which stays a residual
  join filter;
- a correlation that names only outer columns, so it cannot become an
  equi-join key;
- a constant value expression with and without a correlation;
- a subquery inside the `IN` value, both spellings of the outer `IN`;
- `IS NOT NULL` over a subquery predicate and a comparison between two marks,
  the contexts that can tell a NULL mark from a FALSE mark;
- plan pins for the joins the planner chooses in each case.

Benchmarks:
- Q09, a correlated non-negated `IN`. It must not use a null-aware join, a
  direction none of Q01-Q08 covers, and it passes today;
- correctness canaries on Q05-Q08, each comparing the `NOT IN` result with a
  reference that does not use `NOT IN`. All four currently disagree, so they
  are pinned to `false`.

Expected results verified with DuckDB and PostgreSQL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.41%. Comparing base (1e09a2a) to head (6f70c9d).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #25558      +/-   ##
==========================================
- Coverage   82.42%   82.41%   -0.01%     
==========================================
  Files        1138     1138              
  Lines      435429   435429              
  Branches   435429   435429              
==========================================
- Hits       358889   358875      -14     
- Misses      54839    54848       +9     
- Partials    21701    21706       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@adriangb

Copy link
Copy Markdown
Contributor Author

Thanks @xudong963

@adriangb
adriangb added this pull request to the merge queue Sep 21, 2026
Merged via the queue into apache:main with commit 31a4ca0 Sep 21, 2026
41 checks passed
@adriangb
adriangb deleted the na-tests branch September 21, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants