Don't leak ARRAY_AGG's inline ORDER BY into the generated NULL filter [CLAUDE]#7907
Closed
chuenchen309 wants to merge 1 commit into
Closed
Don't leak ARRAY_AGG's inline ORDER BY into the generated NULL filter [CLAUDE]#7907chuenchen309 wants to merge 1 commit into
chuenchen309 wants to merge 1 commit into
Conversation
… [CLAUDE] When transpiling ARRAY_AGG from a null-excluding dialect (e.g. Spark) to one that includes nulls, a NULL FILTER is added. With an inline ORDER BY, ArrayAgg.this is an exp.Order, so the filter was built as `FILTER(WHERE x ORDER BY y ... IS NOT NULL)`, which no dialect can parse. Unwrap the Order to filter on the column itself, matching _add_arrayagg_null_filter's documented "before ORDER BY wrapping" contract and the existing DuckDB path. Co-authored-by AI: Claude (Claude Code). [CLAUDE]
Collaborator
|
Garbage in -> garbage out, see this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When ARRAY_AGG is transpiled from a null-excluding dialect (e.g. Spark) to one that includes nulls, sqlglot adds a
FILTER(WHERE col IS NOT NULL)to preserve the semantics. If the ARRAY_AGG has an inlineORDER BY,ArrayAgg.thisis anexp.Order, so the filter was generated asFILTER(WHERE x ORDER BY y ... IS NOT NULL)— invalid SQL that no target dialect can parse back.The fix unwraps the
Orderso the filter is built from the column itself, which is what_add_arrayagg_null_filterdocuments it expects ("before ORDER BY wrapping") and what the DuckDB path already does. Cases without an inline ORDER BY are unchanged.Verified against the full unit suite (1243 tests) with an added Spark→Presto/Postgres test.
Disclosure: found, fixed, tested and described by an AI agent (Claude Code) running on this account. The account holder reviews every change and is accountable for it.