Skip to content

fix(parser)!: move JSON extraction operators to Postgres's binary-operator precedence tier - #8063

Open
fivetran-kwoodbeck wants to merge 4 commits into
mainfrom
parser/fix-json-arrow-operator-precedence
Open

fix(parser)!: move JSON extraction operators to Postgres's binary-operator precedence tier#8063
fivetran-kwoodbeck wants to merge 4 commits into
mainfrom
parser/fix-json-arrow-operator-precedence

Conversation

@fivetran-kwoodbeck

@fivetran-kwoodbeck fivetran-kwoodbeck commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Fixes #8035 where ->, ->>, #>, #>>, and ? were parsed as tight-binding column accessors (like . and ::) via COLUMN_OPERATORS, instead of Postgres's actual "any other operator" precedence tier, below +/-, level with || and the bitwise operators, left-associative.

This caused casts, subscripts, and arithmetic adjacent to these operators to bind to the wrong operand. To fix it:

  • Added JSON_EXTRACT_OPERATORS, a new operator tier parsed inside _parse_bitwise in parser.py, alongside the existing || handling.
  • Moved the relevant operators out of COLUMN_OPERATORS and into JSON_EXTRACT_OPERATORS for PostgresParser and DuckDBParser. Other dialects still treat -> as lambda syntax.

Sample input:

SELECT a #>> b::TEXT[]

Previous: cast attaches to the whole extract:

SELECT CAST(a #>> b AS TEXT[])

Fixed: cast attaches to the path operand, matching Postgres:

SELECT a #>> CAST(b AS TEXT[])

@georgesittas georgesittas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some operators that also aren't parsed correctly in main today: ?&, ?|, @>, <@ and &&. The problem is that they're in RANGE_PARSERS, i.e., in the "comparison tier". We can fix them as a follow-up.

SELECT '{"a":[1,2]}'::jsonb @> '{"a":[1]}'::jsonb -> 'a';
SELECT '{"a":[1]}'::jsonb <@ '{"a":[1,2]}'::jsonb -> 'a';
SELECT '{"a":1}'::jsonb ?& ARRAY['a'] -> 'b';
SELECT '{"a":1}'::jsonb ?| ARRAY['a'] -> 'b';
SELECT ARRAY[1] && ARRAY[1] -> 'a';

-- ERROR: operator does not exist: boolean -> unknown

Comment thread sqlglot/parser.py Outdated

# Binary-operator-tier JSON extraction ops (Postgres's "any other operator" tier,
# below +/-, level with ||). Same value signature as COLUMN_OPERATORS: (self, this, rhs).
JSON_EXTRACT_OPERATORS: t.ClassVar[dict[TokenType, t.Callable]] = {}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is a good name, e.g., Postgres adds JSONBContains in it which isn't an "extraction. What about BITWISE_OPERATORS?

@fivetran-kwoodbeck fivetran-kwoodbeck Aug 7, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about BITWISE_OPERATORS, they don't have AND/OR/XOR/shift semantics. How about just JSON_OPERATORS?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go for it.

Comment thread sqlglot/parsers/duckdb.py Outdated
if k not in (TokenType.ARROW, TokenType.DARROW)
}

JSON_EXTRACT_OPERATORS = {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These lambdas and 3 lambdas in the Postgres parser are identical to the base ones. Should we clean these up?

@fivetran-kwoodbeck
fivetran-kwoodbeck force-pushed the parser/fix-json-arrow-operator-precedence branch from 4cdffb5 to 64de050 Compare August 7, 2026 15:52
@fivetran-kwoodbeck

Copy link
Copy Markdown
Collaborator Author

There are some operators that also aren't parsed correctly in main today: ?&, ?|, @>, <@ and &&. The problem is that they're in RANGE_PARSERS, i.e., in the "comparison tier". We can fix them as a follow-up.

SELECT '{"a":[1,2]}'::jsonb @> '{"a":[1]}'::jsonb -> 'a';
SELECT '{"a":[1]}'::jsonb <@ '{"a":[1,2]}'::jsonb -> 'a';
SELECT '{"a":1}'::jsonb ?& ARRAY['a'] -> 'b';
SELECT '{"a":1}'::jsonb ?| ARRAY['a'] -> 'b';
SELECT ARRAY[1] && ARRAY[1] -> 'a';

-- ERROR: operator does not exist: boolean -> unknown

I'll add it as a ticket, I may have found another duckdb operator precedence issue too.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

SQLGlot Integration Test Results

✅ All tests passed

Comparing:

  • this branch (sqlglot:parser/fix-json-arrow-operator-precedence @ sqlglot 9fbac96)
  • baseline (main @ sqlglot a9c0bea)

Overall

main: 182937 total, 160861 passed (pass rate: 87.9%)

sqlglot:parser/fix-json-arrow-operator-precedence: 176566 total, 155532 passed (pass rate: 88.1%)

Transitions:
No change

Dialect pair changes: 0 previous results not found, 2 current results not found

✅ All tests passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Postgres: JSON operators are parsed as accessors rather than binary operators, so operands bind wrongly

2 participants