Skip to content

Extend grammar: joins, GROUP BY, ORDER BY/LIMIT, set ops, WMR, windows#7

Merged
def- merged 1 commit into
MaterializeInc:masterfrom
def-:grammar-extensions
Jul 10, 2026
Merged

Extend grammar: joins, GROUP BY, ORDER BY/LIMIT, set ops, WMR, windows#7
def- merged 1 commit into
MaterializeInc:masterfrom
def-:grammar-extensions

Conversation

@def-

@def- def- commented Jul 10, 2026

Copy link
Copy Markdown

Fix two long-standing generation bugs:

  • table_ref::factory was missing braces around the subquery branch, so joined_table was unreachable and no JOIN was ever generated
  • case_expr::out printed the true branch twice, so CASE expressions always had identical branches

New grammar coverage:

  • inner/left/right/full outer joins, nested operands parenthesized
  • GROUP BY (by ordinal) with appended aggregates and HAVING
  • ORDER BY (asc/desc, nulls first/last), LIMIT/OFFSET
  • UNION/INTERSECT/EXCEPT [ALL] with type-templated operands
  • WITH MUTUALLY RECURSIVE with RETURN AT RECURSION LIMIT
  • table functions in FROM (unnest, generate_series, jsonb_each, jsonb_array_elements), optionally WITH ORDINALITY
  • ranking (row_number/rank/dense_rank) and value (lag/lead/ first_value/last_value) window functions, plus window frames
  • IS DISTINCT FROM predicates
  • interesting string constants (unicode, LIKE metacharacters, quoting hazards) instead of only '0'/'1'/'2'/'10'

Noise reduction so the new constructs don't drown CI in known errors:

  • classify map_agg as an aggregate
  • keep pseudo-typed aggregates out of select lists and HAVING
  • untyped literals for anynonarray/anyelement instead of casts that always fail
  • retry CASE/templated select list items until types match exactly
  • drop select_for_update, the Materialize parser rejects FOR

Also report impedance statistics in the --log-json output.

Fix two long-standing generation bugs:
- table_ref::factory was missing braces around the subquery branch, so
  joined_table was unreachable and no JOIN was ever generated
- case_expr::out printed the true branch twice, so CASE expressions
  always had identical branches

New grammar coverage:
- inner/left/right/full outer joins, nested operands parenthesized
- GROUP BY (by ordinal) with appended aggregates and HAVING
- ORDER BY (asc/desc, nulls first/last), LIMIT/OFFSET
- UNION/INTERSECT/EXCEPT [ALL] with type-templated operands
- WITH MUTUALLY RECURSIVE with RETURN AT RECURSION LIMIT
- table functions in FROM (unnest, generate_series, jsonb_each,
  jsonb_array_elements), optionally WITH ORDINALITY
- ranking (row_number/rank/dense_rank) and value (lag/lead/
  first_value/last_value) window functions, plus window frames
- IS DISTINCT FROM predicates
- interesting string constants (unicode, LIKE metacharacters, quoting
  hazards) instead of only '0'/'1'/'2'/'10'

Noise reduction so the new constructs don't drown CI in known errors:
- classify map_agg as an aggregate
- keep pseudo-typed aggregates out of select lists and HAVING
- untyped literals for anynonarray/anyelement instead of casts that
  always fail
- retry CASE/templated select list items until types match exactly
- drop select_for_update, the Materialize parser rejects FOR <lockmode>

Also report impedance statistics in the --log-json output.
@def- def- merged commit ce6e69b into MaterializeInc:master Jul 10, 2026
2 checks passed
def- added a commit to MaterializeInc/materialize that referenced this pull request Jul 13, 2026
Propagate SQLsmith thread failures (OOM, bad return codes) instead of
silently swallowing them, fix the broken --exclude-catalog flag, and
lock the shared result aggregate.

Extend coverage: seed data with NULLs, edge values, an empty and a 10k
row table, indexes, and list/map/array/range columns. Run two instances
per server, every fourth one as an unprivileged role with RBAC checks
enabled. Print SQLsmith's impedance stats so blacklisted grammar
productions become visible.

Suppress the type-unification error family that the extended SQLsmith
grammar (joins, GROUP BY, set ops, WITH MUTUALLY RECURSIVE, window
functions) triggers. Using
MaterializeInc/sqlsmith#7

Test runs:
https://buildkite.com/materialize/release-qualification/builds/1310 &
https://buildkite.com/materialize/nightly/builds/17150 (both green)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant