Add citus.enable_or_clause_arm_pruning for per-shard OR-arm pruning (… - #8859
Open
Colm (colm-mchugh) wants to merge 1 commit into
Open
Colm (colm-mchugh) wants to merge 1 commit into
Colm (colm-mchugh) wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## release-14.0 #8859 +/- ##
=================================================
- Coverage 88.50% 69.33% -19.17%
=================================================
Files 289 289
Lines 64614 64702 +88
Branches 8126 8138 +12
=================================================
- Hits 57188 44863 -12325
- Misses 5085 16732 +11647
- Partials 2341 3107 +766 🚀 New features to boost your workflow:
|
Colm (colm-mchugh)
force-pushed
the
release-14-8642-bp
branch
from
September 23, 2026 16:06
fd3d6fd to
119d59b
Compare
…8642) DESCRIPTION: Adds citus.enable_or_clause_arm_pruning to drop unreachable OR arms per shard ## Summary Adds an optimization, GUC `citus.enable_or_clause_arm_pruning` (**default on**), for multi-shard SELECTs whose `WHERE` clause is a top-level `OR` where each arm constrains the distribution column, e.g.: ```sql SELECT * FROM t WHERE (k = 1 AND v = 7) OR (k = 2 AND v = 7); ``` Today Citus pushes the **full N-arm OR to every shard**, so each shard runs an N-way bitmap OR even though only the arm whose key hashes to that shard can match. With this enabled, each shard's task query keeps only the arms that can match on that shard, so the worker runs a single precise index scan. ## Before / After (2 shards, index on `(k, v)`) Before (`citus.enable_or_clause_arm_pruning = off`, stock behavior): ``` -> Task (shard owning k=1) -> Bitmap Heap Scan Recheck Cond: ((k=1 AND v=7) OR (k=2 AND v=7)) -> BitmapOr -> Bitmap Index Scan Index Cond: (k=1 AND v=7) -> Bitmap Index Scan Index Cond: (k=2 AND v=7) <- dead on this shard -> Task (shard owning k=2) ... same full 2-way BitmapOr ... ``` After (`= on`): ``` -> Task (shard owning k=1): Index Only Scan Index Cond: (k=1 AND v=7) -> Task (shard owning k=2): Index Only Scan Index Cond: (k=2 AND v=7) ``` ## How it works In `SqlTaskList()` (the logical/physical-planner path used by multi-shard SELECTs — these are not router queries), while generating each per-shard task query, drop the arms of top-level OR expressions that cannot match on that task's shard. - `PruneUnreachableOrArms()` recurses through top-level `List`/`AND`; for an `OR` it keeps only the arms whose pruned shard set (from the existing `PruneShards()`) contains the task's shard. An arm with no constraint on the distribution column prunes to all shards and is therefore always kept, so the rewrite is a guaranteed semantic no-op. - Reachability depends only on `(relation, rangeTableId, arm)`, never on the task, so `ReachableShardListForArm()` caches results in a job-wide hash table (`HTAB`) keyed on `(relationId, rangeTableId, arm)`: `PruneShards()` runs once per distinct arm instead of once per (shard x arm). The hash uses the arm's serialized form and the comparator uses node `equal()`, so structurally identical arms from different per-task query copies share one entry and hash collisions never produce a wrong hit. ## Safety - **On by default**; set `citus.enable_or_clause_arm_pruning = off` to restore the previous behavior byte-for-byte (the entire new code path is inside `if (EnableOrClauseArmPruning)`). The full `check-multi` schedule (193 tests) passes with the default on; the only expected-output change is additive `DEBUG` lines in `multi_hash_pruning` (the per-arm `PruneShards()` now runs during task generation) — no result/row changes. - Correctness invariant: an arm is dropped on shard S only when it provably matches no row on S. Two independent code reviews found no correctness issues; joins, self-joins, params, NULLs, reference-table joins, ranges/IN, nested and mixed boolean, and value collisions were all validated. ## Tests `src/test/regress/sql/multi_or_arm_pruning.sql` (wired into `multi_1_schedule`): - 5 before/after per-shard `Filter` comparisons (`shard_filters()` helper), including a hash-collision case that asserts the `>=2`-equality-arms-kept shape and is self-checking (if `k=1`/`k=5` ever stop colliding the grouped filter splits and the test fails rather than silently passing), plus pure-range and mixed range+equality shapes. - 18 result-parity checks (off vs on, multiset equality via `EXCEPT ALL`): basic, shard collision, no-dist-constraint arm, mixed AND/OR, nested OR, range, `IN`, colocated join, cross-table-arm join, reference-table join, null-distribution-key (single-shard) join, repartition (non-colocated) join, prepared/parameterized query, no-WHERE, top-level `NOT`, and no-op cases. - The `NULL`-key join exercises the `!HasDistributionKeyCacheEntry` eligibility guard; the repartition join exercises the `fragmentType != CITUS_RTE_RELATION` guard. - Deterministic and environment-independent (helpers normalize shard ids; no ports/task-order dependence). The `Filter` outputs were verified against an independent ground-truth query formulation, not just off-vs-on parity. Passes the real pg_regress harness. ## Known considerations / follow-ups - The "BEFORE" EXPLAIN line reflects the worker planner's boolean factoring of the OR, which is mildly PG-version-sensitive; may need a variant expected file for PG17/18. The "AFTER" lines (what the feature controls) are robust. - CHANGELOG intentionally not touched (Citus generates it at release time). --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Colm (colm-mchugh)
force-pushed
the
release-14-8642-bp
branch
from
September 24, 2026 14:51
119d59b to
e865538
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…#8642)
DESCRIPTION: Adds citus.enable_or_clause_arm_pruning to drop unreachable OR arms per shard
Summary
Adds an optimization, GUC
citus.enable_or_clause_arm_pruning(default on), for multi-shard SELECTs whoseWHEREclause is a top-levelORwhere each arm constrains the distribution column, e.g.:Today Citus pushes the full N-arm OR to every shard, so each shard runs an
N-way bitmap OR even though only the arm whose key hashes to that shard can
match. With this enabled, each shard's task query keeps only the arms that can
match on that shard, so the worker runs a single precise index scan.
Before / After (2 shards, index on
(k, v))Before (
citus.enable_or_clause_arm_pruning = off, stock behavior):After (
= on):How it works
In
SqlTaskList()(the logical/physical-planner path used by multi-shardSELECTs — these are not router queries), while generating each per-shard task
query, drop the arms of top-level OR expressions that cannot match on that
task's shard.
PruneUnreachableOrArms()recurses through top-levelList/AND; for anORit keeps only the arms whose pruned shard set (from the existingPruneShards()) contains the task's shard. An arm with no constraint on thedistribution column prunes to all shards and is therefore always kept, so the
rewrite is a guaranteed semantic no-op.
(relation, rangeTableId, arm), never on thetask, so
ReachableShardListForArm()caches results in a job-wide hash table(
HTAB) keyed on(relationId, rangeTableId, arm):PruneShards()runs onceper distinct arm instead of once per (shard x arm). The hash uses the arm's
serialized form and the comparator uses node
equal(), so structurally identical arms from different per-task query copies share one entry and hashcollisions never produce a wrong hit.
Safety
citus.enable_or_clause_arm_pruning = offto restorethe previous behavior byte-for-byte (the entire new code path is inside
if (EnableOrClauseArmPruning)). The fullcheck-multischedule (193 tests)passes with the default on; the only expected-output change is additive
DEBUGlines inmulti_hash_pruning(the per-armPruneShards()now runsduring task generation) — no result/row changes.
matches no row on S. Two independent code reviews found no correctness issues;
joins, self-joins, params, NULLs, reference-table joins, ranges/IN, nested
and mixed boolean, and value collisions were all validated.
Tests
src/test/regress/sql/multi_or_arm_pruning.sql(wired intomulti_1_schedule):Filtercomparisons (shard_filters()helper),including a hash-collision case that asserts the
>=2-equality-arms-keptshape and is self-checking (if
k=1/k=5ever stop colliding the groupedfilter splits and the test fails rather than silently passing), plus pure-range
and mixed range+equality shapes.
EXCEPT ALL):basic, shard collision, no-dist-constraint arm, mixed AND/OR, nested OR, range,
IN, colocated join, cross-table-arm join, reference-table join, null-distribution-key (single-shard) join, repartition (non-colocated) join,prepared/parameterized query, no-WHERE, top-level
NOT, and no-op cases.NULL-key join exercises the!HasDistributionKeyCacheEntryeligibilityguard; the repartition join exercises the
fragmentType != CITUS_RTE_RELATIONguard.
ports/task-order dependence). The
Filteroutputs were verified against anindependent ground-truth query formulation, not just off-vs-on parity. Passes
the real pg_regress harness.
Known considerations / follow-ups
the OR, which is mildly PG-version-sensitive; may need a variant expected file
for PG17/18. The "AFTER" lines (what the feature controls) are robust.
DESCRIPTION: PR description that will go into the change log, up to 78 characters