Add plan parent-reduction rule API - #9196
Conversation
Merging this PR will degrade performance by 0.46%
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
3675240 to
79b24a6
Compare
29488db to
6c8e44e
Compare
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Replace the per-node `Plan::optimize` recursion with `children()` + `with_children()`, which lets `PlanRef` implement `vortex_array::expr::traversal::Node`. Optimization rules now live in `plan/optimize.rs` as a `NodeRewriter` and are written once against the generic tree instead of inside every plan kind. `PlanRef` becomes a newtype over `Arc<dyn Plan>` so the foreign `Node` trait can be implemented for it, matching `ScalarFnRef`. It derefs to `dyn Plan`, so call sites are unchanged. Children are now stored uniformly and built eagerly, which removes `LazyPlanChildren` and its per-slot closures. Absent optional slots are omitted from `children()` rather than reported as `None`, so out-of-bounds access has one behaviour across every plan kind instead of the previous mix of `Ok(None)` and an error. Building the tree eagerly means a layout with no plan implementation now fails in `new_plan` rather than when its child is first accessed. The two deferral tests are replaced by tests asserting that eager failure. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012obBhJ8oPZoBbKyeS79yMv Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Introduce `PlanVTable` plus a `Plan<V>` container and a `DynPlan` erased trait, matching how `Layout<V>` and `ScalarFn<V>` are built. Common fields — dtype, row count, children — live in the container and are written once; operator-specific state lives in `V::PlanData` and is reached typed through `Deref` rather than by downcasting. Plan operators no longer hold a layout handle. Each carries everything it needs, so a rewrite can reason about a plan's shape alone: FlatPlan -> SegmentScan read one segment ChunkedPlan -> Concat concatenate children row-wise StructPlan -> Pack assemble a struct from field children DictPlan -> Take index values by codes ListPlan -> ListPack assemble a list from elements and offsets ExpressionPlan -> Eval apply an expression to its child RowIdxPlan -> RowIdx offset row numbers Naming operators for what they compute rather than for what produced them is what lets one rule cover every case: `Concat` of `Concat` flattens on shape alone, and `Take` over `SegmentScan` is the dictionary pushdown, no matter which layout each was lowered from. `new_plan` becomes `lower` in `plan/lower.rs`, the only module that knows about layouts. The dependency runs one way: lowering imports plans, plans never import layouts. `PlanVTable` carries `id` and a `Metadata` codec so a registry and a serialization envelope can be added without reshaping operators. Operators whose state is recoverable serialize their metadata today; `SegmentScan` and `Eval` return `None` until a read context and a bound expression have codecs. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012obBhJ8oPZoBbKyeS79yMv Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
6c8e44e to
08b86f2
Compare
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
08b86f2 to
825351d
Compare
Summary
PlanParentReduceRuleAPI for child-driven physical-plan rewritesPlanParentRuleSetwhose first successful rewrite winsvortex_layout::plan::optimizerStack
This is part 2 of the plan-native scan stack. It is based on #9142 and is followed by #9166.
Scope
This PR only introduces the rule framework and API contract. It registers no concrete rules and
does not change optimization behavior.
Checks
cargo test -p vortex-layoutcargo clippy -p vortex-layout --all-targets --all-features -- -D warningscargo +nightly fmt --all -- --checkgit diff --check