Skip to content

refactor(proto): migrate FilterExec serde#23708

Merged
adriangb merged 2 commits into
apache:mainfrom
Phoenix500526:issue/23499
Jul 21, 2026
Merged

refactor(proto): migrate FilterExec serde#23708
adriangb merged 2 commits into
apache:mainfrom
Phoenix500526:issue/23499

Conversation

@Phoenix500526

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

The centralized protobuf dispatcher couples each execution plan to datafusion-proto. Adding plan serialization therefore requires changes outside the plan implementation.

Move FilterExec encoding and decoding behind its ExecutionPlan hooks. Preserve the existing wire format and retain the old helper methods as deprecated compatibility APIs.

What changes are included in this PR?

Are these changes tested?

Yes

Are there any user-facing changes?

@github-actions github-actions Bot added proto Related to proto crate physical-plan Changes to the physical-plan crate labels Jul 20, 2026
@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details
     Cloning apache/main
    Building datafusion-physical-plan v54.0.0 (current)
       Built [  38.523s] (current)
     Parsing datafusion-physical-plan v54.0.0 (current)
      Parsed [   0.143s] (current)
    Building datafusion-physical-plan v54.0.0 (baseline)
       Built [  37.979s] (baseline)
     Parsing datafusion-physical-plan v54.0.0 (baseline)
      Parsed [   0.152s] (baseline)
    Checking datafusion-physical-plan v54.0.0 -> v54.0.0 (no change; assume patch)
     Checked [   0.662s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  79.312s] datafusion-physical-plan
    Building datafusion-proto v54.0.0 (current)
       Built [  62.061s] (current)
     Parsing datafusion-proto v54.0.0 (current)
      Parsed [   0.019s] (current)
    Building datafusion-proto v54.0.0 (baseline)
       Built [  60.878s] (baseline)
     Parsing datafusion-proto v54.0.0 (baseline)
      Parsed [   0.019s] (baseline)
    Checking datafusion-proto v54.0.0 -> v54.0.0 (no change; assume patch)
     Checked [   0.263s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip

--- failure trait_method_marked_deprecated: trait method #[deprecated] added ---

Description:
A trait method is now #[deprecated]. Downstream crates will get a compiler warning when using this method.
        ref: https://doc.rust-lang.org/reference/attributes/diagnostics.html#the-deprecated-attribute
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/trait_method_marked_deprecated.ron

Failed in:
  method try_into_filter_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:719
  method try_from_filter_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:719

     Summary semver requires new minor version: 0 major and 1 minor checks failed
    Finished [ 124.414s] datafusion-proto

@github-actions github-actions Bot added the auto detected api change Auto detected API change label Jul 20, 2026
@codecov-commenter

codecov-commenter commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 67.14286% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.70%. Comparing base (774f0bc) to head (7172823).

Files with missing lines Patch % Lines
datafusion/proto/src/physical_plan/mod.rs 7.14% 13 Missing ⚠️
datafusion/physical-plan/src/filter.rs 82.14% 4 Missing and 6 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23708      +/-   ##
==========================================
- Coverage   80.71%   80.70%   -0.01%     
==========================================
  Files        1089     1089              
  Lines      368280   368293      +13     
  Branches   368280   368293      +13     
==========================================
- Hits       297250   297237      -13     
- Misses      53314    53341      +27     
+ Partials    17716    17715       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment on lines +2960 to +2963
#[deprecated(
since = "55.0.0",
note = "unused by DataFusion; `FilterExec` serializes itself via `ExecutionPlan::try_to_proto`"
)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have the deprecated method delegate to the new one to avoid code duplication and drift?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

The centralized protobuf dispatcher couples each execution plan to
datafusion-proto. Adding plan serialization therefore requires changes
outside the plan implementation.

Move FilterExec encoding and decoding behind its ExecutionPlan hooks.
Preserve the existing wire format and retain the old helper methods as
deprecated compatibility APIs.

CLOSES apache#23499

Signed-off-by: Jiawei Zhao <Phoenix500526@163.com>
FilterExec serde moved to per-plan hooks, but the deprecated PhysicalPlanNodeExt helpers retained separate implementations. Keeping the wire-format logic in both paths allows the compatibility API to drift from normal dispatch.

Build the existing encode and decode adapter contexts in the deprecated helpers and forward to FilterExec's canonical hooks, preserving the old API without duplicating serialization logic.

Refs apache#23499

Signed-off-by: Jiawei Zhao <Phoenix500526@163.com>

@adriangb adriangb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@adriangb
adriangb enabled auto-merge July 21, 2026 05:35
@adriangb
adriangb added this pull request to the merge queue Jul 21, 2026
Merged via the queue into apache:main with commit 02a902c Jul 21, 2026
40 checks passed
@Phoenix500526
Phoenix500526 deleted the issue/23499 branch July 21, 2026 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto detected api change Auto detected API change physical-plan Changes to the physical-plan crate proto Related to proto crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proto: migrate FilterExec to the try_to_proto hook

3 participants