Skip to content

Prune wrapped Iceberg hour partitions conservatively - #118

Merged
osipovartem merged 1 commit into
embucket-sync-df55.0.0from
fix/df55-hour-wrap-pruning
Oct 15, 2026
Merged

osipovartem merged 1 commit into
embucket-sync-df55.0.0from
fix/df55-hour-wrap-pruning

Conversation

@osipovartem

Copy link
Copy Markdown

Summary

  • Project Hour partition predicates through bounded source-hour intervals and signed i32 wraparound, retaining modern and legacy pre-epoch IDs.
  • Skip pruning when a source range spans all possible hour IDs; preserve the row-level predicate for correctness.
  • Cover negative ranges, inclusive comparisons, and wrapped extremes with manifest-pruning tests.

Validation

  • cargo +1.99.0 test -p datafusion_iceberg --lib pruning_statistics::tests -- --test-threads=4: 35 passed.
  • cargo +1.99.0 test -p datafusion_iceberg --lib -- --test-threads=4: 114 passed, 2 ignored.
  • cargo +1.99.0 clippy -p datafusion_iceberg --all-targets: passed (pre-existing warnings outside this change).
  • cargo +1.99.0 fmt --all -- --check: passed.
  • Independent read-only review: approved.

No per-row work is added; projection is bounded to predicate planning.

@osipovartem
osipovartem merged commit 40cec5c into embucket-sync-df55.0.0 Oct 15, 2026
1 of 2 checks passed
@osipovartem
osipovartem deleted the fix/df55-hour-wrap-pruning branch October 15, 2026 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant