Skip to content

[python] Use Native for Java calendar query authorization masks - #10545

Merged
JingsongLi merged 1 commit into
apache:masterfrom
JingsongLi:codex/native-rest-calendar-masks
Oct 13, 2026
Merged

JingsongLi merged 1 commit into
apache:masterfrom
JingsongLi:codex/native-rest-calendar-masks

Conversation

@JingsongLi

Copy link
Copy Markdown
Contributor

Purpose

Use Native for the 13 calendar column masks that PyPaimon already supports in REST query authorization. Depends on apache/paimon-rust#1123: apache/paimon-rust#1123. Keep this draft until Rust main contains that implementation; Native CI continues to build Rust main.

  • Include calendar fieldRef operands in dependency collection. Hidden mask sources must be read and participate in stale-schema, field identity and cross-mask checks.
  • Align Python fallback with Java DateExtractTransform, including complete DATE / TIMESTAMP validation, NULLs, BCE, ISO weeks, full DATE32 and large timestamp epoch-day wrapping. Reject LTZ, atomic object types, invalid precision and malformed type spellings.
  • Preserve Arrow kernels for ordinary years 1..9999; use numeric Java calendar arithmetic outside their safe range. A 65,536-row YEAR batch remains near baseline (0.296ms vs 0.280ms locally).
  • Exercise append, primary-key and data-evolution tables with strict Native planning/reading. Check all 13 parts on DATE and timestamp(ms/us/ns), hidden source fields, temporal-family rebinding, NULLs, masked user predicates, LIMIT, stale/cross-mask refusal, and captured authorization rules during setup/disabled Native/wire fallback.

Tests

  • Python predicate parser / authorization validation / widening / masking / catalog tests: 338 passed, 2,061 subtests passed.
  • Calendar and existing REST Native authorization integration tests: 106 passed. Native instrumentation recorded 232 plans, 205 reads, 105 writes and 105 commits.
  • 146 actual Java-generated vectors committed; all 13 parts plus NULL checked in Python and the companion Rust PR.
  • Independent actual Java differential check: 10,087 boundary/random vectors, 131,183 scalar comparisons, zero mismatches. Additional fast-path endpoint and empty/all-null checks passed.
  • Focused flake8, Python 3.6 syntax checks and Python license headers passed.

@leaves12138 leaves12138 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete change at d59c516. Python predicate, native calendar mask, fallback, and validation tests passed; no correctness issues found.

@JingsongLi
JingsongLi marked this pull request as ready for review October 13, 2026 14:49
@JingsongLi
JingsongLi merged commit 46426e8 into apache:master Oct 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants