Skip to content

[python] Integrate native REST authorization casts - #10551

Merged
JingsongLi merged 4 commits into
apache:masterfrom
JingsongLi:codex/native-rest-numeric-casts
Oct 14, 2026
Merged

JingsongLi merged 4 commits into
apache:masterfrom
JingsongLi:codex/native-rest-numeric-casts

Conversation

@JingsongLi

@JingsongLi JingsongLi commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Integrate the Java-compatible REST authorization CAST implementation already merged in apache/paimon-rust#1125 into Python Native reads. CAST and authorization expression execution remain in Rust.

Changes

  • Preserve the physical read type while making masked output fields nullable according to the captured grant, including NOT NULL targets, named projections, empty plans and batch readers.
  • Preserve field and schema metadata when producing masked or projected output.
  • Add compact Native integration tests across append, primary-key and data-evolution tables. The tests prohibit Python authorization expression execution and Python read/scan fallback.
  • Cover representative numeric, decimal, text, binary and temporal CASTs, rebinding, authorization short circuiting, hidden columns, filter/limit ordering, exact output storage and invalid runtime classes.
  • Cover Native schema evolution from Parquet LargeUtf8 to INT.
  • Keep the process timezone unchanged in tests and exercise Native authorization/temporal CASTs in separate Shanghai and New York processes in CI, with system tzdata installed.

The PR is limited to five files: 29 production insertions and eight deletions, 319 test lines and a Native CI timezone regression step. The duplicated Python CAST implementation and large Java-vector fixtures have been removed. The full Java capability matrix and reference vectors live in the merged Rust PR.

Verification

  • Compact Native integration suite and existing Native authorization tests before adding three timezone cases: 226 passed.
  • Dedicated Native CAST and temporal regressions: 55 passed per process in UTC, Asia/Shanghai and America/New_York.
  • Additional Native LargeUtf8 schema-evolution regression: passed.
  • Python predicate/authorization/evolution/projection tests: 515 passed, 2,127 subtests.
  • Existing reader/evolution/projection regressions: 91 passed, 23 subtests.
  • Python 3.6 compatibility test directory under the local Python 3.13 runtime: 79 passed.
  • Numeric utility and split serialization regressions: 68 passed, 14 subtests.
  • Flake8, Apache license headers, Python 3.6 syntax and diff checks passed.
  • Independent review reproduced all 41 representative CAST expectations against Java and found no differences.

Native CI continues to build paimon-rust main; no fork or PR revision is pinned.

@JingsongLi JingsongLi changed the title [python] Align REST numeric authorization casts with Java [python] Complete Java scalar REST authorization casts Oct 14, 2026
@JingsongLi JingsongLi changed the title [python] Complete Java scalar REST authorization casts [python] Integrate native REST authorization casts Oct 14, 2026
@JingsongLi
JingsongLi marked this pull request as ready for review October 14, 2026 13:50

@leaves12138 leaves12138 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the latest head 461b1e6. I found no blocking issues. The Python changes correctly keep native authorization execution in Rust and only adapt output nullability, field/schema metadata, projections, and native schema-evolution wiring. Verification passed: Rust binding built from paimon-rust#1125 head c747879; native CAST and schema-evolution tests passed 361/361 with TZ=UTC; adjacent Python authorization, projection, and schema-validation regressions passed 301/301; Python syntax and diff checks passed. The single GitHub Actions check was still queued when this review was submitted.

@JingsongLi
JingsongLi merged commit b634373 into apache:master Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants