Skip to content

[read] Support transformed REST query authorization filters - #1124

Merged
JingsongLi merged 1 commit into
apache:mainfrom
JingsongLi:codex/native-rest-filter-transforms
Oct 14, 2026
Merged

JingsongLi merged 1 commit into
apache:mainfrom
JingsongLi:codex/native-rest-filter-transforms

Conversation

@JingsongLi

@JingsongLi JingsongLi commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

PyPaimon already evaluates transformed REST query-auth row filters, but Native reads reject every non-FIELD_REF leaf. This adds those existing authorization expressions to Rust core without adding public Python binding APIs.

Changes

  • Represent authorization filters as an internal expression tree and compare typed transform outputs through the existing predicate evaluator.
  • Share transform evaluation with column masking: string transforms, all 13 DATE/TIMESTAMP extractors and the existing Java-compatible CAST subset.
  • Collect hidden operands by current field identity, preserving original-data authorization before masking, user filtering and LIMIT.
  • Preserve Java leaf evaluation order, per-row AND/OR short circuit and ordered response.filter entries. Deserialize every literal, including unused literals and unreachable children.
  • Reject non-NULL binary predicate literals, as Java does. Keep unsupported CASTs and rebound CAST executors explicitly unsupported rather than reinterpret their results.
  • Default bare CHAR to length one, matching Java.

CAST support remains deliberately limited to the same validated conversions as Native masks; this does not claim all Java CAST executors or introduce capabilities absent from PyPaimon.

Verification

  • Reproduced Native rejection of an existing UPPER authorization filter before implementation.
  • 48 core query-auth tests, 109 predicate tests and the default-length type regression passed.
  • 38 REST query-auth integration tests passed, including a transformed calendar/string guard before masking, user filtering and LIMIT across append, primary-key and data-evolution modes.
  • Built the Python binding and ran 247 strict Native/Python authorization tests, including hidden operands, all calendar parts and captured-policy fallback; 481 Native reads were exercised.
  • Rust 1.98.0 full-workspace/all-target Clippy with fulltext,vortex and -D warnings passed; cargo fmt --check passed.
  • Compared Python semantics against 103 actual Java oracle cases and independently reviewed authorization enforcement and Java parity.

Companion PyPaimon PR apache/paimon#10548 corrects Python evaluation order and CAST width/remapping semantics and adds the Native regression matrix. Its CI will continue to build Rust main and depends on this PR merging first.

@leaves12138 leaves12138 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the transformed REST query-auth implementation and its authorization ordering, hidden-operand collection, transform/cast handling, and fail-closed behavior. The targeted Rust query-auth tests (70/70) and the new REST integration test pass; no blocking issues found. Approving.

@JingsongLi
JingsongLi merged commit 4ceef6b into apache:main Oct 14, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants