Skip to content

[python] Align empty and metadata-only results with runtime read schemas - #10578

Merged
JingsongLi merged 4 commits into
apache:masterfrom
YannByron:codex/native-schema-ci
Oct 16, 2026
Merged

JingsongLi merged 4 commits into
apache:masterfrom
YannByron:codex/native-schema-ci

Conversation

@YannByron

@YannByron YannByron commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

After #10572, empty Ray row-id results and metadata-only search could expose a different schema from normal reads. Build both output schemas with read_arrow_schema so they follow the same runtime contract.

Align existing read expectations with that contract. The existing search fastpath-versus-lookup and Ray empty-versus-nonempty tests cover the two fixes; the existing native schema tests cover nullability and write constraints. Convert the BinaryRow read fixture once after setup writes, and retain strict value/schema comparisons and invalid-NULL write checks.

Also update the existing unsupported-cast case and nested MAP conversion for current core behavior. This addresses the independent master Native CI failures and contains no CWD changes from #10567.

Tests

  • Local Python 3.13 / PyArrow 19.0.1, fallback reader/writer with pypaimon-rust 0.4.0: 411 passed, 8 skipped, 5 deselected across 12 affected or regression modules. Optional Ray/Lance/Vortex/index dependencies are unavailable locally; native-specific and REST read/write coverage runs in CI.
  • Repository-configured Flake8 and git diff --check passed.
  • Full CI, including Native CI, is pending for this revision. The earlier native run with Rust 95d0e11 passed the retained Ray/search regressions; its sole remaining ARRAY BLOB diagnostic-case mismatch has been corrected and checked against the exact CI error.

YannByron and others added 4 commits October 15, 2026 11:50
…schemas

Use the core read schema for Ray empty results and metadata-only search
results. Keep read expectations separate from logical write constraints
and preserve strict value, schema, and invalid-NULL checks.

Co-Authored-By: Codex <noreply@anthropic.com>
AI-Model: gpt-6
Co-Authored-By: Codex <noreply@ai-tool.com>
AI-Contributed/Feature: 14/14
AI-Contributed/UT: 202/202
Exercise the core schema boundary without requiring a new native wheel.
The test rejects the previous hard-coded non-nullable metadata field.

Co-Authored-By: Codex <noreply@anthropic.com>
AI-Model: gpt-6
Co-Authored-By: Codex <noreply@ai-tool.com>
AI-Contributed/Feature: 0/0
AI-Contributed/UT: 11/11
Match the Arrow ListArray Non-nullable diagnostic while preserving the
ValueError and invalid-NULL write checks.

Co-Authored-By: Codex <noreply@anthropic.com>
AI-Model: gpt-6
Co-Authored-By: Codex <noreply@ai-tool.com>
AI-Contributed/Feature: 0/0
AI-Contributed/UT: 2/2
Reuse the existing Ray and search schema regressions and remove redundant
schema assertions. Normalize the BinaryRow read fixture after setup writes
so individual tests keep their original strict comparisons.

Co-Authored-By: Codex <noreply@anthropic.com>
AI-Model: gpt-6
Co-Authored-By: Codex <noreply@ai-tool.com>
AI-Contributed/Feature: 0/0
AI-Contributed/UT: 104/104
@YannByron
YannByron marked this pull request as ready for review October 15, 2026 08:00

@QuakeWang QuakeWang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit 1b6f7bb into apache:master Oct 16, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants