Skip to content

[python] Fix the row filter on PyArrow 6 - #10557

Merged
JingsongLi merged 1 commit into
apache:masterfrom
sundapeng:fix-py36-row-filter-select
Oct 15, 2026
Merged

JingsongLi merged 1 commit into
apache:masterfrom
sundapeng:fix-py36-row-filter-select

Conversation

@sundapeng

@sundapeng sundapeng commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Purpose

#10539 made FilterRecordBatchReader._filter_batch_by_row call RecordBatch.select. PyArrow 6, which the Python 3.6 job installs, has Table.select but not RecordBatch.select, so that job fails:

FAILED pypaimon/tests/py36/rest_ao_read_write_test.py::RESTAOReadWritePy36Test::test_append_only_reader_with_filter
E       AttributeError: 'pyarrow.lib.RecordBatch' object has no attribute 'select'
pypaimon/read/reader/filter_record_batch_reader.py:100: AttributeError

For example, job 115681294594 of #10552. Master's Python jobs stopped at flake8 before #10552 fixed it, so this failure only showed up afterwards.

Master now fails the same way, for example job 116061121154 on b634373. #10572 contains the same change, but it is waiting for apache/paimon-rust#1128.

This pull request selects the columns on the Table that the next line builds anyway.

Tests

  • CI's Python 3.6 selection (Python 3.6, PyArrow 6.0.1): 1 failed and 119 passed on master, 120 passed with this change (rechecked on b634373).
  • Python 3.11, the 43 test files that cover the row filter and query auth: the same results as on master.
  • flake8 with dev/cfg.ini passes.

API and Format

No.

Documentation

No.

apache#10539 made FilterRecordBatchReader._filter_batch_by_row call
RecordBatch.select, which PyArrow 6 does not have, so the Python 3.6
job fails test_append_only_reader_with_filter with "'RecordBatch'
object has no attribute 'select'". Select the columns on the Table that
the next line builds anyway; PyArrow 6 has Table.select.

@QuakeWang QuakeWang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit 69f6af5 into apache:master Oct 15, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants