Skip to content

fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops - #10444

Open
haohuaijin wants to merge 2 commits into
apache:mainfrom
haohuaijin:fix-inplace-bitwise-out-of-range-bits
Open

fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops#10444
haohuaijin wants to merge 2 commits into
apache:mainfrom
haohuaijin:fix-inplace-bitwise-out-of-range-bits

Conversation

@haohuaijin

@haohuaijin haohuaijin commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

No dedicated issue — found while implementing #10425, whose optimization is the first caller to depend on the behaviour fixed here. Happy to file one if you'd prefer it tracked separately.

Rationale for this change

apply_bitwise_binary_op and apply_bitwise_unary_op are built around the invariant that only the bits in offset_in_bits..offset_in_bits + len_in_bits are modified — the helpers implementing it (align_to_byte, set_remainder_bits, handle_mutable_buffer_remainder_unary) all say so in their doc comments. Two paths didn't honour it:

1. align_to_byte ignored len_in_bits, writing every bit from bit_offset to the byte boundary. If the range started and ended inside the same non-byte-aligned byte, the trailing bits were overwritten with op applied to padding:

let mut left = vec![0b11111111u8, 0b11111111u8];
apply_bitwise_binary_op(&mut left, 1, &[0u8, 0u8], 0, 1, |a, b| a & b);
// expected [0b11111101, 0b11111111], got [0b00000001, 0b11111111]

2. set_remainder_bits zeroed the boundary byte's out-of-range bits, because it read that byte into the low bits of a u64 and masked it with !((1 << remainder_len) - 1). Once the remainder spanned more than one byte, the mask selected nothing. This hit any length leaving a remainder of 9..63 bits that isn't a multiple of 8.

No existing caller was affected, which is why it went unnoticed: BooleanBufferBuilder::append_packed_range copies with |_a, b| b into capacity advance() just zeroed, and the BooleanBuffer/BooleanArray in-place paths run only on uniquely owned buffers and re-wrap with the original offset/length. So the out-of-range bits were either already zero or unobservable.

What changes are included in this PR?

  • align_to_byte takes len_in_bits and masks its write to bit_offset..bit_offset + min(8 - bit_offset, len_in_bits).
  • set_remainder_bits shifts the boundary byte to the position it actually occupies in the word before masking.
  • Both public functions now document the preservation guarantee.

Are these changes tested?

Yes. The gap existed because the two shared test helpers only asserted bits inside the operated range. They now also assert every bit outside it is byte-for-byte identical before and after, which retroactively covers all their call sites — that alone caught bug 2 in 11 pre-existing tests. Added test_ops_ending_inside_the_first_partial_byte (sweeps every offset/len pair inside one partial byte, for AND/OR/XOR and NOT) plus minimal regression tests for the two reproducers.

cargo test -p arrow-buffer passes (342 unit + 54 doc), as do arrow-array, arrow and arrow-select. Clippy and fmt clean.

Are there any user-facing changes?

A behaviour change in two public functions, in the direction of the documented intent: bits outside the requested range are no longer clobbered. No signature changes, and no existing caller could observe the old behaviour. The guarantee is now explicit in the rustdoc.

…ace bitwise ops

`apply_bitwise_binary_op` and `apply_bitwise_unary_op` are documented
internally as modifying only the bits in
`offset_in_bits..offset_in_bits + len_in_bits` -- see the doc comments on
`align_to_byte`, `set_remainder_bits` and
`handle_mutable_buffer_remainder_unary`. Two paths did not honour that
invariant:

* `align_to_byte` wrote every bit from `bit_offset` to the byte boundary,
  ignoring `len_in_bits`. When the requested range started and ended inside
  the same non-byte-aligned byte, the trailing bits of that byte were
  overwritten with the result of `op` applied to padding. For example
  `apply_bitwise_binary_op(left, 1, right, 0, 1, |a, b| a & b)` cleared bits
  2..8.

* `set_remainder_bits` read the boundary byte into the low bits of a `u64`
  and masked it with `!((1 << remainder_len) - 1)`. Whenever the remainder
  spanned more than one byte that mask selected nothing, so the out-of-range
  bits of the boundary byte were dropped to zero instead of being preserved.

No existing caller was affected. `BooleanBufferBuilder::append_packed_range`
copies with `|_a, b| b` into freshly zeroed capacity, and the
`BooleanBuffer`/`BooleanArray` in-place paths only run on uniquely owned
buffers and re-wrap the result with the original offset and length, so the
out-of-range bits were either already zero or unobservable.

The existing tests only asserted the bits inside the operated range. The two
shared test helpers now also assert that every bit outside the range is
identical before and after, which covers all of their call sites, plus
targeted tests for ranges contained in a single partial byte.

The guarantee is now stated on both public functions, since callers that read
the surrounding bits back depend on it.
…vation

- Rename `align_to_byte`'s new parameter to `remaining_len_in_bits`; the
  binary caller passes the already-clamped `bits_to_next_byte` while the
  unary caller passes the full length, so "total bits to process" only
  described one of them.
- Add `debug_assert_ne!(bit_offset, 0)` to `align_to_byte`, making the
  documented "not byte-aligned" precondition checked, and drop the dead
  `& 0xFF` from the write mask now that the bound is explicit.
- Use the slice length instead of recomputing `ceil(remainder_len, 8)` in
  `set_remainder_bits`; the assertion above already proves them equal.
- Cover `len == 8 - offset` in `test_ops_ending_inside_the_first_partial_byte`;
  the exclusive range skipped the byte-boundary case and never ran at all
  for `offset == 7`.
- Add targeted regression tests for the multi-byte remainder bug. It was
  only covered indirectly through the shared helpers, so weakening them
  would have let it regress silently.
@haohuaijin

Copy link
Copy Markdown
Contributor Author

While implementing #10425 I hit corrupted mask tails, and traced it to two bugs in the in-place bitwise helpers — both write outside offset..offset+len, which they're documented not to do:

  • align_to_byte ignored len_in_bits, so when the range started and ended inside one non-byte-aligned byte, it overwrote the rest of that byte.
  • set_remainder_bits read the boundary byte into the low bits of a u64 before masking, so once the remainder spanned more than one byte its out-of-range bits were zeroed instead of preserved.

cc @alamb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant