Skip to content

GH-50707: [C++][Gandiva] fix out-of-bounds read in mask utf8proc length - #50709

Open
Arawoof06 wants to merge 1 commit into
apache:mainfrom
Arawoof06:mask-utf8proc-iterate-length
Open

GH-50707: [C++][Gandiva] fix out-of-bounds read in mask utf8proc length#50709
Arawoof06 wants to merge 1 commit into
apache:mainfrom
Arawoof06:mask-utf8proc-iterate-length

Conversation

@Arawoof06

@Arawoof06 Arawoof06 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

The Gandiva mask stubs walk a utf8 string calling utf8proc_iterate(data + bytes_read, data_len, ...), passing the total data_len as the remaining byte count even though the pointer has already advanced by bytes_read. utf8proc_iterate reads up to strlen bytes, so a value ending in a truncated multi-byte glyph (for example a lead byte 0xF0 as the final byte of an exactly sized value buffer) makes it read continuation bytes past data + data_len. This is reachable from mask(), mask_first_n() and mask_last_n() on untrusted string columns. gdv_mask_last_n_utf8_int32 happens to be shielded by its utf8proc_decompose pre-pass, but gdv_mask_first_n_utf8_int32 and mask_utf8_utf8_utf8_utf8 have no such guard and over-read.

What changes are included in this PR?

Pass data_len - bytes_read (the real remaining length) at all four utf8proc_iterate call sites so utf8proc reports the truncated glyph instead of reading past the buffer. mask_utf8_utf8_utf8_utf8 also lacked the char_len < 0 check that the other two paths already have, so I added it there to reject the now-detected invalid input rather than advancing by a negative length.

Are these changes tested?

Yes. TestMaskTruncatedUtf8NoOverread feeds a value whose reported length stops one byte short of a complete euro sign; before the change both functions consumed the out-of-range byte and returned a masked result, after it they report the truncated input. The existing mask tests still pass.

Are there any user-facing changes?

No change for valid utf8. A value that ends in a truncated multi-byte glyph now surfaces an invalid-utf8 error instead of being silently masked using bytes past its end.

This PR contains a "Critical Fix". The wrong length argument lets utf8proc_iterate read past the end of an exactly sized input buffer.

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #50707 has been automatically assigned in GitHub to PR creator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant