Describe the bug, including details regarding any error messages, version, and platform.
Describe the bug, including details regarding any error messages, version, and platform
Array::ComputeLogicalNullCount segfaults for dictionary arrays whose value type is null(), even though the array passes full validation:
auto type = dictionary(int32(), null());
auto array = DictArrayFromJSON(type, R"([0, 0, null])", "[null]");
ASSERT_OK(array->ValidateFull());
array->ComputeLogicalNullCount(); // segfault
The values array of such a dictionary is a NullArray, which reports null_count == length without having a validity bitmap (buffers[0] is null). The early-out in dict_util::LogicalNullCount only triggers when the dictionary's null count is 0, so the per-index loop unconditionally reads the null validity-bitmap pointer:
https://github.com/apache/arrow/blob/main/cpp/src/arrow/util/dict_util.cc#L56-L59
Note that element-wise Array::IsValid handles the same data correctly (it falls back to null_count == length when there is no bitmap), so the aggregate and per-element views of logical nullness disagree.
Found while working on GH-50338 (#50347).
The crash is also reachable from the bindings through the C++ count kernel, which uses ComputeLogicalNullCount when counting nulls (e.g. pyarrow.compute.count(arr, mode="only_null") on such an array).
Component(s)
C++
Component(s)
C++
Describe the bug, including details regarding any error messages, version, and platform.
Describe the bug, including details regarding any error messages, version, and platform
Array::ComputeLogicalNullCountsegfaults for dictionary arrays whose value type isnull(), even though the array passes full validation:The values array of such a dictionary is a
NullArray, which reportsnull_count == lengthwithout having a validity bitmap (buffers[0]is null). The early-out indict_util::LogicalNullCountonly triggers when the dictionary's null count is 0, so the per-index loop unconditionally reads the null validity-bitmap pointer:https://github.com/apache/arrow/blob/main/cpp/src/arrow/util/dict_util.cc#L56-L59
Note that element-wise
Array::IsValidhandles the same data correctly (it falls back tonull_count == lengthwhen there is no bitmap), so the aggregate and per-element views of logical nullness disagree.Found while working on GH-50338 (#50347).
The crash is also reachable from the bindings through the C++
countkernel, which usesComputeLogicalNullCountwhen counting nulls (e.g.pyarrow.compute.count(arr, mode="only_null")on such an array).Component(s)
C++
Component(s)
C++