You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug] mod returns NaN instead of raising for a zero floating point divisor in ANSI mode, and ignores -0.0 divisors #23894
datafusion-spark's mod shares the try_rem helper in datafusion/spark/src/function/math/modulus.rs with pmod. That helper has two
gaps in its zero divisor handling. Both are visible through mod.
ANSI mode, floating point divisor.try_rem delegates to Arrow's rem
kernel when datafusion.execution.enable_ansi_mode is true. Arrow's rem
only reports division by zero for integer and decimal types. Floating point
divisors follow IEEE 754 and quietly produce NaN. Spark raises REMAINDER_BY_ZERO for a zero divisor of any numeric type.
-0.0 divisor, both modes. The legacy path nulls out zero divisors by
comparing with eq(right, 0). Arrow's comparison kernels order floating
point values totally, so -0.0 is not equal to 0.0 and a -0.0 divisor is
not recognised. Spark's isZero check is a numeric comparison, so -0.0
counts as zero.
pmod was fixed for both cases as part of an audit of that function and no
longer routes through try_rem. mod still does.
mod raises for a zero divisor of any numeric type in ANSI mode and returns NULL in legacy mode, treating -0.0 as zero in both.
Affected versions
Spark 3.5.8, 4.0.4, 4.1.3 and 4.2.0 all behave the same way here. The behaviour
difference across those versions is only the error class name, which changed
from DIVIDE_BY_ZERO to REMAINDER_BY_ZERO in Spark 4.1.
spark_pmod in the same file shows one possible shape for the fix. Its is_zero helper treats -0.0 as zero and its ANSI branch masks the check by
the validity of the dividend, because Spark's remainder expressions are null
intolerant and a NULL dividend short circuits to NULL before the divisor is
validated.
Surfaced by the audit-datafusion-spark-expression skill.
take — I'll fix mod's zero-divisor handling in the shared try_rem helper, mirroring the shape #23898 established for pmod:
ANSI mode: raise for a zero divisor of any numeric type (including floating point, where Arrow's rem quietly returns NaN), with the check masked by the validity of the dividend to match Spark's null-intolerant semantics
Both modes: treat -0.0 as a zero divisor (Arrow's total-order float comparisons don't match it against 0.0)
I'll replicate the is_zero/negative_zero helpers from #23898 so the two PRs stay consistent; whichever lands second can dedupe them in a rebase. Not touching #23897 here since that's a repo-wide error-text policy question.
Activity
u70b3 commentedon Jul 30, 2026
take — I'll fix
mod's zero-divisor handling in the sharedtry_remhelper, mirroring the shape #23898 established forpmod:remquietly returnsNaN), with the check masked by the validity of the dividend to match Spark's null-intolerant semantics-0.0as a zero divisor (Arrow's total-order float comparisons don't match it against0.0)I'll replicate the
is_zero/negative_zerohelpers from #23898 so the two PRs stay consistent; whichever lands second can dedupe them in a rebase. Not touching #23897 here since that's a repo-wide error-text policy question.fix(spark): correct mod ANSI zero divisor and negative zero handling
fix(spark): correct mod ANSI zero divisor and negative zero handling
fix(spark): correct mod ANSI zero divisor and negative zero handling
fix(spark): correct mod ANSI zero divisor and negative zero handling
fix(spark): correct mod ANSI zero divisor and negative zero handling
fix(spark): correct mod ANSI zero divisor and negative zero handling
docs: add an agent skill for auditing datafusion-spark expressions (a…
fix(spark): correct mod ANSI zero divisor and negative zero handling (a…
docs: add an agent skill for auditing datafusion-spark expressions (a…
fix(spark): correct mod ANSI zero divisor and negative zero handling (a…