Skip to content

[Bug] mod returns NaN instead of raising for a zero floating point divisor in ANSI mode, and ignores -0.0 divisors #23894

Closed

Description

@andygrove

Describe the bug

datafusion-spark's mod shares the try_rem helper in
datafusion/spark/src/function/math/modulus.rs with pmod. That helper has two
gaps in its zero divisor handling. Both are visible through mod.

  1. ANSI mode, floating point divisor. try_rem delegates to Arrow's rem
    kernel when datafusion.execution.enable_ansi_mode is true. Arrow's rem
    only reports division by zero for integer and decimal types. Floating point
    divisors follow IEEE 754 and quietly produce NaN. Spark raises
    REMAINDER_BY_ZERO for a zero divisor of any numeric type.

  2. -0.0 divisor, both modes. The legacy path nulls out zero divisors by
    comparing with eq(right, 0). Arrow's comparison kernels order floating
    point values totally, so -0.0 is not equal to 0.0 and a -0.0 divisor is
    not recognised. Spark's isZero check is a numeric comparison, so -0.0
    counts as zero.

pmod was fixed for both cases as part of an audit of that function and no
longer routes through try_rem. mod still does.

To Reproduce

Spark 4.2.0 (verified with pyspark==4.2.0):

SET spark.sql.ansi.enabled = true;
SELECT MOD(CAST(10.5 AS DOUBLE), CAST(0.0 AS DOUBLE));
-- [REMAINDER_BY_ZERO] Remainder by zero. ... SQLSTATE: 22012
SELECT MOD(CAST(10.5 AS DOUBLE), CAST('-0.0' AS DOUBLE));
-- [REMAINDER_BY_ZERO] Remainder by zero. ... SQLSTATE: 22012

SET spark.sql.ansi.enabled = false;
SELECT MOD(CAST(10.5 AS DOUBLE), CAST(0.0 AS DOUBLE));   -- NULL
SELECT MOD(CAST(10.5 AS DOUBLE), CAST('-0.0' AS DOUBLE)); -- NULL

DataFusion:

set datafusion.execution.enable_ansi_mode = true;
SELECT mod(10.5::float8, 0.0::float8);   -- NaN, expected an error
SELECT mod(10.5::float8, -0.0::float8);  -- NaN, expected an error

set datafusion.execution.enable_ansi_mode = false;
SELECT mod(10.5::float8, -0.0::float8);  -- NaN, expected NULL

Expected behavior

mod raises for a zero divisor of any numeric type in ANSI mode and returns
NULL in legacy mode, treating -0.0 as zero in both.

Affected versions

Spark 3.5.8, 4.0.4, 4.1.3 and 4.2.0 all behave the same way here. The behaviour
difference across those versions is only the error class name, which changed
from DIVIDE_BY_ZERO to REMAINDER_BY_ZERO in Spark 4.1.

Relevant code

datafusion/spark/src/function/math/modulus.rs, try_rem.

spark_pmod in the same file shows one possible shape for the fix. Its
is_zero helper treats -0.0 as zero and its ANSI branch masks the check by
the validity of the dividend, because Spark's remainder expressions are null
intolerant and a NULL dividend short circuits to NULL before the divisor is
validated.

Surfaced by the audit-datafusion-spark-expression skill.

Activity

  1. added
    bugSomething isn't working
    sparkChanges to datafusion-spark crate
    on Jul 25, 2026
  2. u70b3 commented on Jul 30, 2026

    @u70b3
    Contributor

    take — I'll fix mod's zero-divisor handling in the shared try_rem helper, mirroring the shape #23898 established for pmod:

    • ANSI mode: raise for a zero divisor of any numeric type (including floating point, where Arrow's rem quietly returns NaN), with the check masked by the validity of the dividend to match Spark's null-intolerant semantics
    • Both modes: treat -0.0 as a zero divisor (Arrow's total-order float comparisons don't match it against 0.0)

    I'll replicate the is_zero/negative_zero helpers from #23898 so the two PRs stay consistent; whichever lands second can dedupe them in a rebase. Not touching #23897 here since that's a repo-wide error-text policy question.

  3. added 5 commits that reference this issue on Jul 31, 2026
    612ea63
    f3c194b
    4641b46
    0270100
    faf495e
  4. added a commit that references this issue on Aug 15, 2026
    07126ed
  5. added a commit that references this issue on Aug 16, 2026
    2da78d6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingsparkChanges to datafusion-spark crate

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions