Skip to content

fix: avoid Float64 percentile_cont interpolation overflow - #25274

Open
aoto-tech wants to merge 2 commits into
apache:mainfrom
aoto-tech:fix-percentile-cont-f64-overflow
Open

aoto-tech wants to merge 2 commits into
apache:mainfrom
aoto-tech:fix-percentile-cont-f64-overflow

Conversation

@aoto-tech

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

percentile_cont can return Infinity for finite Float64 inputs when upper - lower overflows, even though the interpolated result is finite. I hit this with [-DBL_MAX, DBL_MAX] at 0.5; the median should be 0.

What changes are included in this PR?

I keep the current difference-based formula when the subtraction is finite. If both endpoints are finite but their difference overflows, I switch to weighting each endpoint separately. That avoids the infinite intermediate without changing the existing Float16 or non-finite-input paths.

I also added a direct unit test for the overflow branch with symmetric, asymmetric, and quarter-percentile cases, plus an end-to-end SQL test for the reported query.

What is the testing strategy for this PR?

The new unit test failed with inf != 0.0 before the fix.

I ran:

  • cargo test --profile ci -p datafusion-functions-aggregate --lib (233 passed)
  • the new aggregate.slt query through the sqllogictest runner (passed)
  • cargo clippy --profile ci -p datafusion-functions-aggregate --all-targets --all-features --no-deps -- -D warnings
  • cargo fmt --all -- --check

Are there any user-facing changes?

Yes. percentile_cont now returns a finite result when finite Float64 endpoints have an overflowing difference but a finite interpolation result. There is no API change.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Sep 13, 2026
@aoto-tech
aoto-tech force-pushed the fix-percentile-cont-f64-overflow branch from 1182ccc to 0eb20bf Compare September 13, 2026 15:54
@aoto-tech
aoto-tech force-pushed the fix-percentile-cont-f64-overflow branch from 0eb20bf to 1b9f59b Compare September 13, 2026 15:56
@aoto-tech
aoto-tech marked this pull request as ready for review September 13, 2026 16:00
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.91%. Comparing base (85d4cbb) to head (f832f61).

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #25274   +/-   ##
=======================================
  Coverage   81.91%   81.91%           
=======================================
  Files        1134     1134           
  Lines      425631   425653   +22     
  Branches   425631   425653   +22     
=======================================
+ Hits       348647   348667   +20     
  Misses      56304    56304           
- Partials    20680    20682    +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

percentile_cont overflows while interpolating finite Float64 values

2 participants