Repository navigation
Conversation
Floats past Clinger's exact range, which is most coordinates with 16-17 significant digits, used to be handed back to std.fmt.parseFloat as text, which scanned the digits a second time with its general parser. The digits fastFloat already collected now go to Eisel-Lemire directly, in a new std-only module, src/decimal_float.zig, that also holds the Clinger path. std.fmt.parseFloat remains the fallback for the rare cases neither can round.
A float's fraction usually carries most of its digits, so fastFloat now checks for runs of eight digits with one 64-bit load, and combines them into the mantissa with a few multiplies, as simdjson does. The byte loop picks up whatever is left. Integer parts are rarely long enough to be worth the check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Floats outside Clinger's exact range (mantissa ≥ 2^53, e.g. coordinates with 16-17 significant digits) used to go back to
std.fmt.parseFloatas text, which rescanned them with its general parser. On canada.json that was about 20% of decode time.src/decimal_float.zig: a new std-only module,toFloat(T, mantissa, exp10, truncated) ?Tfor f32/f64, which tries Clinger's path and then Eisel-Lemire on the digitsfastFloatalready collected. It returns null for the rare inputs neither can round, andstd.fmt.parseFloatstays the fallback. The power-of-five table is the one from std'sconvert_eisel_lemire.zig, which derives from fast_float. std already has this algorithm, but only behindparseFloat's text parser, so a public "digits to float" API like this one is what could be upstreamed.Benchmarks
i7-6700, median of 5 runs, decode:
On a standalone canada decode binary, cycles per decode drop from 20.3M to 13.1M. The twitter and small changes are code layout on this Skylake. small.json has a single float, and a standalone binary shows +0.5% instructions and the same cycles.
Tests
toFloatagainststd.fmt.parseFloat, bit for bit, on:parseInt, and against every non-digit byte in every position.Passes on Zig 0.16 and 0.17, natively and on s390x, and the JSONTestSuite conformance run passes.