Repository navigation
test: add multi-source accuracy fixtures for log1p - #16627
Draft
batpigandme wants to merge 5 commits into
Draft
batpigandme wants to merge 5 commits into
batpigandme wants to merge 5 commits into
Conversation
Every fixture runner reads the same inputs, so the sets from different
sources can be compared value for value against MPFR. The inputs are
drawn per regime, with the bounds of the package's existing Julia
fixtures, from a seeded PRNG, so the script reproduces them byte for
byte. Regions whose magnitudes span more than a factor of ten are drawn
log-uniformly, for negative ranges too, and the files are JSON only.
Assisted-by: Claude Opus 5.5
---
type: pre_commit_static_analysis_report
description: Results of running static analysis checks when committing changes.
report:
- task: lint_filenames
status: passed
- task: lint_editorconfig
status: passed
- task: lint_markdown_pkg_readmes
status: na
- task: lint_markdown_docs
status: na
- task: lint_markdown
status: na
- task: lint_package_json
status: na
- task: lint_repl_help
status: na
- task: lint_javascript_src
status: passed
- task: lint_javascript_cli
status: na
- task: lint_javascript_examples
status: na
- task: lint_javascript_tests
status: na
- task: lint_javascript_benchmarks
status: na
- task: lint_python
status: na
- task: lint_r
status: na
- task: lint_c_src
status: na
- task: lint_c_examples
status: na
- task: lint_c_benchmarks
status: na
- task: lint_c_tests_fixtures
status: na
- task: lint_shell
status: na
- task: lint_typescript_declarations
status: passed
- task: lint_typescript_tests
status: na
- task: lint_license_headers
status: passed
---
Add runners which evaluate `log1p` on the shared inputs with MPFR (the
reference), Julia, Python's `math` and SciPy, R, Boost, Cephes, and
stdlib's own C implementation. Each runner reads the shared JSON inputs,
echoes them, and records in `meta` the library, its version, what it
calls underneath, and the platform, so a scorer can check every set and
say what each column measures.
The Boost runner reads its inputs with Boost.JSON's precise number
parsing. The two C runners read them with `strtod`, as C has no JSON
parser, and stdlib's C runner builds the package's C sources from its
`manifest.json`, so no add-on is needed.
Assisted-by: Claude Opus 5.5
---
type: pre_commit_static_analysis_report
description: Results of running static analysis checks when committing changes.
report:
- task: lint_filenames
status: passed
- task: lint_editorconfig
status: passed
- task: lint_markdown_pkg_readmes
status: na
- task: lint_markdown_docs
status: na
- task: lint_markdown
status: na
- task: lint_package_json
status: na
- task: lint_repl_help
status: na
- task: lint_javascript_src
status: na
- task: lint_javascript_cli
status: na
- task: lint_javascript_examples
status: na
- task: lint_javascript_tests
status: na
- task: lint_javascript_benchmarks
status: na
- task: lint_python
status: passed
- task: lint_r
status: passed
- task: lint_c_src
status: na
- task: lint_c_examples
status: na
- task: lint_c_benchmarks
status: na
- task: lint_c_tests_fixtures
status: passed
- task: lint_shell
status: na
- task: lint_typescript_declarations
status: passed
- task: lint_typescript_tests
status: na
- task: lint_license_headers
status: passed
---
Score every fixture set on the shared inputs against MPFR, by regime, and
write a summary which records each source's version and what it calls
underneath. The scorer first checks that every set echoes the shared
inputs bit for bit and names the input file it read, so a misread input
fails loudly instead of skewing a score.
Assisted-by: Claude Opus 5.5
---
type: pre_commit_static_analysis_report
description: Results of running static analysis checks when committing changes.
report:
- task: lint_filenames
status: passed
- task: lint_editorconfig
status: passed
- task: lint_markdown_pkg_readmes
status: na
- task: lint_markdown_docs
status: na
- task: lint_markdown
status: na
- task: lint_package_json
status: na
- task: lint_repl_help
status: na
- task: lint_javascript_src
status: passed
- task: lint_javascript_cli
status: na
- task: lint_javascript_examples
status: na
- task: lint_javascript_tests
status: na
- task: lint_javascript_benchmarks
status: na
- task: lint_python
status: na
- task: lint_r
status: na
- task: lint_c_src
status: na
- task: lint_c_examples
status: na
- task: lint_c_benchmarks
status: na
- task: lint_c_tests_fixtures
status: na
- task: lint_shell
status: na
- task: lint_typescript_declarations
status: passed
- task: lint_typescript_tests
status: na
- task: lint_license_headers
status: passed
---
Add the fixture sets each runner generates on the shared inputs, from
MPFR, Julia, Python's `math` and SciPy, R, Boost, Cephes, and stdlib's
own C. All were generated in one session on x86_64 Linux with glibc
2.39, so every value in a package's tables comes from one platform, and
each set's `meta` records the library, its version, what it calls
underneath, and the platform. Every set echoes the shared inputs bit for
bit.
Assisted-by: Claude Opus 5.5
---
type: pre_commit_static_analysis_report
description: Results of running static analysis checks when committing changes.
report:
- task: lint_filenames
status: passed
- task: lint_editorconfig
status: passed
- task: lint_markdown_pkg_readmes
status: na
- task: lint_markdown_docs
status: na
- task: lint_markdown
status: na
- task: lint_package_json
status: na
- task: lint_repl_help
status: na
- task: lint_javascript_src
status: na
- task: lint_javascript_cli
status: na
- task: lint_javascript_examples
status: na
- task: lint_javascript_tests
status: na
- task: lint_javascript_benchmarks
status: na
- task: lint_python
status: na
- task: lint_r
status: na
- task: lint_c_src
status: na
- task: lint_c_examples
status: na
- task: lint_c_benchmarks
status: na
- task: lint_c_tests_fixtures
status: na
- task: lint_shell
status: na
- task: lint_typescript_declarations
status: passed
- task: lint_typescript_tests
status: na
- task: lint_license_headers
status: passed
---
Test the implementation against MPFR on the shared inputs, so its bounds
reflect its own error rather than a reference's. The existing tests
against Julia's fixtures move unchanged to `test.julia.js`, and
`test.scipy.js` tests against SciPy's set. Each regime's bound is the
tightest which passes: exact where every value matches, otherwise 1 ULP,
or 2 against SciPy where SciPy itself is off by 2.
Assisted-by: Claude Opus 5.5
---
type: pre_commit_static_analysis_report
description: Results of running static analysis checks when committing changes.
report:
- task: lint_filenames
status: passed
- task: lint_editorconfig
status: passed
- task: lint_markdown_pkg_readmes
status: na
- task: lint_markdown_docs
status: na
- task: lint_markdown
status: na
- task: lint_package_json
status: na
- task: lint_repl_help
status: na
- task: lint_javascript_src
status: na
- task: lint_javascript_cli
status: na
- task: lint_javascript_examples
status: na
- task: lint_javascript_tests
status: passed
- task: lint_javascript_benchmarks
status: na
- task: lint_python
status: na
- task: lint_r
status: na
- task: lint_c_src
status: na
- task: lint_c_examples
status: na
- task: lint_c_benchmarks
status: na
- task: lint_c_tests_fixtures
status: na
- task: lint_shell
status: na
- task: lint_typescript_declarations
status: passed
- task: lint_typescript_tests
status: na
- task: lint_license_headers
status: passed
---
Contributor
Coverage Report
The above coverage report was generated for the changes in this PR. |
math/base/special/log1plog1p
Member
Author
|
"Lint Copyright Years" failure is expected. It flags Followed precedence from other moved files (e.g., #13218 and #12964), which failed this check the same way when code moved into The warnings for the fixture JSON files are only "no copyright notice found," since JSON fixtures don't carry one. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
type: pre_commit_static_analysis_report
description: Results of running static analysis checks when committing changes.
report:
status: passed
status: passed
status: na
status: na
status: na
status: na
status: na
status: passed
status: na
status: na
status: na
status: na
status: na
status: na
status: na
status: na
status: na
status: na
status: na
status: passed
status: na
status: passed
Description
This pull request:
math/base/special/log1p(10 regimes of 500 points each, plus 19 edge cases), drawn byscripts/inputs.jsfrom a seeded PRNG so they reproduce byte for byte.mathand SciPy, R, Boost, Cephes, and stdlib's own C. Each set'smetarecords the library, its version, what it calls underneath, and the platform.scripts/accuracy.js, which checks that every set echoes the shared inputs bit for bit, then scores each source against MPFR by regime.test.js, moves the existing Julia tests unchanged totest.julia.js, and addstest.scipy.js.Scores against MPFR on the 5,000 regime points:
math, R, Boost (glibc)Related Issues
This pull request has the following related issues:
math/base/specialpackages from relative tolerance testing to ULP difference testing (tracking issue) #11352 (bounds follow its "minimum required ULP" rule)Reference Packages
This pull request followed these packages:
math/base/special/fresnel(Cephes runner),math/base/special/digamma(non-finite values stored as strings)test/fixtures/<source>/, and calling Cephes directly.c/stdlib/builds the package's C from itsmanifest.json, ascompile_c_benchmarkdoes for benchmarks; no fixture runner did this before.strtod, checked exact on 205,019 values on macOS and x86_64 Linux.Questions
"Infinity"and"NaN";digammauses"Inf". Which spelling should be standard?scripts/accuracy.jsstay per package until a few packages use it, or move to a shared tool now?Other
test.js.test.scipy.jscompares against SciPy, so it reaches 2 where SciPy is 2 ULP off. The bounds describe these fixtures, not the function: sampled at 10^6 points per regime, stdlib also reaches 1 ULP insmall_negativeandsmall_positive(about 0.02% of points), where all 500 fixture points are exact.lib/main.js,src/main.c) doesn't hold. That's left for its own fix.test.native.jsis unchanged.Checklist
AI Assistance
If you answered "yes" above, how did you use AI assistance?
Disclosure
The runners, scorer, and tests were written by Claude Code (Opus 5.5), and the fixtures generated in a Claude Code session on x86_64 Linux. I set the design, reviewed every file, and checked the scores against independent reruns. This follows (to the best of my ability) the layout for "accuracy-comparison" tables across languages (AKA the accuracy-visibility project) as discussed with @kgryte.
@stdlib-js/reviewers