Keep the sparse Jensen-Shannon post-processing in the input precision - #2535
Open
maxwbuckley wants to merge 1 commit into
Open
Keep the sparse Jensen-Shannon post-processing in the input precision#2535maxwbuckley wants to merge 1 commit into
maxwbuckley wants to merge 1 commit into
Conversation
The final step of the sparse Jensen-Shannon distance is
[=] __device__(value_t input) { return raft::sqrt(0.5 * input); }
`0.5` is a `double`, so for `value_t == float` the multiply promotes the
expression and `raft::sqrt` resolves to the fp64 overload. Unlike a stray
`double` in an add or multiply, an fp64 sqrt is not one instruction: it is a
Newton-Raphson refinement sequence. The generated SASS for the float
instantiation of the map kernel was
DMUL R14, R4, 0.5
MUFU.RSQ64H R13, R15
DMUL R16, R12, R12
DFMA R16, R14, -R16, 1
... 6 more DMUL/DFMA
ten fp64 instructions per element, which run at 1/64 the fp32 rate on
consumer parts. Writing the constant as `value_t(0.5)` picks the float
overload and reduces this to `FMUL` + `MUFU.RSQ` + four float ops.
`0.5` is exactly representable in binary, so the `double` instantiation is
bit-for-bit unchanged. The `float` instantiation now rounds once instead of
twice, so results move by at most an ulp.
Measured on an RTX 5090 (sm_120a, CUDA 13.2), random CSR inputs, median of 5:
rows x cols, nnz/row map kernel full pairwise_distance
4096 x 4096, 32 263 -> 25 us 6.44 -> 6.20 ms 1.04x
8192 x 4096, 32 1046 -> 319 us 25.31 -> 24.57 ms 1.03x
16384 x 4096, 8 4167 -> 1395 us 29.21 -> 26.42 ms 1.11x
16384 x 16384, 32 4169 -> 1394 us 130.30 -> 127.53 ms 1.02x
The map kernel itself is 3.0-10.4x faster. At 16384 rows it now moves
2.1 GB in 1.39 ms, i.e. it has gone from fp64-throughput-bound to sitting at
the memory roofline; the 4096 case is faster still because its output fits in
L2. End-to-end gains are smaller because the balanced COO SpMV that produces
the input dominates the call.
Output checksums are unchanged to six decimal places across all four shapes.
Contributor
Author
|
@lowener thank you :) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The final step of the sparse Jensen-Shannon distance is a
raft::linalg::mapover the wholea_nrows x b_nrowsoutput:0.5is adouble, so forvalue_t == floatthe multiply promotes the expression andraft::sqrtresolves to the fp64 overload.Unlike a stray
doublein an add or a multiply, an fp64sqrtis not one instruction — it is a Newton-Raphson refinement sequence. The SASS emitted for thefloatinstantiation of the map kernel was:Ten fp64 instructions per output element, each running at 1/64 the fp32 rate on consumer parts.
Fix
Write the constant as
value_t(0.5)so overload resolution picks the float path. The same region becomes:Numerics
0.5is exactly representable in binary, so thedoubleinstantiation is bit-for-bit unchanged. Thefloatinstantiation now rounds once instead of twice, so individual results move by at most an ulp.Measurements
RTX 5090 (sm_120a), CUDA 13.2, random CSR inputs, median of 5 runs,
libcuvs.soswapped between the two builds and interleaved:pairwise_distanceOutput checksums (mean over the full distance matrix) are identical to six decimal places in all four shapes.
At 16384 rows the map kernel now moves 2.1 GB in 1.39 ms, i.e. it sits at the memory roofline rather than being fp64-throughput-bound — which is why it stops at 3x. The 4096 case reaches 10.4x because its output fits in L2.
End-to-end gains are smaller because the balanced COO SpMV that produces the map's input is 85-95% of the call.
Same class of issue as #2531.