You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
HausdorffDistanceMetric percentile returns NaN for a missed prediction, dropping failures from the dataset average #9095
HausdorffDistanceMetric(percentile=...) returns nan when one of the two masks is empty, where percentile=None returns inf for the same input.
nan is not a quieter way of saying inf here. It is this metric's "not applicable" sentinel — it is what both-masks-empty returns — and do_metric_reduction excludes it from the average. A prediction that missed the structure entirely is therefore removed from a dataset score rather than counted as the worst case, and the reported HD95 improves as the model finds fewer structures.
The effect over a dataset. 100 images, every non-empty prediction identically 4px off, so the only variable is how many images the model misses:
images missed
reported HD95
cases averaged (get_not_nans)
0
4.000
100
50
4.000
50
90
4.000
10
99
4.000
1
A model that finds nothing in 99 images out of 100 reports the same HD95 as one that finds the structure every time. A runnable script that prints the whole table is at https://github.com/asifuddin01/A-PR.
Expected behavior
inf, matching percentile=None, so the case is scored rather than discarded. nan stays reserved for both masks empty, where there is genuinely nothing to measure.
Cause
get_surface_distance reports an infinite distance for every boundary voxel when a mask is empty, so an all-infinite tensor reaches _compute_percentile_hausdorff_distance. torch.quantile interpolates linearly between the two order statistics straddling the requested rank:
inf + (inf - inf) * 0.95 -> nan
The quantile of a constant sequence is that constant, so the nan is an artefact of the interpolation rather than a property of the distances. percentile=None escapes it because .max() does not interpolate.
Only the percentile path is affected. SurfaceDistanceMetric returns inf for the same input and SurfaceDiceMetric returns 0.0; both are already honest about a miss.
Separately, #9033 fixes percentile=0 being treated as unset by the if not percentile: guard. It does not cover this: it adds a .min() branch above the torch.quantile call and leaves the interpolation as it is. The two are independent.
I have a fix and regression tests ready and will open a PR against this issue.
Activity
fix(metrics): score a missed prediction in Hausdorff percentile, don'…