Skip to content

[Repo Assist] feat: add explicit-thresholds overloads to multiLabelThresholdMap and calculateMultiLabelROC#380

Draft
github-actions[bot] wants to merge 2 commits intodeveloperfrom
repo-assist/improve-issue-198-multilabel-thresholds-20260424-dabf9617a37643ae
Draft

[Repo Assist] feat: add explicit-thresholds overloads to multiLabelThresholdMap and calculateMultiLabelROC#380
github-actions[bot] wants to merge 2 commits intodeveloperfrom
repo-assist/improve-issue-198-multilabel-thresholds-20260424-dabf9617a37643ae

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

🤖 This is an automated pull request from Repo Assist, an AI assistant.

Closes #198 (partial — the explicit-thresholds items).

Summary

Adds overloads that accept a caller-supplied thresholds: float [] to:

  • ComparisonMetrics.multiLabelThresholdMap
  • ComparisonMetrics.calculateMultiLabelROC

Previously the only option was the zero-argument form that computes BCMs for every distinct prediction value across all labels. For large datasets this is expensive — you may have tens of thousands of distinct prediction scores but only need a coarse 10-point ROC grid.

// fast preview with 5 thresholds instead of all distinct values
let roc =
    ComparisonMetrics.calculateMultiLabelROC(
        actual      = actualLabels,
        predictions = perLabelPredictions,
        thresholds  = [|0.9; 0.7; 0.5; 0.3; 0.1|]
    )
```

## Root cause / design

The no-argument form collects `Array.concat (Array.map snd predictions) |> Array.distinct |> Array.sortDescending` and passes the result to `BinaryConfusionMatrix.thresholdMap`.  When the caller provides their own threshold list they can skip that O(n) collection step and evaluate only the points they care about.

The existing no-argument overloads are **refactored to delegate** to the new ones, eliminating the previously duplicated internal logic.

## Trade-offs

- **Caller responsibility**: when an explicit threshold list is provided the user must ensure the values are sensible (sorted descending is not required  `BinaryConfusionMatrix.thresholdMap` handles any order, but descending is conventional for ROC curves).
- **Prefix threshold**: the output still includes one entry above the global max (`max(all predictions) + 1`) so the first point on the ROC curve always represents the all-negatives classifier.

## Additional fixes in this PR

- Fixed unmatched parenthesis in the original `calculateMultiLabelROC` body (no behaviour change, just compilation cleanliness).
- Renamed unused `k` binding in lambda to `_k`.

## Test Status

5 new test cases added to `Testing.fs`:

```
multi-label threshold map with explicit thresholds
   explicit thresholds: result length
   A: explicit threshold 0-9
   B: explicit threshold 0-5
   C: explicit threshold 0-1
   micro-average present
   macro-average present

Full suite: 1199/1199 passed on .NET 8.

Generated by 🌈 Repo Assist, see workflow run. Learn more.

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@96b9d4c39aa22359c0b38265927eadb31dcf4e2a

… calculateMultiLabelROC

Closes #198 (partial).

The no-argument overloads were the only option previously, which meant every
call computed BCMs for all distinct prediction values. For large datasets this
can be very slow.  New overloads accept a caller-supplied float[] so users can
pass a coarse grid (e.g. [|0.9;0.5;0.1|]) for a quick ROC preview.

Refactored the no-argument overloads to delegate to the new ones, eliminating
duplicate logic.

Also fixes a cosmetic unmatched-paren issue in the original calculateMultiLabelROC
and renames the unused map-key lambda binding to _k.

Tests: 5 new test cases; 1199/1199 pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] QoL improvements for Comparison metrics

0 participants