Skip to content

Handle sklearn version difference in ROCAUC scoring#698

Merged
cristian-tamblay merged 1 commit into
developfrom
fix/rocauc-sklearn-version-compat
Jun 15, 2026
Merged

Handle sklearn version difference in ROCAUC scoring#698
cristian-tamblay merged 1 commit into
developfrom
fix/rocauc-sklearn-version-compat

Conversation

@Creylay

@Creylay Creylay commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Older sklearn versions raise ValueError in roc_auc_score when a class is absent from a split during One-vs Rest (OvR) evaluation, while newer versions (e.g. 1.7.2) return NaN with a warning instead. This caused training jobs to crash on machines with older sklearn while working fine on machines with newer sklearn. The fix wraps the roc_auc_score call in a try/except ValueError so both behaviors normalize to float("nan"), which base_model.calculate_metrics already filters out via math.isfinite.


Type of Change

  • Backend change
  • Frontend change
  • CI / Workflow change
  • Build / Packaging change
  • Bug fix
  • Documentation

Changes (by file)

  • DashAI/back/metrics/classification/roc_auc.py: wrap roc_auc_score call in try/except ValueError to return float("nan") instead of propagating the exception when a class is absent from a split in OvR mode.

Testing (optional)

Train a classification model with an imbalanced dataset (no stratified split) so at least one class is absent from validation/test. Confirm training completes and ROCAUC is omitted from those splits instead of crashing the job.


Notes (optional)

This bug only manifests when: (1) the task is multiclass, (2) the dataset is split without stratification, and (3) at least one class has no samples in the validation or test split. The calculate_metrics call happens inside the training loop (per epoch/step), so the crash occurs early during training rather than at the end.

@Creylay Creylay changed the title Handle ValueError in ROCAUC metric calculation to return NaN for inva… Handle sklearn version difference in ROCAUC scoring Jun 11, 2026
@cristian-tamblay cristian-tamblay merged commit ad9363c into develop Jun 15, 2026
34 of 36 checks passed
@cristian-tamblay cristian-tamblay deleted the fix/rocauc-sklearn-version-compat branch June 15, 2026 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants