Skip to content

Add support for paramsets - #38

Merged
rcannood merged 1 commit into
mainfrom
paramsets
Sep 22, 2026
Merged

rcannood merged 1 commit into
mainfrom
paramsets

Conversation

@rcannood

Copy link
Copy Markdown
Member

Describe your changes

Port of openproblems-bio/task_template#23: run_benchmark runs parameterised methods once per named paramset, from the method's info.variants or a --paramsets yaml file, tags the scores with paramset_name/paramset, and lets --methods_include/--methods_exclude target <method_id>.<paramset_name>. Uses the paramset helpers from common_resources #41 (submodule bumped).

The existing info.variants in this repo predate this and needed cleaning up, since a method with variants now runs only those variants and viash drops empty ones:

  • Removed the *_scran variants (knn, logistic_regression, mlp, naive_bayes, xgboost): they only override preferred_normalization, which is not an argument, for a normalisation this task doesn't produce. These methods now simply run with their defaults.
  • Removed the single empty variant of scanvi_scarches, scgpt_finetuned, scgpt_zeroshot, seurat_transferdata and singler (a no-op).
  • Renamed the remaining variants to short names, as the run id becomes <method>.<paramset>: scprint.{large,medium,small}, scanvi.{all_genes,hvg}, cellmapper_scvi.{hnoca_hvg,gauss_hvg,gauss_all_genes}, cellmapper_linear.{pca,cca}.
  • scanvi's all_genes variant is spelled out as num_hvg: 0 (the script already treats a falsy value as "all genes"), since an empty variant is dropped by viash.
  • cellmapper_linear: the cca variant used use_rep: None, which YAML reads as the string "None" and would make cellmapper look up obsm["None"]. --use_rep is now optional without a default and the variant simply leaves it out, so cellmapper falls back to CCA. Note this changes the component's standalone default from PCA to the CCA fallback.

Tested locally on the test resources: with the config variants, cellmapper_linear runs pca (use_rep=X_pca) and cca (no use_rep, fast_cca fallback) per dataset and knn runs once untagged; with a --paramsets file for mlp and methods_include: [knn, mlp.small], only mlp.small runs with the given arguments. Score rows carry paramset_name/paramset in both cases.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

@rcannood
rcannood merged commit f37b604 into main Sep 22, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant