Bug 2024208: Add a column to denote a magnitude of change based on the modality analysis

[kala-moz]

The Median Diff (%) column has been replaced with a Mode Δ (%) column that reports the largest matched-pair peak shift between the Base and New distributions, derived from the same KDE + mode-detection + modality-matching pipeline that powers the chart and the expanded-row blurb.

Like bootstrapCi, the value is precomputed once in the four data loaders (loader.ts, overTimeLoader.ts, subtestsLoader.ts, subtestsOverTimeLoader.tsx) via precomputeLargestPeakShift and attached to result.modeDeltaPct. KDE + mode matching is expensive enough that running it on every sort comparison or render would tank interactions.

Speedometer 3 preview link

I had some trouble with netlify thus all the open / reopen commands. :/

Update: The Mode Δ (%) column and the Distribution Interpretation row in MannWhitneyCompareMetrics were reading mode counts from two independent sources: the client-side ISJ/SJ KDE pipeline vs. the backend's wider Silverman KDE. Therefore, could could disagree in jarring ways (e.g. "Mode Δ = 4.16%" sitting directly above "No modes or data, possible oversmoothing"). Now, both views are piped through a single precomputed analysis so they can't drift.

Update 2: I precompute modality on replicates when available

Update 3: Flip the sortFunction to ASC semantics so the framework's swap produces the intuitive direction: DESC → biggest improvements first, ASC → biggest regressions first. Comment updated to spell this out (same convention the Sig column already uses).

Update 4: Lazy-precompute Mode Δ analysis to restore load speed. All four loaders no longer call precomputeModalityAnalysis

Update 5: Select by peak magnitude vs largest shift. Also, simplified the language of the tooltip for the column.

[netlify]

✅ Deploy Preview for mozilla-perfcompare ready!

Name	Link
🔨 Latest commit	d27d39a
🔍 Latest deploy log	https://app.netlify.com/projects/mozilla-perfcompare/deploys/6a602f9cc14bc4000766b40b
😎 Deploy Preview	https://deploy-preview-1059--mozilla-perfcompare.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[padenot]

Similarly to #1056 (comment) I only see - ?

[kala-moz]

Similarly to #1056 (comment) I only see - ?

I've decided to also remove the dash since it doesn't give enough information. I've switched to NM which mean No Modes. I've updated the tooltip with more information.

Ahhh, I didn't use replicates. That's were the bug was. Updated subtests preview

Mozilla central preview

[padenot]

There's quite a lot of room in that column, looking at your screenshot, we can say the words in full? Similarly for NM

[padenot]

I was browsing around the deploy link you helpfully provided, and I found:

Here, we have an example of a test that's a bit noisy. We see the main peak doesn't move, but we report a few percent shift. Two thoughts:

• If we go this route, we should take the two peaks of the highest magnitude probably?
• The raw MWU more or less does the right thing already, but is susceptible to outliers

What was the intent behind this? Maybe I'm not looking at cases that greatly benefit from this?

[kala-moz]

There's quite a lot of room in that column, looking at your screenshot, we can say the words in full? Similarly for NM

1.) Say what words in full?
2.) Okay, I see we do have space to write No Modes vs NM.

[kala-moz]

I was browsing around the deploy link you helpfully provided, and I found:

Here, we have an example of a test that's a bit noisy. We see the main peak doesn't move, but we report a few percent shift. Two thoughts:

* If we go this route, we should take the two peaks of the highest magnitude probably?

* The raw MWU more or less does the right thing already, but is susceptible to outliers

What was the intent behind this? Maybe I'm not looking at cases that greatly benefit from this?

@padenot Hmm. I see what you mean. I can switch it to select by peak magnitude. The current selection by shift size was a suggestion from sparky in Bug 2024208 so not set in stone. We wanted to have a magnitude of change that didn't rely on mean / median alone but took advantage of the modality work. But using peak magnitude makes much more sense.

Updated deploy link