Implement registry version check script

[thomashoneyman]

Closes #426 by adding a .#version-check script to report registry versions newer than the versions in the latest package set. This reuses the package set updater candidate traversal instead of being brand-new.

I ran the script against package set 77.13.1 and found 19 pending versions. Not all of them can compile together, but 14 of them do, and that's submitted as purescript/registry#560.

Five versions were intentionally left out:

• barlow-lens@1.0.0 breaks morello@0.4.0, which uses the old Barlow API and declares <0.10.0.
• elmish@0.14.0 breaks elmish-hooks@0.11.0; meanwhile, elmish-html@0.12.0 requires Elmish 0.14, so neither Elmish candidate can move until the dependent packages are compatible.
• hyrule@2.4.0 removes modules still imported by bolson@0.3.9, deku@0.9.24, and ocarina@1.5.4.
• node-child-process@12.0.0 changes the exit API and breaks dotenv@4.0.3 and node-execa@5.0.0.

So this is the point of the script: surface upgrades that automatic daily updates could not accept so maintainers can identify and submit a compatible coordinated batch.

[f-f]

This is cool - two things come to mind:

• we should have a policy of dropping packages that are holding on updates for too long. E.g. picture the scenario where a package that has many dependants is blocked by a package that has not many users - the former should not be held up. Maybe we can have some sort of expiration (e.g. holding a package for 30 days triggers removal or something like that)
• we could wire this up to @pacchettibotti automatically opening issues in the repos that need updating to the newer version, to speed up the process of the package sets catching up to things

[thomashoneyman]

Yeah, I agree. I don't have a specific policy in mind but directionally I'm on board. To what degree should that be encoded into this script's report? We could augment this with more data than it currently gives.

[thomashoneyman]

Also, we could extend this with a planner that makes it more obvious whether the discovered versions will even work together and suggest a batch. Something like:

1. Give every candidate package two choices: its current version or reported candidate.
2. Check every package manifest against the exact proposed package-set versions.
3. Maximize the number of accepted candidates while satisfying all declared ranges.
4. Repeat until stable, since rejecting one candidate can invalidate another; like for example, rejecting Elmish 0.14 also requires rejecting elmish-html@0.12.0.
5. Run one read-only atomic compilation of the selected plan to prove it

For this batch, this would have discovered the 14 packages that work without me having to do a separate manual triage:

• morello excludes Barlow 1.0.
• elmish-hooks excludes Elmish 0.14, which consequently excludes the new elmish-html.
• bolson and deku exclude Hyrule 2.4.
• dotenv and node-execa exclude node-child-process 12.

...this doesn't help with what you're describing, but maybe we can re-think this script such that it's a more useful swiss-army-knife for helping get the package sets updated with the best options when it can't be done automatically due to breaking changes.

[f-f]

To what degree should that be encoded into this script's report?

I to get us a good glance we could have:

• who is holding up who
• and for how long; that is: how many days ago the incompatible version was published
• how many dependants each of the packages involved have

Also I think we could be running this in a weekly cronjob thing in the Registry repo, opening an issue/PR if it finds something

[thomashoneyman]

d283822 has some followups that address your points:

1. The version check now produces JSON and markdown reports, which themselves cover staleness, direct and transitive dependant counts, blocked version ranges, reverse blockers, compiler compatibility, etc.
2. It also includes a branch-and-bound planner which uses our bounded ranges; it considers current and compatible newer releases, maximizes upgrade count and then freshness, and supports coordinated upgrades
3. You can optionally compile the planned payload with --verify so that you know the proposed plan will work
4. Adds a weekly workflow which writes to a rolling issue with that week's report (doesn't actually submit anything)

Tried it out with a live run against package set 78.1.0 / compiler 0.15.15 and it produced a workable plan — including settling on hyrule@2.3.9 as an ugrade since 2.4.0 is blocked.

[thomashoneyman]

Latest commit is a prototype, still hacking on it a little. But as I was working on the version check script, I realized we could be doing a lot more automatically — and still safely — to update the package sets. Our current automatic upgrade is basically just "try all new packages in the last 24hrs, and if any fail, go sequentially and only include ones which succeed." Very naive, and if a package doesn't work in its 24h window for whatever reason it is never retried — even if it would work later on (such as a dependency finally upgrading).

So the new prototype implements one shared planner that can be used by both the package set updater and the version check workflow. Any version of a package that's already in the set which is newer than what's in the set is eligible for any new plan, plus recently uploaded packages even if they've never been in the set. We restrict to a single compiler version but otherwise ignore ranges. We still try all packages first, but then we do bounded best-first latest/intermediate/current searches.

The package set updater, if it is capable of finding a nonempty addition/upgrade plan, can just submit it. If it finds removals, downgrades, etc. then we can have that notify trustees for action, but it will never be automatically done.