bin/seed_package_mirror.sh: dynamically seed all boards & toolchains - #2199
Open
tlaurion wants to merge 1 commit into
Open
bin/seed_package_mirror.sh: dynamically seed all boards & toolchains#2199tlaurion wants to merge 1 commit into
tlaurion wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
tlaurion
force-pushed
the
seed-package-mirror-dynamic
branch
from
September 2, 2026 19:18
14ac392 to
1091d46
Compare
Collaborator
Author
|
@JonathonHall-Purism should be future proof and make sure everything currently maintained is mirrored. |
Rewrite bin/seed_package_mirror.sh to dynamically discover and seed every board instead of a hardcoded list of five representative boards. Fixes linuxboot#2177: the Purism mirror only cached tarballs for the previously hardcoded boards, missing the musl-cross-make component tarballs moved under packages/ by PR linuxboot#2174. Changes: - Discover all boards via boards/*/*.config instead of a hardcoded list, so new boards, architectures, and coreboot forks are picked up automatically. - Run 'make BOARD=<board> packages' for each board. The define_module 'packages:' target (Makefile) pre-downloads every versioned module tarball plus coreboot crossgcc toolchain tarballs. - Clean only the cached tarballs (packages/*/*) and .canary build stamps (build/**/.canary, as in the real.remove_canary_files-* Makefile helper) so module tarballs are re-downloaded + hash-verified and coreboot forks re-emit their crossgcc toolchain, without wiping compiled artifacts. - Redirect make stdin from /dev/null to avoid an intermittent 'syntax error near unexpected token do' under set -e + 'if ! make'. - Tolerate per-board failures (dead upstream URLs) without aborting the whole seed, reporting a summary at the end. - Copy tarballs arch-agnostically (packages/*/) instead of hardcoding x86/ppc64. Signed-off-by: Thierry Laurion <insurgo@riseup.net>
tlaurion
force-pushed
the
seed-package-mirror-dynamic
branch
from
September 2, 2026 19:22
1091d46 to
b924d0a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2177
Problem
bin/seed_package_mirror.shseeded the Purism package mirror using ahardcoded list of five representative boards:
qemu-coreboot-fbwhiptail-tpm1-hotp(base x86)UNTESTED_talos-2(PPC)librem_l1um_v2(TPM2)EOL_librem_l1um(coreboot 4.11)EOL_x230-maximized(io386)After #2174 moved the musl-cross-make component tarballs under
packages/, the mirror did not get all of them cached on demand,so builds served those 7 tarballs from the Purism mirror (
MIRROR_USEDin
build/mirror_fallbacks.log) rather than from primary sources.The hardcoded list is also fragile: boards are added/renamed and
coreboot forks change over time, so the mirror was always at risk of
silently missing newly added module tarballs.
Change
Rewrite
bin/seed_package_mirror.shto:boards/*/*.configinstead of ahardcoded list, so new boards, architectures, and coreboot forks are
picked up automatically with no maintenance.
make BOARD=<board> packagesfor every board. Thedefine_modulepackages:target pre-downloads every versioned module tarball plus thecoreboot crossgcc toolchain tarballs, which is what actually gets copied
into the mirror.
packages/*/*) and.canarybuildstamps (
build/**/.canary, modeled on thereal.remove_canary_files-extract_patch_rebuild_what_changedMakefilehelper) so module tarballs are re-downloaded + hash-verified and
coreboot forks re-emit their crossgcc toolchain, without wiping already
compiled artifacts.
makestdin from/dev/nullto avoid an intermittentbash: syntax error near unexpected token 'do'that occurred underset -e+if ! make ...whenmake/wgetconsumed the script's stdin.coreboot 4.11
acpicatarball) without aborting the whole seed,reporting a summary of failed boards at the end.
packages/*/) instead ofhardcoding
x86/ppc64.Verification
musl-cross-make component tarballs that were the core of Make sure Purism cache all tarballs from merged https://github.com/linuxboot/heads/pull/2174 #2177.
build/mirror_fallbacks.logshows the 7 musl-cross-make tarballs are nowfetched from primary sources (
PRIMARY) instead ofMIRROR_USED..canaryre-runs the Makefile branch that re-fetches the pinned commit and
regenerates the
.canary(observed:.canarypresent after the run,fork dirs preserved), yielding the crossgcc toolchain tarballs without a
full re-clone.