Build a real deployed simulator definition on the container, and survey the associated repositories - #170
Merged
Conversation
`test_build_simulator` moves off `cluster_only`, leaving seven cluster-only tests. It builds a definition copied verbatim from the live deployment rather than one generated from pbest's current defaults, so `singularity build --fakeroot` is exercised against content a deployment really produced. The definition is simulator version 34 from `https://compose.cam.uchc.edu/core/simulator/list`, saved as `tests/fixtures/resources/production_simulator.def` with a README recording where it came from and how to refresh it. Of the 37 versions published there it is the smallest: no conda and no PyPI dependencies, so it clones and installs a single package instead of solving a conda environment. Measured on the containerised cluster it builds in about 70 seconds to a 594 MB image, and the result runs with `bsew` importable inside it. The newest definitions build the full copasi/tellurium/readdy stack, which is minutes and 1.3 GB and does not belong on every pull request. The test asserts the image is produced *and executes*, not merely that the job exited zero. The package introspection the old version did is dropped: it exercised pbest's default dependency set rather than the build, and tied the test to a pin that changes. Two hazards surfaced while doing it. The pre-commit `end-of-file-fixer` appended a newline to the definition, which changed its md5 and so its identity, silently turning a verbatim copy into something that merely resembled one. `tests/fixtures/resources/` is excluded from the whitespace hooks now, and the test asserts the hash still equals the value the deployment recorded, so a future edit fails loudly instead. And asserting the recorded status immediately after the scheduler reports the job done is a race, because the monitor writes on its own schedule. That is the same mistake as F13's test made again in a new place, and it now polls. **This is the first test with a real CI cost**: the suite goes from about 91 seconds to about 279. It buys the first coverage anywhere of the fakeroot build path against real content. If that becomes a problem the cheapest lever is moving this one test to a scheduled job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018TgVbCXXWAWhNjtKTzrxvN
…roduction Opens `docs/ecosystem-repos.md`, read-only, checked against GitHub and PyPI on 2026-09-12. Nothing was changed in any other repository. **bsew and bsander.** Both live in `biosimulators`, not vivarium-collective. Neither is on PyPI: both return 404. Neither had been documented here, and both already reach this repository. The sharper finding is that `bsew` is installed into a *deployed* simulator image by a bare `git clone`, with no branch, tag or commit, so the image depends on whatever `main` is at build time. That is the reproducibility problem from item 6 of the runtime plan, except concrete and in production rather than hypothetical. `bsander` reaches us only through the dead uv.sources override already recorded as F14. Both are load-bearing in the grant narrative -- bsander under A2.2 and A3.3a, bsew under A3.2.engines and in DEVIATIONS -- while being unreleased and idle for nine and twelve months respectively. **The wrappers are no longer called pbg-.** The organisation completed a `pbg-` to `viva-` rename: 54 `viva-` repositories against a single surviving `pbg-`. Anything still looking for `pbg-xxx` finds almost nothing, and cross-references inside the org have not all caught up. **An index already exists**, generated nightly by `viva-catalog` from a GitHub topic, so this document deliberately does not restate the list -- a second hand-maintained index would be stale in weeks. What it records instead is how to read that index and where it does not reach: 7 repositories are absent from it, all of them tooling rather than simulator content, so it answers "what can I compose with" rather than "what does this ecosystem consist of". **The consolidation question is framed, not answered.** The evidence so far points away from the wrappers, which have separate upstreams and cadences and are already addressable, and toward the tooling layer: three paths delegate to process_bigraph.Composite, four partial static checkers exist, and three ways to containerise a composite. Four questions are listed for a decision that is not mine to make. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018TgVbCXXWAWhNjtKTzrxvN
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things.
test_build_simulatormoves offcluster_only, and a read-only survey of the associated repositories is added.1. The build test runs on the container
Leaves seven cluster-only tests. It builds a definition copied verbatim from the live deployment, so
singularity build --fakerootis exercised against content a deployment really produced rather than something a test invented. It asserts the image is produced and executes, not merely that the job exited zero.The fixture is simulator version 34 from
https://compose.cam.uchc.edu/core/simulator/list, saved with a README recording its provenance and how to refresh it. Of the 37 versions published there it is the smallest.The package introspection the old version did is dropped: it exercised pbest's default dependency set rather than the build, and tied the test to a pin that changes.
This one costs real CI time, and that is a decision for you
Four jobs, because CI still runs twice per push. It buys the first coverage anywhere of the fakeroot build path against real content. If that is too much, the cheapest lever is moving this single test to a scheduled job; the second cheapest is fixing the duplicate-CI trigger (finding F4).
Two hazards it surfaced
The formatter silently invalidated the fixture. Pre-commit's
end-of-file-fixerappended a newline, changing its md5 and so its identity, turning a verbatim copy into something that merely resembled one.tests/fixtures/resources/is excluded from the whitespace hooks now, and the test asserts the hash still equals the value the deployment recorded.The same race as F13, in a new place. Asserting the recorded status immediately after the scheduler reports the job done depends on the monitor having polled. It now waits.
2.
docs/ecosystem-repos.mdRead-only survey, checked against GitHub and PyPI on 2026-09-12. Nothing was changed in any other repository.
bsewandbsanderlive inbiosimulators, not vivarium-collective. Neither is on PyPI; both return 404. Neither was documented here, and both already reach this repository. The sharper finding:bsewis installed into a deployed simulator image by a baregit clonewith no branch, tag or commit, so the image depends on whatevermainis at build time. That is item 6 of the runtime plan, concrete and in production rather than hypothetical.bsanderreaches us only through the dead uv.sources override already recorded as F14. Both are load-bearing in the grant narrative while being unreleased and idle for nine and twelve months.The wrappers are no longer called
pbg-. The organisation completed apbg-toviva-rename: 54viva-repositories against a single survivingpbg-. Anything still looking forpbg-xxxfinds almost nothing.An index already exists, generated nightly by
viva-catalogfrom a GitHub topic, so the document deliberately does not restate the list — a second hand-maintained index would be stale in weeks. It records how to read that index and where it does not reach: 7 repositories are absent, all tooling rather than simulator content, so it answers "what can I compose with" rather than "what does this ecosystem consist of".The consolidation question is framed, not answered. Evidence points away from the wrappers, which have separate upstreams and cadences and are already addressable, and toward the tooling layer: three paths delegate to
process_bigraph.Composite, four partial static checkers exist, and three ways to containerise a composite. Four questions are listed for a decision that is not mine to make.Verification
bsewimportable inside it.make check.🤖 Generated with Claude Code
https://claude.ai/code/session_018TgVbCXXWAWhNjtKTzrxvN