test(e2e): make the suite independent of the network and the runner - #959
Merged
Merged
Conversation
✅ Deploy Preview for volview-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
PaulHax
force-pushed
the
e2e-hermetic-ci
branch
from
September 21, 2026 18:01
9feb612 to
7857f83
Compare
Specs and the two 5.0.1 session fixtures fetched from data.kitware.com while a test timer was running, so a slow host failed whichever spec happened to be waiting. All datasets now live in tests/datasets.ts, are downloaded once in onPrepare, and are read from the local server. A failed download aborts the run and names the dataset. CI caches the downloads.
Every page load fetched Roboto from Google Fonts and an unpinned @mdi/font@latest from jsdelivr, both render blocking. An offline or firewalled deployment lost its icons, and the e2e screenshots depended on two CDNs.
Allocating the 1024x1024x256 float volume texture fails with GL_OUT_OF_MEMORY on software GL. The volume mapper then fails to compile its shader with an empty log and the context is lost, which the spec never noticed. The spec covers the wasm heap, not rendering.
Mocha root hooks fail any test during which the page requested a non-local URL or logged a shader compile failure, a lost context or a vtk setContext null access. The sample spec and the two paint specs read the cached prostate archive. Console mirroring moves here so it survives reloadSession.
A wait that carries its own time limit stands in for a state the app should publish, and it is what fails on a slow runner. The 27 files that still have one are listed as debt, so the list can only shrink and a new spec has to wait on page state.
The datasets came from three hosts. They are now assets of one GitHub release, so the only host CI depends on for data is the one it already runs on. A release asset can be replaced, so each file carries a sha256 that is checked on download and against the cache. A mismatch aborts the run before any spec starts.
The labelmap direction spec wrote its config by hand next to a call to writeManifestToFile. The interactor lifecycle spec subscribed to console entries that the root hooks already subscribe every session to. A dataset served from /tmp is named by its url, so openConfigAndDataset no longer repeats the names.
A cache saved by a pull request run is only visible to that pull request, and the e2e workflow only runs on pull requests, so the first run of every PR downloaded all the datasets. A job on main now saves the cache where every PR can restore it. The download moves to a module that needs only node built-ins, so the job runs it with node and skips installing dependencies.
PaulHax
force-pushed
the
e2e-hermetic-ci
branch
from
September 21, 2026 18:05
7857f83 to
9755f79
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An e2e failure should mean the app is wrong. It could also mean a data host was slow or a wait ran out of time, and a WebGL failure could pass unnoticed in one spec while the same kind of failure broke an unrelated one. This removes the network dependency, makes WebGL failures visible where they happen, and stops per-call timeouts from growing.
Test data
tests/datasets.tsand downloaded before any spec or test timer starts. Specs and the two 5.0.1 session fixtures read them from the local server at/tmp/<name>. They are the same real files as before, including the rotated direction matrix prostate series.test-data-1release on this repository, so the only host CI depends on for data is the one it already runs on. Each file has a sha256 that is checked on download and against the cache. One attempt, and a failure aborts the run naming the dataset.mainkeeps one where every PR can restore it. It runs the download module with node and does not install dependencies.Guards (
tests/rootHooks.ts, mocha root hooks)setContextnull access.browser.mock, even one that matches nothing, stalls the app's data loading in Chrome. The sample data spec swaps its one address withaddInitScriptinstead.Fonts (the one change to the app)
index.htmlloaded Roboto from Google Fonts and an unpinned@mdi/font@latestfrom jsdelivr, both render blocking. An offline or firewalled deployment lost its icons. They are now bundled. Font files are not inlined, so unicode-range subsets stay lazy, and only the woff2 icon font is built. A page load fetches four font files.Shader compile failure in
session-large-uri-baseGL_OUT_OF_MEMORYon software GL. The volume mapper then fails to compile its shader with an empty log, and the context is lost afterwards. The spec passed anyway because it only looked for aRangeError. It covers the wasm heap, so it now loads with an axial-only layout.Timeouts
timeout:property orbrowser.pausein specs and page objects. The files that still have one are listed as debt ineslint.config.js, so the list can only shrink. Replacing them with waits on state the app publishes is follow-up work.