Repository navigation
test(keystone): add repeatable native browser smoke coverage - #41
Conversation
There was a problem hiding this comment.
No actionable findings in PR41 at d34b599715a6b448b9a945f2040c87194e735b0a, against base 72c28b9240e5b92bc71171e906083b34b0bf2eb9.
Reviewed all 12 changed files across eight complementary assignments: four GPT-6.1-Sol and four Claude Opus 5.5 reviewers covering launcher/process lifecycle, browser HTTP/SSE and saved-context assertions, Rust fixture isolation, executable negative checks, CI/toolchain/artifacts, documentation, adversarial cleanup, and synthesis. The coordinator independently checked the evidence and rejected nonmaterial documentation suggestions. Opus lanes were static reviews; their shell execution was unavailable because of a host temporary-storage quota.
Independently executed on this exact head:
npm ciand the fullnpm run test:keystone-nativecommand with CI Rust profile settings: typecheck, fresh frontend/Rust harness builds, 2/2 browser scenarios passed (10.2 s). Both owned listeners closed and runtime temporary directories were removed.- The newly built library test binary filtered to
keystone::: 136 passed, 1 intentionally ignored socket harness. - Missing Chromium, absent launcher configuration, and non-loopback URL checks rejected as intended. Wrong fixture identity stopped both scenarios after capability GETs, with zero mutation requests. A TERM-resistant fake cargo and descendant were killed in 3.03 s, exit 130, with no live descendants or runtime leftovers.
- Ordinary live-suite discovery: 30 tests, native fixture excluded. Rustfmt, Prettier on changed frontend files, JavaScript syntax, diff whitespace, YAML/embedded Python parsing, and trigger/toolchain/artifact assertions passed.
Hosted native-smoke CI passed, including 2/2 scenarios (11.4 s) and artifact upload, on GitHub's merge ref for this head/base. Other PR checks were still running at the final snapshot; this is not an all-checks-green claim.
Scope limits: the socket fixture uses synthetic authorization, metadata, rows, model responses, and an execution-service adapter. It does not establish production login/RBAC, DataFusion, encrypted credential storage, production run dispatch, or provider-transport readiness. macOS remains unverified; Windows is unsupported. Reload/resubscription and committed SSE cursor replay were checked, not forced network loss. No real provider calls were made. Earlier author-reported results and the historical paid-call observation were not used as substitutes for the executions above.
Description
Separate frontend and backend tests did not exercise the native browser cancellation contract together. This adds
npm run test:keystone-native: one command that builds the frontend and Rust socket fixture, runs focused browser checks, retains diagnostics, and cleans up its own processes and storage.The two scenarios cover draft save without inference, explicit completion, run links and reload, a durable child with explicitly selected direct-parent evidence and version/hash pins, composing reload/resubscription, the nonterminal cancellation acknowledgement followed by terminal cancellation, and exact persisted SSE cursor replay.
The launcher uses fresh loopback ports, temporary storage and a unique fixture identity checked before browser mutations. It retains stage logs, request logs, SSE evidence, cancellation acknowledgements, traces, screenshots, video and HTML/JSON reports. The fixture uses real native routes, repository and coordinator code with synthetic authorization, rows and model answers. The ordinary live-server suite excludes these cases.
CI runs on relevant PRs, matching pushes to
mainandfeat/native-only-uifor cache warming, and manual dispatch. It installs the repository-pinned Rust version and Chromium, caches compilation dependencies, and retains diagnostics for seven days. Documentation explains setup, coverage boundaries and the separate prior paid-provider observation.Validation
Production login/RBAC, DataFusion, credential encryption and live provider transport are outside this synthetic CI gate. It does not simulate forced network loss or establish deployment readiness. The documented wrong-master-key readiness and generic provider-error findings remain separate follow-up work. No additional billable call was made for this cleanup.
The base branch was fetched and remained at
72c28b9240e5b92bc71171e906083b34b0bf2eb9; no integration changes were necessary.This PR has:
Log ingestion/query integration was exercised during the preceding investigation; it was not rerun for this test-infrastructure-only cleanup.
Need help on this PR? Tag
@codesmith-botwith what you need.