Skip to content

test(keystone): add repeatable native browser smoke coverage - #41

Merged
VibeSWE merged 1 commit into
feat/native-only-uifrom
investigate/keystone-native-readiness
Oct 14, 2026
Merged

VibeSWE merged 1 commit into
feat/native-only-uifrom
investigate/keystone-native-readiness

Conversation

@VibeSWE

@VibeSWE VibeSWE commented Oct 14, 2026 •

Copy link
Copy Markdown

Description

Separate frontend and backend tests did not exercise the native browser cancellation contract together. This adds npm run test:keystone-native: one command that builds the frontend and Rust socket fixture, runs focused browser checks, retains diagnostics, and cleans up its own processes and storage.

The two scenarios cover draft save without inference, explicit completion, run links and reload, a durable child with explicitly selected direct-parent evidence and version/hash pins, composing reload/resubscription, the nonterminal cancellation acknowledgement followed by terminal cancellation, and exact persisted SSE cursor replay.

The launcher uses fresh loopback ports, temporary storage and a unique fixture identity checked before browser mutations. It retains stage logs, request logs, SSE evidence, cancellation acknowledgements, traces, screenshots, video and HTML/JSON reports. The fixture uses real native routes, repository and coordinator code with synthetic authorization, rows and model answers. The ordinary live-server suite excludes these cases.

CI runs on relevant PRs, matching pushes to main and feat/native-only-ui for cache warming, and manual dispatch. It installs the repository-pinned Rust version and Chromium, caches compilation dependencies, and retains diagnostics for seven days. Documentation explains setup, coverage boundaries and the separate prior paid-provider observation.

Validation

  • Complete packaged Rust/frontend/browser workflow: 2 scenarios passed on the final source.
  • Keystone tests from the resulting Rust executable: 136 passed, 1 intentionally ignored socket harness.
  • TypeScript, frontend production build, frontend formatting, Rust formatting and diff whitespace checks passed.
  • Wrong fixture identity rejected before any mutation; missing Chromium produced actionable preflight failure.
  • Interruption test with a process and descendant ignoring SIGTERM: exit 130 in about 3 seconds, no surviving children or runtime directories.
  • Both real fixture listeners closed and temporary runtime storage was removed after the successful run.
  • Ordinary live-suite discovery lists 30 tests and excludes the fixture-only cases.
  • Workflow YAML, embedded Python and trigger/toolchain configuration checked. Hosted GitHub CI has not yet run; macOS remains unverified.

Production login/RBAC, DataFusion, credential encryption and live provider transport are outside this synthetic CI gate. It does not simulate forced network loss or establish deployment readiness. The documented wrong-master-key readiness and generic provider-error findings remain separate follow-up work. No additional billable call was made for this cleanup.

The base branch was fetched and remained at 72c28b9240e5b92bc71171e906083b34b0bf2eb9; no integration changes were necessary.

This PR has:

  • Added comments explaining fixture boundaries, ownership and cleanup.
  • Added documentation for the command, artifacts, CI and validation limits.

Log ingestion/query integration was exercised during the preceding investigation; it was not rerun for this test-infrastructure-only cleanup.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@comment-clanker comment-clanker Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No actionable findings in PR41 at d34b599715a6b448b9a945f2040c87194e735b0a, against base 72c28b9240e5b92bc71171e906083b34b0bf2eb9.

Reviewed all 12 changed files across eight complementary assignments: four GPT-6.1-Sol and four Claude Opus 5.5 reviewers covering launcher/process lifecycle, browser HTTP/SSE and saved-context assertions, Rust fixture isolation, executable negative checks, CI/toolchain/artifacts, documentation, adversarial cleanup, and synthesis. The coordinator independently checked the evidence and rejected nonmaterial documentation suggestions. Opus lanes were static reviews; their shell execution was unavailable because of a host temporary-storage quota.

Independently executed on this exact head:

  • npm ci and the full npm run test:keystone-native command with CI Rust profile settings: typecheck, fresh frontend/Rust harness builds, 2/2 browser scenarios passed (10.2 s). Both owned listeners closed and runtime temporary directories were removed.
  • The newly built library test binary filtered to keystone::: 136 passed, 1 intentionally ignored socket harness.
  • Missing Chromium, absent launcher configuration, and non-loopback URL checks rejected as intended. Wrong fixture identity stopped both scenarios after capability GETs, with zero mutation requests. A TERM-resistant fake cargo and descendant were killed in 3.03 s, exit 130, with no live descendants or runtime leftovers.
  • Ordinary live-suite discovery: 30 tests, native fixture excluded. Rustfmt, Prettier on changed frontend files, JavaScript syntax, diff whitespace, YAML/embedded Python parsing, and trigger/toolchain/artifact assertions passed.

Hosted native-smoke CI passed, including 2/2 scenarios (11.4 s) and artifact upload, on GitHub's merge ref for this head/base. Other PR checks were still running at the final snapshot; this is not an all-checks-green claim.

Scope limits: the socket fixture uses synthetic authorization, metadata, rows, model responses, and an execution-service adapter. It does not establish production login/RBAC, DataFusion, encrypted credential storage, production run dispatch, or provider-transport readiness. macOS remains unverified; Windows is unsupported. Reload/resubscription and committed SSE cursor replay were checked, not forced network loss. No real provider calls were made. Earlier author-reported results and the historical paid-call observation were not used as substitutes for the executions above.

@VibeSWE
VibeSWE merged commit 6e9d963 into feat/native-only-ui Oct 14, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant