Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 113 additions & 0 deletions .github/workflows/keystone-native.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
name: Keystone native smoke

on:
pull_request:
paths:
- 'src/**'
- 'frontend/**'
- 'Cargo.toml'
- 'Cargo.lock'
- 'rust-toolchain.toml'
- 'build.rs'
- 'build_support/**'
- 'scripts/**'
- '.github/workflows/keystone-native.yaml'
workflow_dispatch:
# Warm reusable base-branch Rust caches; PR merge-ref caches cannot serve
# subsequent PRs. Keep the same relevant-file scope as the automatic PR gate.
push:
branches: ['main', 'feat/native-only-ui']
paths:
- 'src/**'
- 'frontend/**'
- 'Cargo.toml'
- 'Cargo.lock'
- 'rust-toolchain.toml'
- 'build.rs'
- 'build_support/**'
- 'scripts/**'
- '.github/workflows/keystone-native.yaml'

permissions:
contents: read

concurrency:
group: keystone-native-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
native-smoke:
runs-on: ubuntu-24.04
timeout-minutes: 120
env:
# DataFusion debug symbols and incremental objects can exhaust runner disk.
CARGO_PROFILE_DEV_DEBUG: '0'
CARGO_PROFILE_TEST_DEBUG: '0'
CARGO_INCREMENTAL: '0'
CARGO_BUILD_JOBS: '4'
steps:
- uses: actions/checkout@v4

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24'
cache: npm
cache-dependency-path: frontend/package-lock.json

- name: Read repository Rust version
id: rust-version
run: |
python3 - <<'PY'
import os
import tomllib
from pathlib import Path

channel = tomllib.loads(Path('rust-toolchain.toml').read_text())['toolchain']['channel']
with open(os.environ['GITHUB_OUTPUT'], 'a') as output:
output.write(f'channel={channel}\n')
PY

- name: Set up pinned Rust toolchain
uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ steps.rust-version.outputs.channel }}

- name: Cache Rust dependencies
uses: Swatinem/rust-cache@v2
with:
shared-key: keystone-native
cache-on-failure: true

- name: Install native build dependencies
# Default features exercise the native server without Kafka dependencies.
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential pkg-config cmake clang libssl-dev

- name: Install frontend dependencies
working-directory: frontend
run: npm ci

- name: Install Chromium
working-directory: frontend
run: npx playwright install --with-deps chromium

- name: Exercise Keystone against a fresh native server
# The same local command builds the UI and Rust server, runs focused
# browser tests, and cleans up ephemeral ports and storage. Synthetic
# provider responses cover the product boundary without paid calls or
# provider secrets; this gate does not establish real-provider readiness.
working-directory: frontend
run: npm run test:keystone-native

- name: Upload native smoke diagnostics
if: always()
uses: actions/upload-artifact@v4
with:
name: keystone-native-${{ github.run_id }}-${{ github.run_attempt }}
path: frontend/test-results/keystone-native/
retention-days: 7
# Dependency/preflight failures can happen before a run directory exists.
if-no-files-found: ignore
70 changes: 70 additions & 0 deletions frontend/docs/keystone-browser-validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,3 +193,73 @@ screenshot specs and all **60** Keystone cases. Results are in
Screenshot comparisons are only meaningful inside the CI container. On a developer machine the
screenshot specs may fail because of local font and rendering differences; that is not a
regression signal, and baselines must not be refreshed from such a run.

## Native socket workflow

From `frontend`, run:

```sh
npm ci
# One-time prerequisite; the launcher does not install browsers:
npx playwright install chromium
npm run test:keystone-native
```

The launcher requires Node 22.12 or later, this repository's Rust toolchain and its native build
prerequisites. The launcher uses Linux/macOS process groups; Linux has been exercised locally,
macOS remains unverified, and Windows process cleanup is not supported. Use
`npm run test:keystone-native -- --help` for environment options. `TMPDIR` selects temporary
storage, `CARGO_TARGET_DIR` permits reuse of compilation dependencies, and
`PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` selects an explicitly installed Chromium.

This command typechecks and builds the actual frontend into a private temporary directory,
builds the Rust library test harness and selects its executable from Cargo JSON output. It
launches only the ignored native socket fixture and only `keystone-native-readiness.spec.ts`.
Both HTTP listeners bind fresh OS-assigned loopback ports. A ready-file handshake and unique
fixture response header are checked before browser mutations. There is no external-server URL,
server reuse, HMR, existing `dist` dependency, native API interception or provider credential.
The ordinary `test:live` suite excludes these cases.

The fixture exercises actual Actix native routes, durable repository commits, coordinator,
projections, native adapter and SSE. Authentication/current grants, user and dataset metadata,
scoped sample rows and model answers are synthetic. Production login/RBAC middleware, DataFusion,
encrypted credential storage, real provider transport and production deployment are outside this
workflow. The explicit `[diagnostic:hold]` prompt token keeps generation composing for the cancel
case. The cancellation response acknowledges a still-running run; terminal cancellation arrives
through persisted progress after execution unwinds.

The checks cover draft save/navigation with zero implicit runs, one explicit completion, run-link
navigation/reload, explicit selection of one direct-parent source for a durable child draft,
composing reload/resubscription, cancellation and terminal reload. Cursor replay is compared with
the committed stream's tail. Reload/resubscription does not simulate packet loss or prove every
network-disconnect recovery path.

Each invocation retains stage logs, its runtime manifest, request/status logs, cancellation
acknowledgement, SSE replay, Playwright JSON/HTML report, screenshots, videos and traces under
`test-results/keystone-native/<unique-id>`. `KEYSTONE_NATIVE_ARTIFACTS` changes the parent folder.
On failure, read the named stage log and open `<unique-id>/report/index.html`; traces can be opened
with `npx playwright show-trace <trace.zip>`. Artifacts contain synthetic fixture data. Runtime
storage and the temporary frontend build are removed on success, failure and SIGINT/SIGTERM;
only process groups started by this invocation are stopped. No existing services are killed.

The `Keystone native smoke` workflow invokes the same command automatically for relevant pull
requests and pushes to `main` or `feat/native-only-ui`, and also supports manual dispatch. It
uploads `test-results/keystone-native/` on success or failure for seven days. The new hosted CI
workflow has not yet been exercised; local execution and static workflow review are separate
validation evidence.

### Live validation boundary

On 2026-10-14, a separately authorized paid run using the configured default model
`xiaomi/mimo-v2.6-flash` completed against a local production runtime with synthetic telemetry.
The request explicitly supplied `zdr: true`, `data_collection: "deny"`,
`allow_fallbacks: false`, `require_parameters: true`,
`max_price: { "prompt": 0.2, "completion": 1 }`, `max_tokens: 2048`, and disabled/excluded
reasoning. Success establishes that this request completed; it does not independently verify
provider retention practices or identify the serving endpoint. The adapter did not expose the
actual billed cost. This observation is
separate from the repeatable fixture command and does not establish production readiness or
future endpoint availability. Two independently reproduced P2 findings remain unfixed: reopening
the encrypted credential store with a different master key can still advertise readiness, and
execution failures can present generic messages rather than an actionable cause. Credential,
account and private runtime details are deliberately excluded from these documents.
14 changes: 7 additions & 7 deletions frontend/docs/keystone-foundations.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ initialized.
> **Status: merge-readiness fixes implemented and validated.** The frontend CI pipeline and the
> clean backend validation both passed on the final source, subject to the baseline and
> host-toolchain notes in [validation status](#validation-status). This document makes no
> production-readiness claim: none of these fixes has been deployed or exercised against a running
> server or a paid provider.
> production-readiness claim. Later local runtime and paid-provider observations, with remaining
> readiness findings, are recorded in [native browser validation](keystone-browser-validation.md#live-validation-boundary).

## Implemented backend

Expand Down Expand Up @@ -496,11 +496,11 @@ These limits come from the merge-readiness review and remain after its fixes.
running. Quarantined files are kept until an operator removes them, and a full quarantine leaves
unreferenced files counting toward the store's entry cap. Request-body copies of a credential
are not zeroized.
- **Provider.** No paid live call has validated the configured model against the required privacy
routing. See [provider foundations](keystone-providers.md#privacy-routing-requirements).
- **Not yet exercised on a running server.** The fixes in this change are covered by automated
tests only. They have not been deployed, no existing installation has been restarted with them,
and no paid inference has run against them.
- **Provider/runtime validation.** One authorized paid MiMo run completed on 2026-10-14 against
synthetic telemetry in a local runtime. This does not establish future endpoint availability or
production readiness; billed cost was not exposed. Wrong-master-key readiness and generic
failure messages remain independently reproduced P2 findings. See
[live validation boundary](keystone-browser-validation.md#live-validation-boundary).
- Distributed coordinators, broader tools/SQL, sharing, dashboard promotion, retention/archival,
encrypted-store rotation, real token streaming and advisory-provider activation remain follow-up
work, not capabilities silently enabled by this increment.
9 changes: 9 additions & 0 deletions frontend/docs/keystone-local-smoke.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ A recipe for running native Keystone on one development machine with synthetic d
from any other Parseable installation. It is a local test, not a release deployment. Replace every
`<placeholder>` with a value for your machine; nothing here is a working credential.

For repeatable browser checks without credentials, use
[`npm run test:keystone-native`](keystone-browser-validation.md#native-socket-workflow). It owns
a fresh synthetic socket fixture and frontend build. The recipe below instead exercises the
production runtime, authentication and query engine.

## Runtime layout

Create one private runtime directory and keep everything for this instance inside it:
Expand Down Expand Up @@ -130,3 +135,7 @@ internal datasets such as `pstats` are never offered, and the backend rejects th

Smoke artifacts (logs, screenshots, browser results) belong outside the repository, for example
under `$RUNTIME` or a private scratch directory. They are not part of this change.

Local production-runtime and paid-provider observations from 2026-10-14, including remaining
unfixed readiness findings, are recorded in the
[live validation boundary](keystone-browser-validation.md#live-validation-boundary).
9 changes: 5 additions & 4 deletions frontend/docs/keystone-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

The native runtime now includes optional live generation and encrypted credential entry.
They remain unavailable until operator deployment inputs and an admin-entered credential
are present. Automated verification uses fixtures and local TLS only. No paid call has
validated the configured model against the privacy routing requirements below.
are present. Automated verification uses fixtures and local TLS only. A separately authorized
paid MiMo run completed on 2026-10-14; actual billed cost was not exposed. See the
[live validation boundary](keystone-browser-validation.md#live-validation-boundary).

`ModelProvider` and the separate advisory `DecisionProvider` receive a
`FrozenProviderContext` verified from the server-built `PinnedContext`. The wire
Expand Down Expand Up @@ -116,8 +117,8 @@ that meets both requirements within the price caps. The documentation does not
state the `zdr: true` behavior when no ZDR endpoint exists; Keystone assumes the
request fails and never falls back. Such a run fails with the upstream rejection
rather than sending data to a less restricted endpoint. The default model was chosen
for price; its availability under these requirements has not been tested with a
paid call, so no call has confirmed which endpoints serve it. Before
for price; one controlled paid MiMo run completed on 2026-10-14. This does not
guarantee future availability or identify every qualifying endpoint. Before
enabling Keystone, check the model's endpoint list and run one controlled call;
choose another model with `KEYSTONE_GENERATION_MODEL` if none qualifies.

Expand Down
Loading
Loading