Add native Agents, operations, setup and network-policy settings

frontend/README.md

@@ -66,13 +66,18 @@ This creates `build.zip` with a top-level `dist/` directory, including `dist/ind
 - Datasets: search, create logs/metrics/traces with an optional custom partition, manage information and ingestion/storage statistics, inspect schema, explore, set/clear retention, configure/disable hot tier when the server enables it, and delete with typed name confirmation. Time partitions cannot be set at creation. Static schema upload, tags/labels, existing partition editing and index management are not supported.
 - Dashboards: server-backed lists and detail routes, search/tags/favourites/sort/pagination, ownership, full-document saves with conflict resolution, SQL and PromQL tiles, six variable types, URL time ranges and selections, classic import/export, and explicit migration of old browser-local dashboards. Demo dashboards use independent in-memory state per client.
 - Team: native users and one-time passwords, role and privilege management, default OIDC role configuration, provider group mappings and role provenance, and API key creation/copy/deletion. Search and 25-row pagination apply to every tab; authorization stays with the backend.
+- Agents: schema-discovered GenAI telemetry datasets, bounded recent records, time ranges, local search, record inspection and trace handoff.
+- Cluster: registered node reachability and resource metrics with independent request states; read-only. Errors: paginated OTLP error spans and trace handoff; classic grouped issue analytics remain unavailable.
+- Workspace setup: `/onboarding` discovers accessible datasets and opens the matching explorer; `/new` explains server scope and provides connection/setup actions. Hosted workspace creation is not exposed by this server.
+- Network access: edit the server’s outbound alert-target policy, including domain/CIDR rules, private IP access and TLS verification.
+- Keystone and LLM settings: native capability pages with explicit limitations and deployment-dependent classic links. See [native screen scope](docs/prism-parity.md#native-missing-screen-scope-2026-10-12).
 - Component gallery at `/components`, plus isolated Storybook stories for UI, PromQL, charts, Traces, APM and ingestion setup.
 
 The default log query reads at most 100 rows, with choices up to 10,000. Message search assumes a `message` field. The histogram separately aggregates every matching event through `/query`, using the same captured bounds and predicates as the rows. Its 48 buckets divide elapsed time evenly; they are not calendar-day buckets. AND/OR filters have two levels, at most 12 groups and 50 rules, and support `=`/`!=`; SQL NULL values are excluded by both operators. Saved log views use server `/filters` CRUD, while copy links encode versioned explorer state with fixed absolute bounds. Context uses the existing `/query/context` endpoint with a 101-event bound and requires `p_timestamp` plus a string `message`, `body` or `log`. Large/wide results virtualize rows and columns; sorting and export cover loaded rows only. Wrapped virtual rows show a 96px preview; event details retain the full value. Timezones change display and wall-clock editing while requests retain UTC instants; DST gaps are rejected and repeated times require an earlier/later choice. Demo SQL intentionally supports only a small SELECT subset and rejects unsupported syntax. Full SQL requires a server. SQL multi-dataset generation uses `UNION ALL BY NAME`, fills missing columns with NULL and requires compatible types for common columns, and keeps the 100-row default; demo does not support unions. Incoming query links open or focus a tab while preserving other drafts. Reopening an already-open saved query focuses that tab and preserves its edits. The saved SQL library uses persistent browser local storage, while draft workspaces use session storage independent to each browser tab and survive navigation/reload. Both are scoped to the server and signed-in user (with a separate demo scope); they are not server-synced, and result rows are not persisted. Query handoff parameters are consumed after opening the tab so reloading an edited draft does not reopen the original query. SQL chart previews use the executed query and captured time bounds. Adding a chart to an existing dashboard preserves that dashboard’s time range; a new dashboard starts with the executed range.
 
 Logs implementation keeps persisted settings in `logViewReducer.ts`, with explicit dataset reset, saved-view and classic-link transitions. `logFilters.ts` owns expression limits and additions; adding within a group and narrowing the whole expression remain distinct operations. `logQuery.ts` compiles canonical criteria once for the page's row and histogram queries. Changes to client-only filter IDs or empty groups do not refresh the captured time window. Histogram responses carry their bucket geometry to the renderer. Both table modes use `logTable.tsx` for sorting, labels and cells. `logsContract.ts` validates server payloads; `demoLogs.ts` implements the demo methods separately. Saved-view instants, server timestamps and timezone wall-clock inputs retain separate validators because their accepted formats differ.
 
-Server-synced SQL saved queries, SQL AI assistance, and original Prism pixel parity remain future iterations. Traces/APM/ingestion contracts, reference screenshots, deliberate limits and test commands are documented in [observability validation](docs/observability-validation.md). No unusable paid-feature navigation or upgrade controls are exposed.
+Server-synced SQL saved queries, SQL AI assistance, and original Prism pixel parity remain future iterations. Traces/APM/ingestion contracts, reference screenshots, deliberate limits and test commands are documented in [observability validation](docs/observability-validation.md). Native routes for Keystone and LLM settings explain unsupported management capabilities and link to classic UI when mounted alongside it; they do not expose upgrade controls or simulated writes.
 
 ## Components and selectors
 

frontend/docs/api-contracts.md

@@ -206,3 +206,15 @@ APM uses typed SQL builders through the existing `/api/v1/query` client, with sc
 Add data creates datasets with `X-P-Log-Source` set to `json`, `otel-logs`, `otel-metrics` or `otel-traces`. Generated JSON commands use `/api/v1/ingest`; OTLP commands use `/v1/{signal}` with JSON encoding. Configuration contains environment-variable credential placeholders. Destinations with custom partitions are explicitly unsupported by guided setup: the required fields are explained, configuration/copying/verification remain unavailable, and selecting or creating an unpartitioned dataset restores setup. Verification reads dataset statistics, schema and info: the first check records a baseline, later lifetime-count increases report new stored events without claiming sender identity. Endpoint edits retain the baseline; destination changes reset it.
 
 See [observability validation](observability-validation.md) for bounds, caps, error behavior, live evidence and reference screenshots.
+
+## Native operations and setup screens
+
+Agents uses existing dataset list, schema, metadata and SQL requests. Discovery recognizes flattened `gen_ai.agent.name`, `gen_ai.agent.id`, `gen_ai.operation.name`, `gen_ai.request.model` and `gen_ai.tool.name`; direct dataset links defer the full schema scan until the selector is focused or rescanned. Discovery is limited to four concurrent schema requests and skips forbidden/missing datasets. Queries quote dataset/field identifiers, use the dataset time partition, and return at most 100 recent non-null GenAI records in captured UTC bounds. Search applies only to those loaded records. Run aggregation, evaluations and pricing are not inferred. Unknown demo dataset names cannot return synthetic records.
+
+Cluster reads `GET /api/v1/cluster/info` and `/api/v1/cluster/metrics` through the shared authenticated client. The first returns node identity, storage/staging paths, reachability and nullable status/error text. Metrics are independent: a denied metrics request does not hide node health. Nonfinite Rust metric values serialized as null display as unavailable per field. Missing cluster routes (404/405) show a deployment-mode explanation; 403 and 401 retain shared permission/session handling. No node deletion is exposed.
+
+Errors uses the existing trace-list contract with `options: "errorSpans"`, selectable UTC ranges, server sorting and pagination. The selected dataset must advertise `otel-traces`. Failed metadata inspection is distinguished from confirmed incompatibility and can be retried. Dataset/range changes reset pagination and cancel obsolete requests. Trace handoff preserves the captured query bounds. Classic grouped-issue and frequency endpoints are absent and are not called.
+
+Network access reads and replaces `GET/PUT /api/v1/alert-target-policy`. The complete object contains `allowPrivate`, `allowInvalidTls`, `allowedDomains`, `allowedCidrs`, `deniedDomains`, and `deniedCidrs`. All fields are required; the server is authoritative for authorization, CIDR validation and allow/deny overlaps. This policy governs outbound alert delivery, not ingress access to Parseable. The editor retains failed-save drafts and protects unsaved navigation. Demo writes are isolated within each client. The backend caches policy per node and loads it on startup; cross-node propagation is not guaranteed by a successful save.
+
+Onboarding uses the existing telemetry-aware dataset inventory and Add data routes. It does not call the side-effecting demo-data endpoint or provision a hosted workspace. Keystone and LLM management remain unsupported by this checkout: native deep links explain the limitation and offer deployment-dependent classic handoff only when classic is available. No provider keys are read or stored.

frontend/docs/prism-parity.md

@@ -121,3 +121,21 @@ This scope records implementation behavior, not a claim of full runtime or pixel
 Implemented on `feat/missing-observability-screens`: native trace list/detail and keyboard waterfall; SQL-derived span inventory, service insights and observed dependency map; JSON/OTLP/Collector ingestion configuration and measured arrival checks. Dataset navigation uses confirmed source metadata, including a Logs fallback for older JSON-backed datasets marked as traces. No paid APM endpoints or upgrade prompts are introduced. Original OSS APM supplied an enterprise call to action, so this is functional scope rather than pixel parity with a working original APM screen.
 
 The [observability validation record](observability-validation.md) documents original/new screenshots, contracts, capped results, fixture behavior and final test evidence. The native trace list also corrects identical retries with distinct ingestion timestamps so returned records agree with aggregate counts.
+
+## Native missing-screen scope (2026-10-12)
+
+The previously classic-only routes below now resolve to native pages. `classicUi.ts` marks them `limited`, distinguishing a usable native subset or capability explanation from full classic parity. The existing Traces, APM, Add data and telemetry-aware dataset routes retain their behavior.
+
+| Routes                        | Native behavior                                                                                                                                                                                  | Deliberate limits                                                                                                                          |
+| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ |
+| `/agents`, `/agents/:dataset` | Discover flattened GenAI fields through dataset schemas; query the latest 100 telemetry records in a selected time range; search loaded records; inspect full records; open compatible trace IDs | Individual events/spans, not classic run aggregation, evaluations, pricing or token-cost analytics. Demo uses explicit synthetic fixtures. |
+| `/cluster`                    | Show registered distributed nodes, reachability, status, storage/staging locations and resource metrics from the existing cluster endpoints                                                      | Read-only. Standalone servers can report cluster administration unavailable. Node and metrics failures remain separate.                    |
+| `/errors`, `/errors/:dataset` | Discover accessible OTLP trace datasets; list, sort and paginate error spans; open their trace waterfall                                                                                         | Classic grouped error issues and frequency endpoints are absent from this backend; arbitrary log issue aggregation is not fabricated.      |
+| `/onboarding`                 | Discover accessible datasets, link each to its confirmed explorer, and guide ingestion and team setup                                                                                            | Dataset presence is not presented as proof of event arrival. Existing Add data owns ingestion configuration and verification.              |
+| `/new`                        | Explain server workspace scope; open connection settings or native setup                                                                                                                         | No API in this checkout provisions a separate hosted workspace.                                                                            |
+| `/keystone`, `/keystone/:id`  | Native availability page and classic investigation handoff where available                                                                                                                       | No supported investigation/conversation API in this checkout. No claim that an ID exists or is accessible.                                 |
+| `/settings/llm-key`           | Native availability pages and deployment-dependent classic handoff                                                                                                                               | No provider-key management API. No credentials are read, stored or submitted.                                                              |
+
+Network access (`/settings/network-access`) edits the real outbound alert-target policy via `GET` and `PUT /api/v1/alert-target-policy`. It is distinct from ingress access control. The full policy includes private-IP and invalid-TLS switches plus allowed/denied domains and CIDRs. Backend authorization and validation remain authoritative; failed saves retain the draft. The policy cache is per server node and loaded at startup; cross-node propagation is a backend limitation, not asserted by the UI.
+
+Contracts were checked against `src/handlers/http/modal/query_server.rs`, `src/handlers/http/modal/server.rs`, `src/handlers/http/cluster/mod.rs`, `src/handlers/http/traces.rs`, and the classic route/asset evidence. Keystone metastore methods and LLM tool-execution endpoints do not constitute management HTTP contracts. Shared client handling preserves cancellation, server errors and session recovery for the added cluster reads.

frontend/docs/validation-review.md

@@ -639,3 +639,19 @@ The browser tests use deterministic transports. They check generated SQL and
 limited-result tooltip values, chart rendering, save/reload, explicit legacy
 bindings, and opaque metadata preservation. They do not establish live backend
 ingestion or cross-browser coverage.
+
+## Native missing screens (2026-10-12)
+
+[Shared Mayfly completion record](https://mayfly-celld.tail835e0d.ts.net/w/46ncdGypknzAj7dnRkJ8bQ?page=d665474a-2bc8-445f-a8e7-4949ea6e32e3#J63UryiLZKvb1G5vnY7RVQhXPDhXy-GpIhjXiCB_TzM) documents implementation commit `83503697`, scope limits, review findings and validation caveats, and is linked from the earlier Traces/APM/ingestion wiki page. At the initial wiki publication, this implementation was committed locally and had not been pushed or deployed.
+
+Implemented on `feat/native-missing-screens` from `origin/main` (`81d1e0c9`), with five Sol 6.1 agents covering implementation, independent regressions and Storybook QA, plus four Opus 5.5 review rounds. See [scope and deliberate limits](prism-parity.md#native-missing-screen-scope-2026-10-12). Network access is the real outbound alert-target policy editor; Keystone/LLM management and hosted workspace provisioning remain explicitly unavailable.
+
+Validation of the final implementation:
+
+- Production build and TypeScript pass. All **2,166 unit tests pass**, with one pre-existing skipped test.
+- All **30 focused browser tests pass** in one serial run against a fresh production bundle: Agents, operations, onboarding, settings, outbound policy, stale dataset/session behavior and `/next` navigation. Server calls are mocked; these results do not claim a live distributed-cluster or policy-propagation check.
+- **22 production mount/stale-asset tests pass**, including hard loads of every added screen at both `/` and `/next`. The **26 classic-link/onboarding tests** also passed during integration.
+- All **49 native Storybook checks pass** across a serial run and one immediate isolated retry of a transient empty iframe. Coverage includes loading/empty/permission/error states, keyboard inspection and focus restoration, time-range request bodies, policy save/discard/validation, and narrow dark-theme accessibility.
+- Opus findings addressed include network API scope, full-policy payloads, domain rules that could never match the backend, authorization-header allowlist help, permission-safe discard, unique field descriptions, missing metric values, failed dataset verification, unnecessary discovery requests, duplicate refresh requests and native URL preservation. An independent regression caught and fixed synthetic demo data appearing under unknown dataset names.
+
+Broader suites are **not clean** on this host. The 493-test development-server run was stopped after repeated `net::ERR_INSUFFICIENT_RESOURCES`, blank lazy pages and Chromium failures: 238 passed, 45 failed, 3 interrupted, 1 skipped and 206 not run. The full 221-test Storybook run had 162 passes and 59 failures (38 existing screenshot assertions and 21 runtime/resource failures). A representative grouped-bars screenshot was rebuilt and tested from an isolated `origin/main` archive: it fails by the same 1,093 pixels, and its actual PNG is byte-identical to this branch (`c21e1b584291ac5ee37e020c75832fb14dbec1566610613dc06c792f9249b06d`). No baselines were updated. The obsolete sidebar expectation was updated for Agents/Errors and passed on retry; the reported APM 24-hour test also passed independently. The final focused run used a production bundle to avoid the development server's large module graph.

frontend/e2e-prod/mounts.spec.ts

@@ -43,6 +43,21 @@ for (const mount of ['', '/next']) {
       await expect(page).toHaveURL(`${mount}/add-data`);
       await expect(page.getByRole('heading', { name: 'Add data', exact: true })).toBeVisible();
 
+      for (const [path, title] of [
+        ['/agents/agent_telemetry', 'Agents'],
+        ['/cluster', 'Cluster'],
+        ['/errors/demo_traces', 'Errors'],
+        ['/onboarding', 'Get started'],
+        ['/new', 'Set up a workspace'],
+        ['/keystone/example', 'Keystone'],
+        ['/settings/llm-key', 'LLM settings'],
+        ['/settings/network-access', 'Network access'],
+      ]) {
+        await page.goto(`${mount}${path}`);
+        await expect(page.getByRole('heading', { name: title, exact: true })).toBeVisible();
+      }
+      await expect(page.getByLabel('Allowed domains', { exact: true })).toBeVisible();
+
       expect(failed).toEqual([]);
       expect(paths.filter((path) => !path.startsWith(`${mount}/`))).toEqual([]);
       // Lazy chunks import the entry module that is already running.