🤖 fix: clarify event-only watch options after Kubernetes defaulting

[ThomasK33]

Problem

An event-only workspace watch with no resourceVersion is rejected by the deployed aggregated API. Kubernetes first defaults it into an initial-events request, which this storage intentionally does not implement. Clients need a documented way to request the supported local event stream.

During #109 validation on an owned KIND deployment, this request failed:

GET /apis/aggregation.coder.com/v1alpha1/namespaces/coder/coderworkspaces?watch=true&fieldSelector=metadata.name=<canonical-name>
HTTP/1.1 400 Bad Request

The Status body reported reason: BadRequest and:

invalid watch options: sendInitialEvents=true is not supported for this watch endpoint

In the same run, a watch with resourceVersion=12345&timeoutSeconds=2 returned HTTP 200. A focused matrix for absent/zero/current-token RV and explicit sendInitialEvents=false is planned; the latter cases are not yet runtime-verified.

Mechanism and evidence boundaries

1. vendor/k8s.io/apimachinery/pkg/apis/meta/internalversion/defaults.go:25-38 injects sendInitialEvents=true and resourceVersionMatch=NotOlderThan when WatchList is enabled, RV is empty or 0, and neither option was supplied.
2. vendor/k8s.io/apiserver/pkg/endpoints/handlers/get.go applies that defaulting before storage dispatch.
3. internal/aggregated/storage/watch.go:44-59 rejects initial events and RV matching. Both workspace and template storage call this validator. TestWatchRejectsDefaultedLegacyWatchListOptions explicitly expects this rejection; it is not evidence of an accidentally missing exception.
4. These files, the tests, and the vendored feature definitions are unchanged between 🤖 fix: make workspace resource versions reflect backend changes #109's candidate and its base 2fcd31d42a345a80841e2f5133ee8f796388da78. This establishes unchanged relevant source, not a separate live reproduction on the base.
5. The workspace endpoint was observed directly. Template HTTP behavior still needs coverage. The preliminary harness audit missed the defaulting stage; its prediction that the unqualified request would reach streaming HTTP 200 was incorrect.

Acceptance criteria

1. Test requests through generic option decoding/defaulting/validation, not only direct storage calls. Cover absent RV, 0, a nonzero opaque RV, explicit sendInitialEvents=false, explicit initial events, and RV matching for both resources.
2. Document the exact supported event-only request form and the difference between option acceptance and storage ignoring an accepted RV. Keep the no-replay/no-out-of-band-event limitations explicit.
3. Preserve honest initial-event semantics. Do not silently accept sendInitialEvents=true while omitting the promised initial events. Any expansion of watch behavior needs a separate design decision.
4. Keep 🤖 fix: make workspace resource versions reflect backend changes #109's new watch explanation accurate before its publication. A new versioning change must not imply every watch query is accepted.
5. Run the applicable test/docs gates and record HTTP evidence, screenshot/video, and exact owned-resource cleanup for any implementation.

Ownership and sequencing

Owner: maintainer desk. Track the compatibility/documentation work separately from #109's representation fingerprint. The fourth full #109 run remains failed, and full-suite relaunches for that validation tranche are stopped. Only separately authorized, bounded diagnostics and the missing acceptance scenarios may use the retained owned fixture; this issue does not authorize retries or a watch redesign.

---

Generated with xum • Model: coder:openai/gpt-6-astra • Thinking: xhigh

[ThomasK33]

Bounded HTTP diagnostic result

The four workspace probes completed on #109's unchanged candidate. Every curl transfer exited 0 and produced complete response headers:

Query addition to watch=true	Observed result
No RV or initial-event options	400 BadRequest: unsupported sendInitialEvents=true
resourceVersion=0	Same 400
resourceVersion=<fresh opaque token from GET>	200
sendInitialEvents=false without resourceVersionMatch	422 Invalid

The last probe disproved the prediction that explicit false would work. Generic validation checks whether SendInitialEvents is present, not whether it is true, and requires resourceVersionMatch=NotOlderThan (vendor/k8s.io/apimachinery/pkg/apis/meta/internalversion/validation/validation.go:56-58). Direct storage acceptance of false is therefore not sufficient to establish HTTP acceptance.

The diagnostic packet remains FAIL because that predicted 200 was not observed. Its post-probe unchanged-object/build-count check was not reached. The four probes were GET/watch requests; no backend mutation was requested. Owned clients stopped and recorded media was preserved. This is not an exhaustive matrix of all possible query combinations, and template HTTP behavior remains untested.

The documentation correction will recommend the observed working form: supply the current nonzero opaque resourceVersion, omit initial-event/RV-match options, and explain that storage ignores the token only after request options are accepted. No replay or initial-event delivery is promised. There will be no diagnostic rerun or full-suite relaunch to relabel these results green.

---

Generated with xum • Model: coder:openai/gpt-6-astra • Thinking: xhigh

[ThomasK33]

Completed by #119 and #115

#119 landed as 116e2ecf5933cf1710f76a19590726d8105e4d8b. The merged tree is identical to the reviewed head. Together with the watch documentation from #115, this meets the acceptance criteria:

1. Tests through the generic request path, both resources. TestIntegrationWatchOptionsThroughGenericAPIServer sends real HTTP watch requests through the in-process aggregated API server, so they pass through option decoding, WatchList defaulting and validation before storage. It covers coderworkspaces and codertemplates. Every case expects exactly one status:

   Case (watch=true plus)	Result (both resources)
   nothing, or resourceVersion=0	400: sendInitialEvents=true is not supported
   resourceVersion=<token from GET>	200, stream opens
   sendInitialEvents=false	422: requires resourceVersionMatch=NotOlderThan
   sendInitialEvents=false&resourceVersionMatch=NotOlderThan	400: resourceVersionMatch not supported
   sendInitialEvents=true&resourceVersionMatch=NotOlderThan	400: sendInitialEvents=true is not supported
   resourceVersionMatch=NotOlderThan, with or without a token	422: forbidden unless sendInitialEvents is provided

   Templates behave the same as the workspace results observed live during 🤖 fix: make workspace resource versions reflect backend changes #109. The test asserts that the WatchList gate is enabled in the normal server configuration; it does not force it.

2. Documented request form. docs/how-to/deploy-aggregated-apiserver.md (from 🤖 fix: make workspace resourceVersion a fingerprint of the returned representation #115) describes the supported event-only watch: pass the current token as resourceVersion and omit the other options. It also covers why the token is ignored once accepted, the lack of replay and out-of-band events, and the 400/422 cases. The tests confirm that text, so no docs change was needed.

3. Honest initial-event semantics. No behaviour changed. A mutation check that removed the storage's sendInitialEvents=true rejection made the affected cases fail for both resources.

4. 🤖 fix: make workspace resource versions reflect backend changes #109's watch explanation stays accurate and matches the tested behaviour.

5. Gates and evidence.

   • Local gates passed on the exact head. The PR carries a live recording of the test run, labeled as an in-process HTTP test with a mock Coder backend.
   • PR CI passed, including the Kind E2E (12 cases). The merge-queue run on the landed commit also passed, with the apiserverapp tests and all 12 E2E cases.
   • ghcr.io/coder/coder-k8s:main was published from 116e2ecf for linux/amd64 and linux/arm64.
   • The tests use an in-process server and mock backend, so no cluster resources needed cleanup.

The earlier bounded diagnostic stays recorded as failed, because its prediction for sendInitialEvents=false was wrong. Closing as completed.

---

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh • Cost: $1422.27