ci: nothing execs into k3s before it has written its kubeconfig

.github/actions/e2e-boot/action.yml

@@ -58,8 +58,9 @@ runs:
         ' > /tmp/kueue-download.log 2>&1 &
         echo $! > /tmp/kueue-download.pid
 
+        # No docker exec before k3s writes its kubeconfig: one during its root cgroup evacuation kills the kubelet
         nohup bash -c '
-          timeout 120 bash -c "until docker exec k3s crictl info >/dev/null 2>&1; do sleep 2; done"
+          timeout 120 bash -c "until [ -s /etc/rancher/k3s/k3s.yaml ] && docker exec k3s crictl info >/dev/null 2>&1; do sleep 2; done"
           docker exec k3s crictl pull docker.io/library/python:3.14-slim
           echo $? > /tmp/crictl-python.exit
         ' > /tmp/crictl-python.log 2>&1 &
@@ -72,7 +73,7 @@ runs:
       run: |
         KUEUE_VERSION="${KUEUE_VERSION:-v0.20.1}"
         nohup bash -c '
-          timeout 120 bash -c "until docker exec k3s crictl info >/dev/null 2>&1; do sleep 2; done"
+          timeout 120 bash -c "until [ -s /etc/rancher/k3s/k3s.yaml ] && docker exec k3s crictl info >/dev/null 2>&1; do sleep 2; done"
           docker exec k3s crictl pull registry.k8s.io/kueue/kueue:'"$KUEUE_VERSION"'
           echo $? > /tmp/crictl-kueue.exit
         ' > /tmp/crictl-kueue.log 2>&1 &

docs/operations/cicd.md

@@ -78,7 +78,7 @@ The pushes run in parallel and each failure is reported on its own. Fork PRs can
 
 Each E2E job gets its own runner with k3s and the full compose stack, pulling the images it just built. Two composite actions set that up, and the frontend job installs Node and Playwright between them so that work overlaps with k3s booting.
 
-`e2e-boot` logs in to GHCR, starts `docker compose pull` and then `mongo redis shared-ca kafka` in the background, and starts k3s in a privileged container named `k3s`, from the `rancher/k3s` image, with its network policy controller off. No E2E test checks policy enforcement, and the worker still creates the NetworkPolicy object. The container publishes 6443 and mounts the runner's `/etc/rancher/k3s`, so the kubeconfig lands where the cert-generator service reads it, the same as with a host install. k3s runs in a container because its pods then get their network interfaces inside it. With k3s on the runner itself, every executor pod changed the runner's network, and Chromium aborted Playwright navigations in flight with `net::ERR_NETWORK_CHANGED`. Also in the background, it downloads the Kueue manifest and pre-pulls `python:3.14-slim`, the default runtime and the only one the tests execute, into k3s with `docker exec k3s crictl`, so the first executor pods don't wait on an image pull. Jobs that install Kueue also pre-pull `registry.k8s.io/kueue/kueue` that way, and `e2e-ready` waits for that pull before it waits for the controller, since the image alone once took over five minutes from registry.k8s.io.
+`e2e-boot` logs in to GHCR, starts `docker compose pull` and then `mongo redis shared-ca kafka` in the background, and starts k3s in a privileged container named `k3s`, from the `rancher/k3s` image, with its network policy controller off. No E2E test checks policy enforcement, and the worker still creates the NetworkPolicy object. The container publishes 6443 and mounts the runner's `/etc/rancher/k3s`, so the kubeconfig lands where the cert-generator service reads it, the same as with a host install. k3s runs in a container because its pods then get their network interfaces inside it. With k3s on the runner itself, every executor pod changed the runner's network, and Chromium aborted Playwright navigations in flight with `net::ERR_NETWORK_CHANGED`. Also in the background, it downloads the Kueue manifest and, once k3s has written its kubeconfig, pre-pulls `python:3.14-slim`, the default runtime and the only one the tests execute, into k3s with `docker exec k3s crictl`, so the first executor pods don't wait on an image pull. Nothing execs into the container before that kubeconfig exists: at startup k3s moves every process out of the container's root cgroup and then enables the cgroup controllers there, and a `docker exec` landing in between made that fail with "device or resource busy", so the kubelet exited and the node never came up. Jobs that install Kueue also pre-pull `registry.k8s.io/kueue/kueue` that way, and `e2e-ready` waits for that pull before it waits for the controller, since the image alone once took over five minutes from registry.k8s.io.
 
 `e2e-ready` copies k3s's kubeconfig for the runner's kubectl, waits for the node, writes `backend/kubeconfig.yaml` pointing at `host.docker.internal:6443` and creates the `integr8scode` namespace. Unless `install-kueue` is `false`, it applies the Kueue manifest, waits for the controller and creates the `ResourceFlavor`, `ClusterQueue` and `LocalQueue` (`kueue.x-k8s.io/v1beta2`), then waits for the image pre-pulls. Kueue starts with a certificate bootstrap that answers `/readyz` with 404 until the kubelet has mounted its new certificate secret, and it checks for the file with an exponential backoff, so on a slow run the controller takes over two minutes. If it isn't Available within 300 seconds, the step prints the `kueue-system` pods, their descriptions, events and the controller's last 300 log lines before it fails. It copies config.test.toml over config.toml, waits for the background infra start, runs `docker compose up -d --no-build` and polls `/api/v1/health/live` (and the frontend, when asked). It then waits until the five Kafka consumer groups report `Stable` in `kafka-consumer-groups --describe --state`. On a fresh Kafka most of them start at the newest message, so a request sent while they were still joining would never be read, and an execution test would time out.