Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions .github/workflows/klaude-candidate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ jobs:
env:
# Repair attempts after the initial update; the baseline is already published.
MAX_REPAIRS: '3'
KLAUDE_BRANCH: klaude/auto-${{ inputs.candidate }}
KLAUDE_TEST_NAME: klaud-${{ github.run_id }}-${{ inputs.candidate }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
Expand Down Expand Up @@ -102,8 +103,9 @@ jobs:
The pinned image must run as shipped: no engine/serving-stack patches, source or
container-file rewrites, site-packages edits, monkey-patching, or forked/rebuilt engine wheels.
If one is required, reject the image as incompatible.
Keep the PR draft. After a targeted attempt passes, append one valid perf-changelog.yaml
entry at the physical tail with the PR URL; preserve all prior bytes and validate it.
Keep the PR draft through targeted attempts. After one passes, append one valid
perf-changelog.yaml entry at the physical tail with the PR URL; preserve all prior
bytes and validate it.

Use uv and applicable InferenceX checks. Check every exact telemetry cluster from the
current recipe/runner configuration with repeated --cluster arguments:
Expand Down Expand Up @@ -147,10 +149,11 @@ jobs:
healthy work merely to finish the session. The read-only Stop hook checks owned run status;
it does not handle repairs or reporting for you. Do not disable it or alter its run marker.

After a targeted attempt succeeds, commit/push the changelog entry and apply
full-sweep-enabled as the sole sweep-related label. Wait for its run-sweep.yml run
on the exact PR head to finish successfully;
this reusable PR sweep is the final validation. If it exposes a candidate failure, repair within
After a targeted attempt succeeds, commit/push the changelog entry, mark the PR ready
with gh pr ready (this does not request review), and apply full-sweep-enabled as the sole
sweep-related label. Wait for its run-sweep.yml run on the exact PR head to finish
successfully with reusable artifacts; this is the final validation. If it fails, remove
full-sweep-enabled and return the PR to draft before any repair push, then repair within
the remaining budget and repeat. Do not stage results, post /reuse-sweep-run, request review or merge.

PR and final report: English/Simplified Chinese Markdown table, one row for the baseline
Expand Down
9 changes: 9 additions & 0 deletions KLAUD_DEBUG.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,15 @@ as occupied. When capacity is the stop reason, cancel and confirm owned runs,
close the draft PR, and delete its remote Klaud branch so a later sweep can
select the candidate again. Closing the PR alone does not make it eligible.

### 7.3 Final reusable sweeps require a ready PR

`run-sweep.yml` skips PR jobs while the PR is a draft. After targeted validation,
append the changelog entry, mark the PR ready, then apply `full-sweep-enabled`.
If that sweep fails, remove the label and return the PR to draft before pushing
a repair, or each intermediate push starts another full sweep. The Klaud Stop
hook tracks a labeled final sweep by candidate branch and exact head SHA and
requires a successful run with reusable artifacts.

---

## 8. gh CLI gotchas
Expand Down
6 changes: 3 additions & 3 deletions docs/klaude.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ The `klaude-plan` artifact explicitly includes only `candidates.json`, `open-prs

After checkout and context preparation, the candidate workflow hands control to Klaud Cold with `CLAUDE_PAT`, `ANTHROPIC_API_KEY` and the private API read key. Klaud Cold resolves the public observation to one live master-config family, checks the current image and existing PRs, and verifies the reviewed target IDs and their live capacity **before edits or branch/PR creation**. It then claims its branch before spending GPU time, makes a real change and opens a draft PR. Every generated PR title starts with `[Klaud Cold] ` followed by an English / Simplified Chinese description. It never @mentions GitHub users/teams or requests reviews/re-reviews; those are automatic. It rechecks open PRs immediately before claiming the branch because review is a snapshot, not a lock against new human PRs. Ambiguous, retired or already updated candidates stop without a sweep. It owns commits, pushes, dispatches, monitoring, diagnosis, repairs and bilingual PR updates within the same session.

The prompt asks Klaud Cold to measure only updated-image attempts with existing `e2e-tests.yml` on `main`, passing the exact measured SHA through `inputs.ref` and the complete generated `test-config` family command through `generate-cli-command`. It reads current `configs/*-master.yaml`, `configs/runners.yaml` and the existing matrix-generator CLI instead of maintaining another recipe catalog. Default evals, all recipe points, physical `nodes:N` labels, MTP chat templates and artifact contracts stay intact. Targeted repair attempts use no sweep label. Once a targeted attempt passes, Klaud appends the required `perf-changelog.yaml` entry at the physical tail with the PR URL, preserving all prior bytes, and applies `full-sweep-enabled` as the sole sweep-related label. It waits for the resulting `run-sweep.yml` run on the exact PR head to succeed; this final PR sweep produces artifacts eligible for staging and reuse. Klaud does not stage them, authorize reuse, request review or merge. If the full sweep exposes a candidate failure, it repairs within the remaining budget and repeats the final sweep.
The prompt asks Klaud Cold to measure only updated-image attempts with existing `e2e-tests.yml` on `main`, passing the exact measured SHA through `inputs.ref` and the complete generated `test-config` family command through `generate-cli-command`. It reads current `configs/*-master.yaml`, `configs/runners.yaml` and the existing matrix-generator CLI instead of maintaining another recipe catalog. Default evals, all recipe points, physical `nodes:N` labels, MTP chat templates and artifact contracts stay intact. Targeted repair attempts use no sweep label and keep the PR draft. Once one passes, Klaud appends the required `perf-changelog.yaml` entry at the physical tail with the PR URL, preserving all prior bytes, marks the PR ready without requesting review and applies `full-sweep-enabled` as the sole sweep-related label. It waits for the resulting `run-sweep.yml` run on the exact PR head to succeed with reusable artifacts. If the final sweep fails, Klaud removes the label and returns the PR to draft before any repair push, then repairs within the remaining budget and repeats. Klaud does not stage results, authorize reuse, request review or merge.

Before every dispatch, `check-capacity --cluster ID` checks the exact targets; repeat `--cluster` for every possible target. Exit 0 requires all targets to pass freshness, availability and below-20% utilization checks. If capacity is the stop reason, Klaud cancels and confirms its owned runs, closes any draft PR and deletes its remote Klaud branch so a later sweep can retry the candidate. It does not wait for recovery, promise automatic continuation or create a placeholder PR. The command prints no capacity details.

Expand All @@ -51,7 +51,7 @@ The baseline comes from the **public dashboard API at `https://inferencex.semian

Dispatching a run or opening a draft is not completion. Stay in the same session with `gh run watch --interval 60`, resuming after tool timeouts, and inspect job-level state because queued workflows can contain running jobs. A failed benchmark matrix may leave eval jobs active. Diagnose the first server error rather than teardown symptoms; repair within the existing scope, budget and capacity rule. A tool denial calls for an allowed tool or command, not early success. Finish the PR table with terminal attempt results, then report the stop reason, repair count, confirmed child completion and PR URL. Never end with a promise to monitor later or cancel healthy work just to stop.

The [Claude Code Stop hook](https://code.claude.com/docs/en/hooks#stop) runs `python -m utils.klaude check-stop`. It reads every page of `e2e-tests.yml` dispatches since candidate preparation and matches `e2e Test - $KLAUDE_TEST_NAME`. Each attempt must use the exact unchanged marker `klaud-<parent-run-id>-<candidate-id>`. Unfinished matches or unavailable/incomplete listings block a normal stop and tell the same agent to continue. Other candidates' runs are ignored. No runs or all-terminal runs allow stopping. The hook performs no dispatch, cancellation, repair or model call. GitHub job limits, Claude's built-in Stop-hook loop cap, API errors, interrupts and abrupt termination can still prevent completion; this is not an external supervisor. No custom timeout or continuation budget is added.
The [Claude Code Stop hook](https://code.claude.com/docs/en/hooks#stop) runs `python -m utils.klaude check-stop`. It reads every page of `e2e-tests.yml` dispatches since candidate preparation and matches `e2e Test - $KLAUDE_TEST_NAME`. Each attempt must use the exact unchanged marker `klaud-<parent-run-id>-<candidate-id>`. Unfinished matches or unavailable/incomplete listings block a normal stop and tell the same agent to continue. Once the candidate's open PR has `full-sweep-enabled`, the hook also follows `run-sweep.yml` by candidate branch and exact head SHA. It blocks while the PR is draft, the exact run is absent or unfinished, the run fails, or a successful run lacks reusable benchmark/eval artifacts. Other candidates' runs are ignored. The hook performs no dispatch, cancellation, repair or model call. GitHub job limits, Claude's built-in Stop-hook loop cap, API errors, interrupts and abrupt termination can still prevent completion; this is not an external supervisor. No custom timeout or continuation budget is added.

After the action, `diagnostics` reuses the planner's sanitizer to upload only `candidate-diagnostics.json` in `klaude-candidate-<candidate-id>`. It retains the action outcome, allowlisted termination subtype, boolean error status, numeric duration/turn/cost metrics and fixed permission-denial categories. Unknown subtypes become `unknown`; missing or unreadable execution files are recorded as unavailable. Raw messages, results, commands, paths, errors, credentials and private telemetry are excluded. These steps use `always()` when the runner is still available; they never update the PR or resume Claude. Planner diagnostics gain the same termination fields.

Expand Down Expand Up @@ -88,7 +88,7 @@ The app also has useful **unpublished UI readers**: `/api/unofficial-run` normal

Klaud Cold is instructed to change only the selected master family's image and its already referenced, unshared srt-slurm recipe YAML images/backend compatibility settings. Model, precision, topology, speculative decoding, workload, commands, resources and recipe references stay fixed. `model.container` and any `identity.container.image` must match the master image. Shared scripts, launchers, libraries, workflow/control files and unrelated families stay unchanged. The pinned image must run as shipped: engine/serving-stack patches, source or container-file rewrites, site-packages edits, monkey-patching and forked/rebuilt wheel overlays are prohibited. If a candidate image requires one, Klaud rejects it as incompatible. Klaud Cold runs focused checks using **uv**; there is no separate patch validator after the agent.

After targeted validation passes, Klaud appends the required changelog entry without altering historical bytes. The candidate remains a draft through the final `full-sweep-enabled` run. Existing checks and human review remain in force; no automatic staging, reuse authorization, review request, merge or policy bypass is authorized.
After targeted validation passes, Klaud appends the required changelog entry without altering historical bytes, marks the candidate ready because `run-sweep.yml` skips draft PRs, then applies `full-sweep-enabled`. A failed final sweep returns the PR to draft and removes the label before repairs. Existing checks and human review remain in force; no automatic staging, reuse authorization, review request, merge or policy bypass is authorized.

## Workflow operation and credentials

Expand Down
Loading