fix(workflow): disambiguate "workflow" to mean a swamp workflow#1478
Conversation
The Claude Code harness ships a built-in Workflow tool that auto-triggers on the bare keyword "workflow" at the input-parsing layer. That collides with swamp's own first-class workflow concept (a declarative YAML DAG of model-method steps authored via `swamp workflow create`). When a user drops the word "swamp" the harness can win and spin up agent task lists, worktrees, or cron/remote scheduling instead of authoring a durable workflow. Pin the term in the generated user-facing CLAUDE.md (written during repo_init / repo_update) with a new rule that maps "workflow" / "automate" / "orchestrate" / "automated job" to the swamp meaning and gates agent orchestration behind explicit opt-in vocabulary. Phrasing stays tool-agnostic since the same template serves Cursor, opencode, codex, and kiro alongside Claude. Mirror the guidance into the swamp-dev CLAUDE.md (Claude-Code-only, so it names the literal harness tools). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rd clash Claude Code's built-in Workflow tool auto-triggers on the bare keyword "workflow" at the harness input layer, before any skill is evaluated. The swamp-workflow skill's trigger list made this worse by listing the same bare verbs the harness cues on — automate, automation, orchestrate, create/run/execute workflow. Remove those bare collision tokens and namespace every trigger to the swamp concept so the two-word phrase "swamp workflow" becomes the discriminator. Add an explicit NOT-clause to the description and a "## Skill boundary" section to the body stating the skill is unrelated to the Claude Code Workflow tool, agent task lists, worktrees, and cron/remote-agent scheduling. Lock the boundary into the eval suite: re-namespace the cron-shaped positive seed and add adversarial should_trigger:false seeds for the generic-task, schedule-agent, and worktree phrasings. Pairs with the CLAUDE.md rule from the previous commit, which gives the model in-context authority to still route un-namespaced phrasings to the skill — covering the under-triggering risk this trade introduces. Verified: tessl skill review 0.925 (both judges above threshold); promptfoo trigger evals pass for all swamp-workflow seeds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Adversarial Review
Critical / High
None found.
Medium
-
Positive trigger eval may be fragile after trigger namespacing —
trigger_evals.jsonline 29:"Execute the nightly-sync workflow and show me the results"hasshould_trigger: true, but the SKILL.md description no longer lists bare"execute workflow"as a trigger — it now requires"swamp workflow run","swamp workflow validate", etc. The word "nightly-sync" doesn't contain "swamp", so trigger matching depends on whether the eval harness matches the broader description text ("running workflows") rather than the explicit trigger list. The PR body says promptfoo evals all pass, so this may be fine empirically, but the gap between the tightened trigger phrases and the unchanged positive evals could make the eval suite less reliable as a regression guard. Similar concern applies to lines 9 ("Show me the run history for the deployment workflow") and 23 ("Inspect the execution logs from the last workflow run") — neither uses "swamp" as a prefix. -
Inconsistency between CLAUDE.md and generated CLAUDE.md on "RemoteTrigger" — The dev
CLAUDE.md(line 19) referencesRemoteTriggerin the list of harness tools to avoid. The generatedrepo_service.tstemplate (line 819) also references "remote agent" as an explicit opt-in keyword but doesn't mentionRemoteTriggerby tool name. The SKILL.md boundary section (new lines 28-32) also referencesRemoteTrigger. Since the generated CLAUDE.md is tool-agnostic (serves Cursor, codex, etc.), mentioningRemoteTriggerin the dev CLAUDE.md but omitting it from the generated CLAUDE.md is consistent with intent — but worth confirming this asymmetry is deliberate.
Low
-
Eval seed
"Create a swamp workflow that runs the data-sync model every night."changed from generic to namespaced —trigger_evals.jsonline 19: The original eval"Create an automated job that runs the data sync every night"tested the skill's ability to match the ambiguous phrasing that motivated this PR. Replacing it with the swamp-prefixed version removes the one positive eval that exercised the "bare verb → swamp workflow" routing the CLAUDE.md rule is meant to enable. This weakens the eval suite's coverage of the intended behavior, though it's consistent with the trigger-narrowing strategy. -
SKILL.md line count — The SKILL.md body is already at ~640 lines (the
## Skill boundarysection adds 9 lines to an already-over-500-line file). CLAUDE.md says "Keep SKILL.md body under 500 lines." This pre-dates this PR but the PR makes it slightly worse. Non-blocking since it's pre-existing.
Verdict
PASS — This PR is pure documentation/guidance with no runtime code changes beyond a template string addition that has test coverage. The changes are internally consistent and well-tested. The trigger eval coverage concern is real but the PR body confirms promptfoo evals pass, and the CLAUDE.md rule provides a second routing layer for bare "workflow" phrasings. No blocking issues.
There was a problem hiding this comment.
Code Review
Blocking Issues
None.
Suggestions
- Pre-existing: SKILL.md exceeds 500-line body limit. The swamp-workflow SKILL.md is already ~642 lines before this PR (which adds 9 more). CLAUDE.md says to keep it under 500 lines and split into
references/files. Not introduced by this PR, but worth addressing in a follow-up.
Summary
Clean, well-scoped PR. The two-layer approach (CLAUDE.md rule + skill trigger namespacing) is a sensible defense-in-depth strategy for the keyword collision. The repo_service.ts changes stay within the Domain Service's responsibility of generating CLAUDE.md content, and the new test follows existing patterns exactly. No DDD, security, or convention concerns.
Summary
Claude Code ships a built-in Workflow tool that auto-triggers on the bare keyword "workflow" at the harness input layer — before any skill is evaluated. The swamp repo has its own first-class workflow concept (a declarative YAML DAG of model-method steps, authored via
swamp workflow create). These collide: when a user says "create a workflow that runs model X then Y" (dropping "swamp"), the harness can win and spin up agent task lists / worktrees / cron scheduling instead of producing a durableworkflows/*.yamlDAG.This PR closes the gap with two layers of pure-documentation/guidance, no behavior change to the CLI:
1. Generated CLAUDE.md rule (
repo_service.ts) — the user-facing CLAUDE.md written duringrepo_init/repo_updategains a rule pinning "workflow" / "automate" / "orchestrate" / "automated job" to the swamp meaning, gating agent-orchestration mechanisms behind explicit opt-in vocabulary. Phrasing is tool-agnostic since the same template serves Cursor, opencode, codex, and kiro alongside Claude. Existing repos pick it up via the managed-section migration onswamp repo update. The swamp-devCLAUDE.mdmirrors the guidance (Claude-Code-only, so it names the literal harness tools).2. swamp-workflow skill hardening — the skill's trigger list previously listed the same bare verbs the harness cues on (
automate,orchestrate,create/run/execute workflow), actively contributing to the ambiguity. Those are removed and every trigger namespaced to the swamp concept, with an explicit NOT-clause in the description and a## Skill boundarysection in the body. Adversarialshould_trigger: falseeval seeds lock the boundary in.The two layers are intentionally paired: namespacing the skill triggers trades "false-positives-to-harness" for "false-negatives-to-nothing", and the CLAUDE.md rule gives the model in-context authority to still route un-namespaced phrasings to the skill.
Caveat
This is soft model guidance. The harness
workflowkeyword fires above the instruction/skill tier, so on a borderline prompt the model can still be pulled toward the orchestration reading. A hard guarantee would requiredisableWorkflows/permissions.deny: ["Workflow"], which is deliberately out of scope here.Test Plan
deno fmt --check,deno lint— cleandeno run test— 6512 passed, 0 failed (includes a newrepo_service_test.tscase asserting the workflow-disambiguation rule lands in the generated CLAUDE.md)swamp-workflow— 0.925 avg, both judges above the 0.90 thresholddeno run eval-skill-triggers --model sonnet) — allswamp-workflowseeds pass, including the 3 new adversarial negatives ("create a workflow to run these three steps", "schedule a recurring agent to run nightly", "set up a worktree workflow"). The 3 unrelated suite failures (swamp-data, swamp-vault) are pre-existing onmainand out of scope.🤖 Generated with Claude Code