From 4d7b9a1b44107309faf67e6020f1de23b4d05674 Mon Sep 17 00:00:00 2001 From: River Lynn Bailey Date: Mon, 27 Jul 2026 07:58:39 -0600 Subject: [PATCH 1/3] docs(research): draft the context-engineering-guidelines research report (pre-validation) --- ...context-engineering-guidelines-claude-5.md | 258 ++++++++++++++++++ 1 file changed, 258 insertions(+) create mode 100644 docs/research/context-engineering-guidelines-claude-5.md diff --git a/docs/research/context-engineering-guidelines-claude-5.md b/docs/research/context-engineering-guidelines-claude-5.md new file mode 100644 index 00000000..19d78b16 --- /dev/null +++ b/docs/research/context-engineering-guidelines-claude-5.md @@ -0,0 +1,258 @@ +# Research: Anthropic's New Context Engineering Guidelines and What They Mean for Han's Plugin-Building Guidance + +This report answers an open-ended question: what did Anthropic publish as its new context engineering guidance for +Claude 5-generation models, and how should the skill-building and agent-building guidance in `han-plugin-builder` be +updated in response? Evidence mode: strict. + +## Summary + +Anthropic has published new context engineering guidance for its Claude 5-generation models, and most of it confirms +what Han's plugin-building guidance already teaches. The new guidance names six shifts: write instructions that state a +goal and trust the model's judgment instead of spelling out rigid rules, design tool interfaces so the parameters +explain themselves instead of relying on usage examples, load context progressively instead of up front, remove +instructions that repeat across surfaces, let the model save its own memory instead of maintaining memory files by +hand, and point the model at code, test suites, and rubrics instead of long prose descriptions. Anthropic reports it +cut its own coding assistant's system prompt by more than eighty percent with no measured loss. + +Han's guidance already covers progressive disclosure, lean descriptions, context budgets, and single-role agents, so +no restructuring is needed. The recommended update is a small, targeted one: refresh the guidance file that describes +how each Claude model wants its instructions written, because it was last checked four days before the new +announcement and one of its claims now points the opposite direction from the new guidance, and fold the handful of +genuinely new rules into the existing files where each belongs. The finding rests on Anthropic's own announcement, +confirmed in its specifics by independent press coverage, and on a direct inventory of Han's guidance files; the +quantitative claims trace back to Anthropic's own internal measurements only. Overall: well-corroborated on the +direction, single-source on the numbers. + +- **Confidence:** Medium-High + +## Research Results + +### The six shifts Anthropic announced for Claude 5-generation models + +Anthropic's announcement (A1) names six changes to how context should be engineered for Claude 5-generation models, +each framed as a before-and-after against prior practice. The six shifts are corroborated by independent secondary +coverage (A11), though all coverage derives from the same original Anthropic disclosure: + +1. **Prescriptive rules become judgment-based instructions.** The announcement's own example: the old rule "Never + write multi-paragraph docstrings" becomes "Write code that reads like the surrounding code: match its comment + density, naming, and idiom" (A1, quoted verbatim in A11). +2. **Tool usage examples become expressive tool interfaces.** Parameter names and enumerations should themselves hint + at correct usage, replacing worked examples in tool descriptions (A1). +3. **Upfront context becomes progressive disclosure.** Verification and review guidance moves out of the system prompt + into Skills the model loads selectively, and tools use deferred loading for non-essential functionality (A1). +4. **Duplicated instructions are removed.** An instruction that appears in both the system prompt and a tool + description is kept in one place only (A1). +5. **Manual memory files become automatic memory.** The model saves its own memory instead of requiring hand-edited + CLAUDE.md updates (A1). The specific prior mechanic replaced (a "#" hotkey) is stated only in A1 [single-source]. +6. **Markdown specs become richer references.** Code, HTML artifacts, test suites, and rubrics are preferred over + prose descriptions of desired behavior (A1). + +The headline quantitative claim is that Anthropic removed over eighty percent of Claude Code's system prompt with no +measured loss on coding evaluations (A1, restated by A11). Secondary press adds a more specific figure, roughly 800 +tokens down to about 164, and reports an incoming `claude doctor` / `/doctor` command for rightsizing CLAUDE.md files +and skills (A11). Because every outlet derives from the same Anthropic disclosure, these figures are convergent +secondary coverage of one source, not independent measurement [single-source]. + +### The established context-engineering corpus behind the new rules + +The new announcement sits on top of a consistent body of Anthropic engineering guidance (A2 through A8) that predates +it. Its core claims, corroborated across several independent Anthropic artifacts: + +- **Context is a finite, depleting resource.** Model performance degrades as tokens accumulate ("context rot"), so + the goal is the smallest set of high-signal tokens that produces the desired outcome (A2, corroborated + operationally by A3's high-signal tool responses, A6's automatic stale-tool-result clearing, and A8's CLAUDE.md + pruning test "would removing this line cause a mistake?"). +- **System prompts sit at the right altitude.** Specific enough to guide, flexible enough to avoid brittle + if-else logic, organized with structural markup (A2). The framing is A2's own [single-source], but A1's docstring + example is a concrete instance of the same shift. +- **Tools avoid overlap and pass a definitiveness test.** A human should be able to tell which tool applies; bloated + tool sets are a named failure mode (A2, corroborated by A3's consolidation and naming rules and A5's + prefer-specialized-tools heuristic). +- **Curated canonical examples beat exhaustive edge-case lists** (A2) [single-source]. +- **Just-in-time retrieval beats pre-loading, with a hybrid recommended.** Lightweight identifiers loaded at runtime, + critical data loaded up front (A2, corroborated by A7's three-level skill loading model and A8's split of + frequently-needed material into CLAUDE.md and occasional material into skills). +- **Compaction and external memory handle long horizons.** Summarize and reinitiate, clear stale tool results first, + and keep structured notes outside the context window (A2, corroborated by A6's shipped context-editing and memory + tool features). A6's percentage improvements are Anthropic's internal benchmarks [single-source]. +- **Subagents get clean contexts and return condensed summaries.** A lead agent decomposes work; each subagent needs + an explicit objective, output format, tool guidance, and boundaries, or agents duplicate work (A2, A5, corroborated + at the product level by A8). A5's figures, including that multi-agent systems use roughly fifteen times the tokens + for a 90.2 percent quality gain on complex research, are internal single-study numbers [single-source]. +- **Skills are the unit of progressive disclosure.** Name and description are the trigger signal; the SKILL.md body + loads on trigger; supplementary files load only when a sub-scenario needs them; scripts handle deterministic + operations (A7, corroborated by A8). + +Anthropic's harness post (A9) adds three named failure modes for long-running agent work: agentic laziness, declaring +a multi-part job done early; self-preferential bias, favoring one's own output when judging it; and goal drift, losing +the objective across many turns. It also names workflow patterns including adversarial verification, tournament, and +loop-until-done, and advises reserving heavy multi-agent machinery for high-value tasks [single-source]. A companion +post on working with Claude Fable (A10) recommends practices such as a deliberate blind-spot pass, prototypes over +detailed specs for subjective work, and treating source code as the best reference [single-source]. + +### What Han's guidance corpus already covers + +The `han-plugin-builder` guidance corpus (48 files, inventoried at A13) already encodes most of the established +corpus, often with more operational specificity than Anthropic's own posts: + +- Three-level progressive disclosure with a 500-line SKILL.md ceiling (A16, matching A7). +- Context-rot rationale, attention competition, lost-in-the-middle position effects, and a 15 to 40 percent + utilization peak (A15, matching and extending A2). +- Description budgets: a 1024-character target, the Claude Code 1536-character per-entry cap, and the roughly one + percent listing budget (A16). +- Single-role agents that generate or evaluate, never both (A17), which matches A9's self-preferential-bias failure + mode from a different angle. +- Multi-agent economics with a 45 percent saturation threshold sourced to a 2025 Google, DeepMind, and MIT study + (A17), consistent with A5's reserve-for-high-value-tasks advice. +- Scripts for deterministic operations, dispatch namespacing, and just-in-time guidance reading (A13, A18, matching + A7). + +### Where the new guidance and Han's docs diverge + +Five deltas surfaced, one of them a direct tension: + +1. **A dated per-model claim now points the wrong way.** `per-model-authoring.md` (A14) was last checked against + Anthropic's published guidance on 2026-07-20, four days before the new announcement. It states that Opus 4.8 and + Sonnet 5 "follow instructions literally and do not generalize on their own, so they want each behavior spelled out" + while Fable 5 wants short, goal-based instructions. A1's shift to judgment-based instructions is framed for the + whole Claude 5 generation, which includes Sonnet 5. This is a codebase-versus-web conflict to resolve by + re-verification against Anthropic's current per-model guidance, not by silently adopting either side. +2. **No explicit de-duplication rule.** The corpus separates description from body content but has no rule stating + that an instruction must not repeat across surfaces (skill body, agent definition, vendored references), which A1 + names as its own shift. +3. **Code-based references are only partially covered.** The corpus prescribes scripts for deterministic operations, + but A1's broader preference for test suites, rubrics, and code artifacts over prose specifications has no + counterpart in the guidance. +4. **Automatic memory and the rightsizing tool are unrepresented.** Neither automatic memory-saving (A1) nor the + `/doctor` command (A11) [single-source] appears in the guidance. The `/doctor` claim rests on secondary press only + and is not yet doc-worthy as fact. +5. **Two of three named failure modes are unrepresented.** Self-preferential bias is already encoded as the + single-role rule; agentic laziness and goal drift (A9) [single-source] are not named anywhere in the corpus. + +### Evidence-gathering integrity note + +One fetched page (A8) contained directive-style text ahead of its article body instructing the fetcher to retrieve a +documentation index file. Per this skill's rules, that text was recorded as a claim and not acted on, and the +web-facing agents ran with no repository or user context to surrender. + +## Options to Consider + +### O1: Targeted delta update to the existing guidance files + +- **What it is:** Refresh `per-model-authoring.md` against the new announcement, re-verifying the Sonnet 5 + literal-instructions claim; fold each confirmed new rule into the existing decision-scoped file where it belongs + (judgment-based altitude into the instruction-writing guidance, an explicit de-duplication rule, code-based + references guidance); cite A1 with its retrieval date; carry single-source items as caveated notes. +- **Trade-offs:** Smallest change and preserves the corpus's decision-scoped structure, but touches several files and + requires judgment about where each rule lands. +- **Rests on:** (A1, A11) for the shifts; (A13, A14, A15, A16, A17) for what the corpus holds today. +- **Evidence status:** corroborated + +### O2: One new dedicated context-engineering reference doc + +- **What it is:** Add a single new reference file carrying the Claude 5 shifts, cross-linked from the existing files. +- **Trade-offs:** One clean edit, but the corpus routes authors to guidance by decision point, and a topic-shaped file + would either duplicate content the decision-scoped files already carry or leave authors two places to look. That + duplication is exactly what A1's fourth shift warns against, and it breaks the repo's one-canonical-source + convention. +- **Rests on:** (A1), (A13). +- **Evidence status:** corroborated + +### O3: Full corpus restructure around the new framing + +- **What it is:** Reorganize the guidance corpus around "thin prompts, thick artifacts" as its central organizing + idea. +- **Trade-offs:** The cost far exceeds the evidence. Most of the new guidance's substance is already present in the + corpus, and the strongest justification for radical change, the eighty percent prompt cut, is an uncorroborated + self-reported figure [single-source]. +- **Rests on:** (A1, A11). +- **Evidence status:** single-source (caveated) for the claims that would justify the scale of change + +### O4: No change; wait for official documentation + +- **What it is:** Treat the blog post as directional and wait for Anthropic's official docs to confirm before editing + anything. +- **Trade-offs:** Zero effort now, but it knowingly leaves a dated per-model file whose Sonnet 5 claim is in tension + with corroborated new guidance, and the corpus's own convention is to date-stamp and refresh model-behavior claims. +- **Rests on:** (A14) for the staleness it would leave in place. +- **Evidence status:** corroborated (the staleness is verifiable; the case for waiting is the weakness) + +## Recommendation + +- **Recommendation:** O1, the targeted delta update. Start with `per-model-authoring.md`: re-verify its Sonnet 5 and + Opus claims against Anthropic's current published guidance and update its last-checked date, since a guidance file + that points authors the opposite direction from current upstream guidance is the one live defect found. Then add + the three genuinely new rules to the files that own those decisions: an explicit de-duplication rule, the + preference for code-based references (test suites, rubrics) over prose specifications, and the judgment-over-rules + instruction altitude for Claude 5-generation models. Carry the single-source items (the eighty percent figure, the + `/doctor` command, the named failure modes, the Fable working-style tips) as caveated, dated notes rather than + asserted facts. +- **Evidence basis:** The six shifts rest on corroborated evidence (A1 confirmed in its specifics by A11, and + consistent with the independent established corpus A2 through A8). The claim that Han's corpus already encodes the + established guidance rests on codebase evidence (A13 through A18) cross-checked against the web sources. The + per-model tension rests on dated codebase evidence (A14) against the corroborated A1 shift. The quantitative + figures, the `/doctor` command, and the A9 and A10 material are single-source and are recommended only as caveated + notes, never as the basis of a guidance rule. + +## Validation + +_Pending: adversarial validation findings will be recorded here._ + +## Sources + +| ID | Source | Link / location | Retrieved | Trust class | Summary (one line) | Evidence status | +| --- | --------------------------------------------------- | -------------------------------------------------------------------------------------------- | ---------- | ----------- | ---------------------------------------------------------------------------------------- | -------------------------------------------- | +| A1 | The New Rules of Context Engineering for Claude 5 | | 2026-07-27 | web | Six shifts for Claude 5-generation models; 80% system-prompt cut claim | corroborated by A11 (derivative) | +| A2 | Effective Context Engineering for AI Agents | | 2026-07-27 | web | Context rot; right altitude; JIT retrieval; compaction; note-taking; subagent summaries | corroborated by A3, A5, A6, A7, A8 | +| A3 | Writing Tools for AI Agents | | 2026-07-27 | web | Tool namespacing, consolidation, unambiguous params, actionable errors | corroborated by A2, A4, A5 | +| A4 | Building Effective AI Agents | | 2026-07-27 | web | Workflows vs agents; five composable patterns; start simple; invest in tool design | corroborated by A3, A5 | +| A5 | How We Built Our Multi-Agent Research System | | 2026-07-27 | web | Orchestrator/subagent briefs, effort scaling, parallelism; internal 15x/90.2% figures | corroborated by A2, A8; figures single-source | +| A6 | Context Management on the Claude Developer Platform | | 2026-07-27 | web | Shipped context editing and memory tool; internal percentage gains | corroborated by A2; figures single-source | +| A7 | Equipping Agents for the Real World with Skills | | 2026-07-27 | web | SKILL.md contract; three-level progressive disclosure; scripts for determinism | corroborated by A2, A8 | +| A8 | Claude Code Best Practices | | 2026-07-27 | web | Minimal CLAUDE.md; skills for occasional context; subagents for investigation and review | corroborated by A2, A7; contained directive text (recorded, not followed) | +| A9 | Dynamic Workflows in Claude Code | | 2026-07-27 | web | Failure modes (agentic laziness, self-preferential bias, goal drift); workflow patterns | single source (caveated) | +| A10 | A Field Guide to Claude Fable | | 2026-07-27 | web | Fable-specific working practices: blind-spot pass, prototypes, source as reference | single source (caveated) | +| A11 | Secondary press coverage of A1 | and others | 2026-07-27 | web | Restates the six shifts, ~800-to-~164-token figure, incoming `claude doctor` command | corroborates A1; derivative of the same disclosure | +| A12 | Prompt Engineering Overview (platform docs) | | 2026-07-27 | web | Thin routing page; no substantive claims | single source (thin) | +| A13 | Han guidance corpus inventory | `han-plugin-builder/skills/guidance/` (48 files) | n/a | codebase | Full inventory of skill-building and agent-building guidance | codebase anchor | +| A14 | Per-model authoring guidance | `han-plugin-builder/skills/guidance/references/per-model-authoring.md` | n/a | codebase | Last checked 2026-07-20; Sonnet 5/Opus 4.8 literal vs Fable 5 goal-based instructions | in tension with A1 | +| A15 | Context hygiene guidance | `han-plugin-builder/skills/guidance/references/skill-building-guidance/context-hygiene.md` | n/a | codebase | Attention competition, lost-in-the-middle, 15-40% utilization, compaction budget | consistent with A2 | +| A16 | Progressive disclosure and description budgets | `han-plugin-builder/skills/guidance/references/skill-building-guidance/progressive-disclosure.md` and `skill-description-length.md` | n/a | codebase | Three loading levels, 500-line body ceiling, 1024/1536-char description budgets | consistent with A7 | +| A17 | Agent guidance (domain focus, economics) | `han-plugin-builder/skills/guidance/references/agent-building-guidelines/` | n/a | codebase | Single-role rule, 50-token role identity, 45% multi-agent threshold | consistent with A5, A9 | +| A18 | Builder skills (interview and review checklists) | `han-plugin-builder/skills/skill-builder/SKILL.md` and `agent-builder/SKILL.md` | n/a | codebase | Evidence-first interviews; conformance review checklists mapping to guidance files | codebase anchor | + +### A1: The New Rules of Context Engineering for Claude 5 Generation Models — recommendation-bearing + +- **Link / location:** +- **Retrieved:** 2026-07-27 +- **Trust class:** web +- **Summary:** Anthropic's announcement of six shifts in context engineering for Claude 5-generation models: judgment-based + instructions over prescriptive rules, expressive tool interfaces over usage examples, progressive disclosure over + upfront context, de-duplicated instructions, automatic memory over manual CLAUDE.md edits, and code-based references + over markdown specs. Reports an over-eighty-percent cut to Claude Code's system prompt with no measured evaluation + loss, and recommends keeping CLAUDE.md to repository gotchas while moving situational guidance into Skills. +- **Evidence status:** corroborated by A11 on content; the quantitative figures trace to this single Anthropic disclosure + +### A11: Secondary press coverage — recommendation-bearing + +- **Link / location:** (dated + 2026-07-24), plus + and forum and blog coverage +- **Retrieved:** 2026-07-27 +- **Trust class:** web +- **Summary:** Independent outlets restating the six shifts, the docstring example verbatim, the roughly 800-to-164-token + figure, and an incoming `claude doctor` command for rightsizing CLAUDE.md files and skills. All derive from the same + Anthropic disclosure, so they confirm the announcement's content without independently verifying its measurements. +- **Evidence status:** corroborates A1; derivative, not independent measurement + +### A14: per-model-authoring.md — recommendation-bearing + +- **Link / location:** `han-plugin-builder/skills/guidance/references/per-model-authoring.md` +- **Retrieved:** n/a +- **Trust class:** codebase +- **Summary:** States it was last checked against Anthropic's published guidance on 2026-07-20 for Sonnet 5, Opus 4.8, + and Fable 5. Claims Opus 4.8 and Sonnet 5 follow instructions literally and want each behavior spelled out, while + Fable 5 wants short, goal-based instructions and degrades under exhaustive checklists. The new announcement frames + judgment-based instructions as a property of the whole Claude 5 generation, putting this file's Sonnet 5 claim in + tension with current upstream guidance. +- **Evidence status:** codebase anchor; in tension with A1, resolution requires re-verification From 4928083dff0b7218642484d7373e094d107b020a Mon Sep 17 00:00:00 2001 From: River Lynn Bailey Date: Mon, 27 Jul 2026 08:02:39 -0600 Subject: [PATCH 2/3] docs(research): record the adversarial validation findings and narrow the recommendation to verify-and-caveat --- ...context-engineering-guidelines-claude-5.md | 325 ++++++++++++------ 1 file changed, 218 insertions(+), 107 deletions(-) diff --git a/docs/research/context-engineering-guidelines-claude-5.md b/docs/research/context-engineering-guidelines-claude-5.md index 19d78b16..9a1f4b78 100644 --- a/docs/research/context-engineering-guidelines-claude-5.md +++ b/docs/research/context-engineering-guidelines-claude-5.md @@ -6,32 +6,33 @@ updated in response? Evidence mode: strict. ## Summary -Anthropic has published new context engineering guidance for its Claude 5-generation models, and most of it confirms -what Han's plugin-building guidance already teaches. The new guidance names six shifts: write instructions that state a -goal and trust the model's judgment instead of spelling out rigid rules, design tool interfaces so the parameters -explain themselves instead of relying on usage examples, load context progressively instead of up front, remove -instructions that repeat across surfaces, let the model save its own memory instead of maintaining memory files by -hand, and point the model at code, test suites, and rubrics instead of long prose descriptions. Anthropic reports it -cut its own coding assistant's system prompt by more than eighty percent with no measured loss. - -Han's guidance already covers progressive disclosure, lean descriptions, context budgets, and single-role agents, so -no restructuring is needed. The recommended update is a small, targeted one: refresh the guidance file that describes -how each Claude model wants its instructions written, because it was last checked four days before the new -announcement and one of its claims now points the opposite direction from the new guidance, and fold the handful of -genuinely new rules into the existing files where each belongs. The finding rests on Anthropic's own announcement, -confirmed in its specifics by independent press coverage, and on a direct inventory of Han's guidance files; the -quantitative claims trace back to Anthropic's own internal measurements only. Overall: well-corroborated on the -direction, single-source on the numbers. - -- **Confidence:** Medium-High +Anthropic has published new context engineering guidance for its Claude 5-generation models, and nearly all of it +confirms what Han's plugin-building guidance already teaches. The new guidance names six shifts: write instructions +that state a goal and trust the model's judgment instead of spelling out rigid rules, design tool interfaces so the +parameters explain themselves instead of relying on usage examples, load context progressively instead of up front, +remove instructions that repeat across surfaces, let the model save its own memory instead of maintaining memory +files by hand, and point the model at code, test suites, and rubrics instead of long prose descriptions. Anthropic +reports it cut its own coding assistant's system prompt by more than eighty percent with no measured loss. + +Adversarial validation cut the needed update down to a small one. Two gaps the initial research claimed turned out +not to exist: the guidance already carries a rule against restating what the platform provides, and its +code-reference coverage is broader than first credited. The recommended update is narrow: re-verify the one guidance +file that describes how each Claude model wants its instructions written against Anthropic's official per-model +documentation, refresh its checked date, and record the remaining new items only as dated, caveated notes, because +they rest on Anthropic's own unverified self-reporting. The direction of the new guidance is consistent across +Anthropic's published material, but every quantitative claim traces to Anthropic alone, so this report treats the +announcement as directional vendor guidance rather than settled fact. + +- **Confidence:** Medium ## Research Results ### The six shifts Anthropic announced for Claude 5-generation models Anthropic's announcement (A1) names six changes to how context should be engineered for Claude 5-generation models, -each framed as a before-and-after against prior practice. The six shifts are corroborated by independent secondary -coverage (A11), though all coverage derives from the same original Anthropic disclosure: +each framed as a before-and-after against prior practice. Secondary press (A11) restates the six shifts, but every +outlet derives from the same Anthropic disclosure, so the announcement's content is confirmed while its claims remain +a single party's self-report: 1. **Prescriptive rules become judgment-based instructions.** The announcement's own example: the old rule "Never write multi-paragraph docstrings" becomes "Write code that reads like the surrounding code: match its comment @@ -48,15 +49,15 @@ coverage (A11), though all coverage derives from the same original Anthropic dis prose descriptions of desired behavior (A1). The headline quantitative claim is that Anthropic removed over eighty percent of Claude Code's system prompt with no -measured loss on coding evaluations (A1, restated by A11). Secondary press adds a more specific figure, roughly 800 -tokens down to about 164, and reports an incoming `claude doctor` / `/doctor` command for rightsizing CLAUDE.md files -and skills (A11). Because every outlet derives from the same Anthropic disclosure, these figures are convergent -secondary coverage of one source, not independent measurement [single-source]. +measured loss on coding evaluations (A1). Secondary press adds a more specific figure, roughly 800 tokens down to +about 164, and reports an incoming `claude doctor` / `/doctor` command for rightsizing CLAUDE.md files and skills +(A11). No disclosed methodology accompanies the figure, and no independent measurement exists; validation (V5) +classifies the whole quantitative chain as an interested party's self-report [single-source]. ### The established context-engineering corpus behind the new rules The new announcement sits on top of a consistent body of Anthropic engineering guidance (A2 through A8) that predates -it. Its core claims, corroborated across several independent Anthropic artifacts: +it. Its core claims, corroborated across several separate Anthropic artifacts: - **Context is a finite, depleting resource.** Model performance degrades as tokens accumulate ("context rot"), so the goal is the smallest set of high-signal tokens that produces the desired outcome (A2, corroborated @@ -98,63 +99,68 @@ corpus, often with more operational specificity than Anthropic's own posts: - Three-level progressive disclosure with a 500-line SKILL.md ceiling (A16, matching A7). - Context-rot rationale, attention competition, lost-in-the-middle position effects, and a 15 to 40 percent utilization peak (A15, matching and extending A2). +- A standing rule against restating what the toolchain or platform already provides, in place since 2026-06-16 + (A15), which covers the substance of A1's de-duplication shift. - Description budgets: a 1024-character target, the Claude Code 1536-character per-entry cap, and the roughly one percent listing budget (A16). - Single-role agents that generate or evaluate, never both (A17), which matches A9's self-preferential-bias failure mode from a different angle. - Multi-agent economics with a 45 percent saturation threshold sourced to a 2025 Google, DeepMind, and MIT study (A17), consistent with A5's reserve-for-high-value-tasks advice. -- Scripts for deterministic operations, dispatch namespacing, and just-in-time guidance reading (A13, A18, matching - A7). - -### Where the new guidance and Han's docs diverge - -Five deltas surfaced, one of them a direct tension: - -1. **A dated per-model claim now points the wrong way.** `per-model-authoring.md` (A14) was last checked against - Anthropic's published guidance on 2026-07-20, four days before the new announcement. It states that Opus 4.8 and - Sonnet 5 "follow instructions literally and do not generalize on their own, so they want each behavior spelled out" - while Fable 5 wants short, goal-based instructions. A1's shift to judgment-based instructions is framed for the - whole Claude 5 generation, which includes Sonnet 5. This is a codebase-versus-web conflict to resolve by - re-verification against Anthropic's current per-model guidance, not by silently adopting either side. -2. **No explicit de-duplication rule.** The corpus separates description from body content but has no rule stating - that an instruction must not repeat across surfaces (skill body, agent definition, vendored references), which A1 - names as its own shift. -3. **Code-based references are only partially covered.** The corpus prescribes scripts for deterministic operations, - but A1's broader preference for test suites, rubrics, and code artifacts over prose specifications has no - counterpart in the guidance. -4. **Automatic memory and the rightsizing tool are unrepresented.** Neither automatic memory-saving (A1) nor the - `/doctor` command (A11) [single-source] appears in the guidance. The `/doctor` claim rests on secondary press only - and is not yet doc-worthy as fact. -5. **Two of three named failure modes are unrepresented.** Self-preferential bias is already encoded as the - single-role rule; agentic laziness and goal drift (A9) [single-source] are not named anywhere in the corpus. - -### Evidence-gathering integrity note +- Code-based reference guidance across three files: scripts for deterministic validation, canonical examples ("3 + well-chosen examples match 9"), and example test suites as success criteria (A19), covering most of A1's sixth + shift. +- Dispatch namespacing and just-in-time guidance reading (A13, A18, matching A7). + +### Where the new guidance and Han's docs genuinely diverge + +The initial research claimed five gaps; validation refuted or narrowed three of them. What remains: + +1. **One dated per-model file to re-verify, not rewrite.** `per-model-authoring.md` (A14) was last checked against + Anthropic's official per-model documentation on 2026-07-20, four days before the announcement. Its claim that + Opus 4.8 and Sonnet 5 "follow instructions literally and do not generalize on their own, so they want each + behavior spelled out" sits on a different axis from A1's shift: literal instruction-following is a behavior claim, + while A1 describes removing redundant instruction volume (V2). The two are not contradictory, so the file is not + known to be wrong. It is dated against a moved target, went through its own spec review on the day it was written + (V8), and should be re-verified against the same official per-model pages it already cites, not against the + marketing announcement. +2. **Automatic memory and the rightsizing tool are unrepresented.** Neither automatic memory-saving (A1) nor the + `/doctor` command (A11) [single-source] appears anywhere in the corpus (confirmed by search, V4). Both rest on + the announcement and derivative press only, so they qualify as caveated notes at most, not guidance rules. +3. **Two of three named failure modes are unrepresented.** Self-preferential bias is already encoded as the + single-role rule; agentic laziness and goal drift (A9) [single-source] are not named in the corpus (V4). Same + evidence ceiling: caveated notes at most. + +Two claimed gaps did not survive validation: the missing de-duplication rule exists in the context-hygiene guidance +(V1 refuted it), and the code-based-reference coverage is broader than the "scripts only" characterization (V3). + +### Evidence-gathering integrity notes One fetched page (A8) contained directive-style text ahead of its article body instructing the fetcher to retrieve a -documentation index file. Per this skill's rules, that text was recorded as a claim and not acted on, and the -web-facing agents ran with no repository or user context to surrender. +documentation index file. That text was recorded as a claim and not acted on, and the web-facing agents ran with no +repository or user context to surrender. Validation (V6) flagged that the equivalent screening result was not +explicitly logged for A1, A6, A9, and A10, which share the same blog format; that screening should be repeated and +logged before any doc edit cites those pages. ## Options to Consider ### O1: Targeted delta update to the existing guidance files -- **What it is:** Refresh `per-model-authoring.md` against the new announcement, re-verifying the Sonnet 5 - literal-instructions claim; fold each confirmed new rule into the existing decision-scoped file where it belongs - (judgment-based altitude into the instruction-writing guidance, an explicit de-duplication rule, code-based - references guidance); cite A1 with its retrieval date; carry single-source items as caveated notes. -- **Trade-offs:** Smallest change and preserves the corpus's decision-scoped structure, but touches several files and - requires judgment about where each rule lands. -- **Rests on:** (A1, A11) for the shifts; (A13, A14, A15, A16, A17) for what the corpus holds today. -- **Evidence status:** corroborated +- **What it is:** Refresh `per-model-authoring.md`, and fold each new rule into the existing decision-scoped file + where it belongs: judgment-based altitude into the instruction-writing guidance, a de-duplication rule, and + code-based references guidance. +- **Trade-offs:** This was the pre-validation recommendation, and validation showed two of its three additions + duplicate rules the corpus already carries (V1, V3). As scoped, it would add redundant content, which is the + failure mode the new guidance itself warns against. +- **Rests on:** (A1, A11) for the shifts; (A13, A14, A15, A19) for what the corpus holds. +- **Evidence status:** corroborated for the shifts; the gap claims it depended on were refuted by codebase evidence ### O2: One new dedicated context-engineering reference doc - **What it is:** Add a single new reference file carrying the Claude 5 shifts, cross-linked from the existing files. - **Trade-offs:** One clean edit, but the corpus routes authors to guidance by decision point, and a topic-shaped file - would either duplicate content the decision-scoped files already carry or leave authors two places to look. That - duplication is exactly what A1's fourth shift warns against, and it breaks the repo's one-canonical-source - convention. + would duplicate content the decision-scoped files already carry or leave authors two places to look. That + duplication is what A1's fourth shift and the corpus's own existing rule (A15) both warn against. - **Rests on:** (A1), (A13). - **Evidence status:** corroborated @@ -162,9 +168,9 @@ web-facing agents ran with no repository or user context to surrender. - **What it is:** Reorganize the guidance corpus around "thin prompts, thick artifacts" as its central organizing idea. -- **Trade-offs:** The cost far exceeds the evidence. Most of the new guidance's substance is already present in the - corpus, and the strongest justification for radical change, the eighty percent prompt cut, is an uncorroborated - self-reported figure [single-source]. +- **Trade-offs:** The cost far exceeds the evidence. Most of the new guidance's substance is already present, and the + strongest justification for radical change, the eighty percent prompt cut, is an uncorroborated self-reported + figure [single-source]. - **Rests on:** (A1, A11). - **Evidence status:** single-source (caveated) for the claims that would justify the scale of change @@ -172,37 +178,142 @@ web-facing agents ran with no repository or user context to surrender. - **What it is:** Treat the blog post as directional and wait for Anthropic's official docs to confirm before editing anything. -- **Trade-offs:** Zero effort now, but it knowingly leaves a dated per-model file whose Sonnet 5 claim is in tension - with corroborated new guidance, and the corpus's own convention is to date-stamp and refresh model-behavior claims. -- **Rests on:** (A14) for the staleness it would leave in place. -- **Evidence status:** corroborated (the staleness is verifiable; the case for waiting is the weakness) +- **Trade-offs:** Zero effort now, but it leaves `per-model-authoring.md` date-stamped against guidance that has + since moved, and the corpus's own convention is to keep model-behavior claims dated and refreshed. +- **Rests on:** (A14). +- **Evidence status:** corroborated (the datedness is verifiable; the case for waiting is the weakness) + +### O5: Verify and caveat + +- **What it is:** Re-verify `per-model-authoring.md` against Anthropic's current official per-model documentation + (the same source class it already cites) and refresh its last-checked date, checking its four in-corpus referrers + for stale cross-references (V7). Cross-link the existing de-duplication and code-reference rules to the new + announcement as supporting citation where useful. Record the genuinely absent items, automatic memory, the + `/doctor` command, and the agentic-laziness and goal-drift failure modes, as dated single-source notes or decline + them until official documentation confirms them. Add no new rules. +- **Trade-offs:** The smallest update that resolves the one verifiable defect (a dated model-behavior file) without + adding redundant or weakly-sourced content. Its cost is restraint: if the announcement's shifts later gain official + documentation, a second pass will be needed. +- **Rests on:** (A14) for the dated file; (A15, A19) for the rules that already exist; (A1, A11) for what to caveat. +- **Evidence status:** corroborated ## Recommendation -- **Recommendation:** O1, the targeted delta update. Start with `per-model-authoring.md`: re-verify its Sonnet 5 and - Opus claims against Anthropic's current published guidance and update its last-checked date, since a guidance file - that points authors the opposite direction from current upstream guidance is the one live defect found. Then add - the three genuinely new rules to the files that own those decisions: an explicit de-duplication rule, the - preference for code-based references (test suites, rubrics) over prose specifications, and the judgment-over-rules - instruction altitude for Claude 5-generation models. Carry the single-source items (the eighty percent figure, the - `/doctor` command, the named failure modes, the Fable working-style tips) as caveated, dated notes rather than - asserted facts. -- **Evidence basis:** The six shifts rest on corroborated evidence (A1 confirmed in its specifics by A11, and - consistent with the independent established corpus A2 through A8). The claim that Han's corpus already encodes the - established guidance rests on codebase evidence (A13 through A18) cross-checked against the web sources. The - per-model tension rests on dated codebase evidence (A14) against the corroborated A1 shift. The quantitative - figures, the `/doctor` command, and the A9 and A10 material are single-source and are recommended only as caveated - notes, never as the basis of a guidance rule. +- **Recommendation:** O5, verify and caveat. Re-verify `per-model-authoring.md` against Anthropic's official + per-model documentation and refresh its checked date, checking the four files that reference it before editing + (the guidance SKILL.md, the specialization guidance, and the two portable assets). Treat the announcement's shifts + as supporting citations for rules the corpus already carries, and hold the genuinely new items (automatic memory, + `/doctor`, the named failure modes) to dated, caveated notes until Anthropic's official documentation confirms + them. Before making any edit that cites the blog sources, repeat the directive-text screening that was logged for + the best-practices page and log the result (V6). +- **Evidence basis:** The six shifts and their direction rest on Anthropic's announcement (A1) restated by derivative + press (A11) and consistent with Anthropic's separate established corpus (A2 through A8); this is internally + consistent vendor guidance, not independently verified fact, and validation (V5) requires it be carried as a + self-report. The claim that Han's corpus already encodes the established guidance rests on codebase evidence (A13, + A15, A16, A17, A18, A19) verified directly by validation (V1, V3). The datedness of `per-model-authoring.md` rests + on codebase evidence (A14). No part of the recommendation rests on the uncorroborated quantitative figures. ## Validation -_Pending: adversarial validation findings will be recorded here._ +### V1: The claimed missing de-duplication rule exists + +- **Strategy:** Challenge the Evidence +- **Investigation:** `context-hygiene.md` lines 177-182 carry the rule "Do not restate what the toolchain or platform + already provides"; git history shows it was introduced 2026-06-16, over a month before the announcement. +- **Result:** Refuted (the gap claim) +- **Impact:** The "add a de-duplication rule" action was removed from the recommendation; at most the existing rule + gains a supporting citation. + +### V2: The per-model "tension" conflates two different axes + +- **Strategy:** Challenge the Assumptions +- **Investigation:** Full read of `per-model-authoring.md`. Its claim, sourced to Anthropic's official per-model + prompting pages, is about literal instruction-following behavior. The announcement describes removing redundant + instruction volume, a density axis, not a specificity-hurts-Sonnet-5 axis. +- **Result:** Partially Refuted +- **Impact:** The framing that the file is already wrong was withdrawn. Re-verification remains warranted because the + file is date-stamped against guidance that has since moved. + +### V3: Code-based-reference coverage is broader than claimed + +- **Strategy:** Challenge the Evidence +- **Investigation:** `writing-effective-instructions.md` carries "Use scripts for deterministic validation" (lines + 445-472) and "Include canonical examples" citing "3 well-chosen examples match 9" (lines 396-443); + `success-criteria-and-testing.md` line 41 covers example test suites. +- **Result:** Partially Refuted +- **Impact:** The "add code-based references guidance" action was removed from the recommendation; existing rules may + gain a supporting citation. + +### V4: The options list omitted a smaller option + +- **Strategy:** Challenge the Options Framing +- **Investigation:** With V1 and V3 resolved, the only confirmed absences are automatic memory / `/doctor` and the + agentic-laziness and goal-drift vocabulary (verified absent by search), all single-source. +- **Result:** Confirmed +- **Impact:** O5 (verify and caveat) was added and became the recommendation. + +### V5: The announcement's corroboration is derivative, not independent + +- **Strategy:** Challenge the Evidence-Gathering Integrity +- **Investigation:** The secondary outlets restate the same disclosure and add no independent measurement; discounting + them removes no information. The announcement and three related sources sit on Anthropic's marketing blog, not its + documentation domains, and the eighty percent figure has no disclosed methodology. +- **Result:** Confirmed +- **Impact:** "Corroborated by A11" language was downgraded throughout to restatement of a single self-report, and + the recommendation forbids resting any guidance rule on the quantitative claims. + +### V6: Directive-text screening was logged for only one of four blog sources + +- **Strategy:** Challenge the Evidence-Gathering Integrity +- **Investigation:** The registry records a directive-language check for the best-practices page only; A1, A6, A9, and + A10 share the same format and domain but have no logged screening result. +- **Result:** Partially Refuted (a process gap, not a confirmed injection) +- **Impact:** Carried as a remaining risk and as a precondition in the recommendation: screen and log before citing + those pages in doc edits. + +### V7: No referrer check existed for the file the recommendation edits + +- **Strategy:** Challenge the Recommendation +- **Investigation:** Search found `per-model-authoring.md` referenced by the guidance SKILL.md, + `specialization-and-model-selection.md`, and the two portable assets (`guidance-portable-SKILL.md`, + `rule-index-body.md`). +- **Result:** Confirmed (a gap, not yet a broken caller) +- **Impact:** The referrer check was added to the recommendation so an edit cannot leave a stale cross-reference. + +### V8: The per-model file has review provenance + +- **Strategy:** Challenge the Assumptions +- **Investigation:** `docs/plans/per-model-authoring-guidance/` holds a feature specification and implementation plan + with same-day commits (2026-07-20) showing the file's claims passed a spec-review and synthesis cycle citing + Anthropic's official per-model pages. +- **Result:** Partially Refuted (of the framing that the claim was thin) +- **Impact:** Re-verification must compare against Anthropic's current official per-model documentation, the source + class the file already cites, and note explicitly if those official pages have not themselves changed. + +### Adjustments Made + +Validation reshaped the report. The pre-validation recommendation (O1, a targeted multi-file delta update) did not +survive: two of its three content additions duplicated rules the corpus already carries (V1, V3), and the urgency of +its per-model rewrite was overstated (V2, V8). O5 was added (V4) and became the recommendation. Corroboration +language for the announcement was downgraded (V5), the screening precondition was added (V6), and the referrer check +was added to the recommended edit (V7). The confidence rating was lowered from Medium-High to Medium. + +### Confidence Assessment + +- **Confidence:** Medium +- **Remaining Risks:** Every claim about the new guidance traces to Anthropic's own publications; no independent + measurement of any quantitative figure exists. The `/doctor` command rests on secondary press only. The + directive-text screening for four of the blog sources (V6) has not yet been re-run and logged. The validator did + not re-fetch the web sources; its attack tested internal consistency and fit against the repository, so if the + announcement's content differs from how the registry characterizes it, the affected findings would need + re-checking. Anthropic's official per-model pages were not fetched in this pass, so whether they changed after + 2026-07-20 is unknown; that check is the first step of the recommended update. ## Sources | ID | Source | Link / location | Retrieved | Trust class | Summary (one line) | Evidence status | | --- | --------------------------------------------------- | -------------------------------------------------------------------------------------------- | ---------- | ----------- | ---------------------------------------------------------------------------------------- | -------------------------------------------- | -| A1 | The New Rules of Context Engineering for Claude 5 | | 2026-07-27 | web | Six shifts for Claude 5-generation models; 80% system-prompt cut claim | corroborated by A11 (derivative) | +| A1 | The New Rules of Context Engineering for Claude 5 | | 2026-07-27 | web | Six shifts for Claude 5-generation models; 80% system-prompt cut claim | restated by A11 (derivative); self-report | | A2 | Effective Context Engineering for AI Agents | | 2026-07-27 | web | Context rot; right altitude; JIT retrieval; compaction; note-taking; subagent summaries | corroborated by A3, A5, A6, A7, A8 | | A3 | Writing Tools for AI Agents | | 2026-07-27 | web | Tool namespacing, consolidation, unambiguous params, actionable errors | corroborated by A2, A4, A5 | | A4 | Building Effective AI Agents | | 2026-07-27 | web | Workflows vs agents; five composable patterns; start simple; invest in tool design | corroborated by A3, A5 | @@ -212,14 +323,15 @@ _Pending: adversarial validation findings will be recorded here._ | A8 | Claude Code Best Practices | | 2026-07-27 | web | Minimal CLAUDE.md; skills for occasional context; subagents for investigation and review | corroborated by A2, A7; contained directive text (recorded, not followed) | | A9 | Dynamic Workflows in Claude Code | | 2026-07-27 | web | Failure modes (agentic laziness, self-preferential bias, goal drift); workflow patterns | single source (caveated) | | A10 | A Field Guide to Claude Fable | | 2026-07-27 | web | Fable-specific working practices: blind-spot pass, prototypes, source as reference | single source (caveated) | -| A11 | Secondary press coverage of A1 | and others | 2026-07-27 | web | Restates the six shifts, ~800-to-~164-token figure, incoming `claude doctor` command | corroborates A1; derivative of the same disclosure | +| A11 | Secondary press coverage of A1 | and others | 2026-07-27 | web | Restates the six shifts, ~800-to-~164-token figure, incoming `claude doctor` command | derivative of A1; adds no independent verification | | A12 | Prompt Engineering Overview (platform docs) | | 2026-07-27 | web | Thin routing page; no substantive claims | single source (thin) | | A13 | Han guidance corpus inventory | `han-plugin-builder/skills/guidance/` (48 files) | n/a | codebase | Full inventory of skill-building and agent-building guidance | codebase anchor | -| A14 | Per-model authoring guidance | `han-plugin-builder/skills/guidance/references/per-model-authoring.md` | n/a | codebase | Last checked 2026-07-20; Sonnet 5/Opus 4.8 literal vs Fable 5 goal-based instructions | in tension with A1 | -| A15 | Context hygiene guidance | `han-plugin-builder/skills/guidance/references/skill-building-guidance/context-hygiene.md` | n/a | codebase | Attention competition, lost-in-the-middle, 15-40% utilization, compaction budget | consistent with A2 | +| A14 | Per-model authoring guidance | `han-plugin-builder/skills/guidance/references/per-model-authoring.md` | n/a | codebase | Last checked 2026-07-20; Sonnet 5/Opus 4.8 literal vs Fable 5 goal-based instructions | dated against A1; different axis per V2 | +| A15 | Context hygiene guidance | `han-plugin-builder/skills/guidance/references/skill-building-guidance/context-hygiene.md` | n/a | codebase | Attention competition, compaction budget, and the existing no-restating rule (lines 177-182) | consistent with A2; covers A1's fourth shift | | A16 | Progressive disclosure and description budgets | `han-plugin-builder/skills/guidance/references/skill-building-guidance/progressive-disclosure.md` and `skill-description-length.md` | n/a | codebase | Three loading levels, 500-line body ceiling, 1024/1536-char description budgets | consistent with A7 | | A17 | Agent guidance (domain focus, economics) | `han-plugin-builder/skills/guidance/references/agent-building-guidelines/` | n/a | codebase | Single-role rule, 50-token role identity, 45% multi-agent threshold | consistent with A5, A9 | | A18 | Builder skills (interview and review checklists) | `han-plugin-builder/skills/skill-builder/SKILL.md` and `agent-builder/SKILL.md` | n/a | codebase | Evidence-first interviews; conformance review checklists mapping to guidance files | codebase anchor | +| A19 | Code-reference and example guidance | `writing-effective-instructions.md` (lines 396-472) and `success-criteria-and-testing.md` (line 41), under `skill-building-guidance/` | n/a | codebase | Scripts for deterministic validation, canonical examples, example test suites | covers most of A1's sixth shift | ### A1: The New Rules of Context Engineering for Claude 5 Generation Models — recommendation-bearing @@ -231,19 +343,7 @@ _Pending: adversarial validation findings will be recorded here._ upfront context, de-duplicated instructions, automatic memory over manual CLAUDE.md edits, and code-based references over markdown specs. Reports an over-eighty-percent cut to Claude Code's system prompt with no measured evaluation loss, and recommends keeping CLAUDE.md to repository gotchas while moving situational guidance into Skills. -- **Evidence status:** corroborated by A11 on content; the quantitative figures trace to this single Anthropic disclosure - -### A11: Secondary press coverage — recommendation-bearing - -- **Link / location:** (dated - 2026-07-24), plus - and forum and blog coverage -- **Retrieved:** 2026-07-27 -- **Trust class:** web -- **Summary:** Independent outlets restating the six shifts, the docstring example verbatim, the roughly 800-to-164-token - figure, and an incoming `claude doctor` command for rightsizing CLAUDE.md files and skills. All derive from the same - Anthropic disclosure, so they confirm the announcement's content without independently verifying its measurements. -- **Evidence status:** corroborates A1; derivative, not independent measurement +- **Evidence status:** restated by A11 without independent verification; carried as a vendor self-report per V5 ### A14: per-model-authoring.md — recommendation-bearing @@ -252,7 +352,18 @@ _Pending: adversarial validation findings will be recorded here._ - **Trust class:** codebase - **Summary:** States it was last checked against Anthropic's published guidance on 2026-07-20 for Sonnet 5, Opus 4.8, and Fable 5. Claims Opus 4.8 and Sonnet 5 follow instructions literally and want each behavior spelled out, while - Fable 5 wants short, goal-based instructions and degrades under exhaustive checklists. The new announcement frames - judgment-based instructions as a property of the whole Claude 5 generation, putting this file's Sonnet 5 claim in - tension with current upstream guidance. -- **Evidence status:** codebase anchor; in tension with A1, resolution requires re-verification + Fable 5 wants short, goal-based instructions and degrades under exhaustive checklists. Validation (V2) established + this is a behavior claim on a different axis from the announcement's density shift, and (V8) that it carries its own + same-day review provenance; the defect is datedness, not established error. +- **Evidence status:** codebase anchor; re-verification against Anthropic's official per-model pages is the recommended action + +### A15: context-hygiene.md — recommendation-bearing + +- **Link / location:** `han-plugin-builder/skills/guidance/references/skill-building-guidance/context-hygiene.md` + (lines 177-182 for the no-restating rule) +- **Retrieved:** n/a +- **Trust class:** codebase +- **Summary:** Carries the corpus's context-economy rationale (attention competition, position effects, utilization + peak, compaction budgets) and, since 2026-06-16, the rule "Do not restate what the toolchain or platform already + provides," which validation (V1) confirmed covers the substance of the announcement's de-duplication shift. +- **Evidence status:** codebase anchor; verified by V1 From 4b1ab785f6369acb04019ad2a4a4d99445c19613 Mon Sep 17 00:00:00 2001 From: River Lynn Bailey Date: Mon, 27 Jul 2026 08:08:11 -0600 Subject: [PATCH 3/3] docs(research): apply the readability rewrite to the context-engineering research report --- ...context-engineering-guidelines-claude-5.md | 131 ++++++++++-------- 1 file changed, 71 insertions(+), 60 deletions(-) diff --git a/docs/research/context-engineering-guidelines-claude-5.md b/docs/research/context-engineering-guidelines-claude-5.md index 9a1f4b78..06145a44 100644 --- a/docs/research/context-engineering-guidelines-claude-5.md +++ b/docs/research/context-engineering-guidelines-claude-5.md @@ -7,21 +7,27 @@ updated in response? Evidence mode: strict. ## Summary Anthropic has published new context engineering guidance for its Claude 5-generation models, and nearly all of it -confirms what Han's plugin-building guidance already teaches. The new guidance names six shifts: write instructions -that state a goal and trust the model's judgment instead of spelling out rigid rules, design tool interfaces so the -parameters explain themselves instead of relying on usage examples, load context progressively instead of up front, -remove instructions that repeat across surfaces, let the model save its own memory instead of maintaining memory -files by hand, and point the model at code, test suites, and rubrics instead of long prose descriptions. Anthropic -reports it cut its own coding assistant's system prompt by more than eighty percent with no measured loss. +confirms what Han's plugin-building guidance already teaches. The new guidance names six shifts: + +1. Write instructions that state a goal and trust the model's judgment, instead of spelling out rigid rules. +2. Design tool interfaces so the parameters explain themselves, instead of relying on usage examples. +3. Load context progressively, instead of all at once up front. +4. Remove instructions that repeat across surfaces. +5. Let the model save its own memory, instead of maintaining memory files by hand. +6. Point the model at code, test suites, and rubrics, instead of long prose descriptions. + +Anthropic reports it cut its own coding assistant's system prompt by more than eighty percent with no measured loss. Adversarial validation cut the needed update down to a small one. Two gaps the initial research claimed turned out not to exist: the guidance already carries a rule against restating what the platform provides, and its -code-reference coverage is broader than first credited. The recommended update is narrow: re-verify the one guidance -file that describes how each Claude model wants its instructions written against Anthropic's official per-model -documentation, refresh its checked date, and record the remaining new items only as dated, caveated notes, because -they rest on Anthropic's own unverified self-reporting. The direction of the new guidance is consistent across -Anthropic's published material, but every quantitative claim traces to Anthropic alone, so this report treats the -announcement as directional vendor guidance rather than settled fact. +code-reference coverage is broader than first credited. + +The recommended update stays narrow. Re-verify the one guidance file that describes how each Claude model wants its +instructions written, checking it against Anthropic's official per-model documentation, and refresh its checked +date. Record the remaining new items only as dated, caveated notes, because they rest on Anthropic's own unverified +self-reporting. The direction of the new guidance is consistent across Anthropic's published material, but every +quantitative claim traces to Anthropic alone. So this report treats the announcement as directional vendor guidance +rather than settled fact. - **Confidence:** Medium @@ -60,9 +66,9 @@ The new announcement sits on top of a consistent body of Anthropic engineering g it. Its core claims, corroborated across several separate Anthropic artifacts: - **Context is a finite, depleting resource.** Model performance degrades as tokens accumulate ("context rot"), so - the goal is the smallest set of high-signal tokens that produces the desired outcome (A2, corroborated + the goal is the smallest set of high-signal tokens that produces the desired outcome (A2). This is corroborated operationally by A3's high-signal tool responses, A6's automatic stale-tool-result clearing, and A8's CLAUDE.md - pruning test "would removing this line cause a mistake?"). + pruning test ("would removing this line cause a mistake?"). - **System prompts sit at the right altitude.** Specific enough to guide, flexible enough to avoid brittle if-else logic, organized with structural markup (A2). The framing is A2's own [single-source], but A1's docstring example is a concrete instance of the same shift. @@ -70,26 +76,29 @@ it. Its core claims, corroborated across several separate Anthropic artifacts: tool sets are a named failure mode (A2, corroborated by A3's consolidation and naming rules and A5's prefer-specialized-tools heuristic). - **Curated canonical examples beat exhaustive edge-case lists** (A2) [single-source]. -- **Just-in-time retrieval beats pre-loading, with a hybrid recommended.** Lightweight identifiers loaded at runtime, - critical data loaded up front (A2, corroborated by A7's three-level skill loading model and A8's split of - frequently-needed material into CLAUDE.md and occasional material into skills). +- **Just-in-time retrieval beats pre-loading, with a hybrid recommended.** Lightweight identifiers load at runtime; + critical data loads up front (A2). This is corroborated by A7's three-level skill loading model and by A8's split + of frequently-needed material into CLAUDE.md and occasional material into skills. - **Compaction and external memory handle long horizons.** Summarize and reinitiate, clear stale tool results first, and keep structured notes outside the context window (A2, corroborated by A6's shipped context-editing and memory tool features). A6's percentage improvements are Anthropic's internal benchmarks [single-source]. -- **Subagents get clean contexts and return condensed summaries.** A lead agent decomposes work; each subagent needs - an explicit objective, output format, tool guidance, and boundaries, or agents duplicate work (A2, A5, corroborated - at the product level by A8). A5's figures, including that multi-agent systems use roughly fifteen times the tokens - for a 90.2 percent quality gain on complex research, are internal single-study numbers [single-source]. +- **Subagents get clean contexts and return condensed summaries.** A lead agent decomposes the work. Each subagent + needs an explicit objective, an output format, tool guidance, and boundaries, or agents end up duplicating work + (A2, A5, corroborated at the product level by A8). A5's figures, including that multi-agent systems use roughly + fifteen times the tokens for a 90.2 percent quality gain on complex research, are internal single-study numbers + [single-source]. - **Skills are the unit of progressive disclosure.** Name and description are the trigger signal; the SKILL.md body loads on trigger; supplementary files load only when a sub-scenario needs them; scripts handle deterministic operations (A7, corroborated by A8). -Anthropic's harness post (A9) adds three named failure modes for long-running agent work: agentic laziness, declaring -a multi-part job done early; self-preferential bias, favoring one's own output when judging it; and goal drift, losing -the objective across many turns. It also names workflow patterns including adversarial verification, tournament, and -loop-until-done, and advises reserving heavy multi-agent machinery for high-value tasks [single-source]. A companion -post on working with Claude Fable (A10) recommends practices such as a deliberate blind-spot pass, prototypes over -detailed specs for subjective work, and treating source code as the best reference [single-source]. +Anthropic's harness post (A9) adds three named failure modes for long-running agent work. Agentic laziness is +declaring a multi-part job done early. Self-preferential bias is favoring one's own output when judging it. Goal +drift is losing the objective across many turns. The post also names workflow patterns including adversarial +verification, tournament, and loop-until-done, and advises reserving heavy multi-agent machinery for high-value tasks +[single-source]. + +A companion post on working with Claude Fable (A10) recommends practices such as a deliberate blind-spot pass, +prototypes over detailed specs for subjective work, and treating source code as the best reference [single-source]. ### What Han's guidance corpus already covers @@ -117,13 +126,13 @@ corpus, often with more operational specificity than Anthropic's own posts: The initial research claimed five gaps; validation refuted or narrowed three of them. What remains: 1. **One dated per-model file to re-verify, not rewrite.** `per-model-authoring.md` (A14) was last checked against - Anthropic's official per-model documentation on 2026-07-20, four days before the announcement. Its claim that + Anthropic's official per-model documentation on 2026-07-20, four days before the announcement. Its claim is that Opus 4.8 and Sonnet 5 "follow instructions literally and do not generalize on their own, so they want each - behavior spelled out" sits on a different axis from A1's shift: literal instruction-following is a behavior claim, - while A1 describes removing redundant instruction volume (V2). The two are not contradictory, so the file is not - known to be wrong. It is dated against a moved target, went through its own spec review on the day it was written - (V8), and should be re-verified against the same official per-model pages it already cites, not against the - marketing announcement. + behavior spelled out." That claim sits on a different axis from A1's shift (V2): literal instruction-following is + a behavior claim, while A1 describes removing redundant instruction volume. The two are not contradictory, so the + file is not known to be wrong. It is dated against a moved target, and it went through its own spec review on the + day it was written (V8). It should be re-verified against the same official per-model pages it already cites, not + against the marketing announcement. 2. **Automatic memory and the rightsizing tool are unrepresented.** Neither automatic memory-saving (A1) nor the `/doctor` command (A11) [single-source] appears anywhere in the corpus (confirmed by search, V4). Both rest on the announcement and derivative press only, so they qualify as caveated notes at most, not guidance rules. @@ -131,16 +140,17 @@ The initial research claimed five gaps; validation refuted or narrowed three of single-role rule; agentic laziness and goal drift (A9) [single-source] are not named in the corpus (V4). Same evidence ceiling: caveated notes at most. -Two claimed gaps did not survive validation: the missing de-duplication rule exists in the context-hygiene guidance -(V1 refuted it), and the code-based-reference coverage is broader than the "scripts only" characterization (V3). +Two claimed gaps did not survive validation. The missing de-duplication rule exists in the context-hygiene guidance +(V1 refuted it). The code-based-reference coverage is broader than the "scripts only" characterization (V3). ### Evidence-gathering integrity notes -One fetched page (A8) contained directive-style text ahead of its article body instructing the fetcher to retrieve a +One fetched page (A8) contained directive-style text ahead of its article body, instructing the fetcher to retrieve a documentation index file. That text was recorded as a claim and not acted on, and the web-facing agents ran with no -repository or user context to surrender. Validation (V6) flagged that the equivalent screening result was not -explicitly logged for A1, A6, A9, and A10, which share the same blog format; that screening should be repeated and -logged before any doc edit cites those pages. +repository or user context to surrender. + +Validation (V6) flagged that the equivalent screening was not logged for A1, A6, A9, and A10, which share the same +blog format. That screening should be repeated and logged before any doc edit cites those pages. ## Options to Consider @@ -158,9 +168,9 @@ logged before any doc edit cites those pages. ### O2: One new dedicated context-engineering reference doc - **What it is:** Add a single new reference file carrying the Claude 5 shifts, cross-linked from the existing files. -- **Trade-offs:** One clean edit, but the corpus routes authors to guidance by decision point, and a topic-shaped file - would duplicate content the decision-scoped files already carry or leave authors two places to look. That - duplication is what A1's fourth shift and the corpus's own existing rule (A15) both warn against. +- **Trade-offs:** This is one clean edit. But the corpus routes authors to guidance by decision point, so a + topic-shaped file would duplicate content the decision-scoped files already carry, or leave authors with two places + to look. That duplication is what A1's fourth shift and the corpus's own existing rule (A15) both warn against. - **Rests on:** (A1), (A13). - **Evidence status:** corroborated @@ -168,7 +178,7 @@ logged before any doc edit cites those pages. - **What it is:** Reorganize the guidance corpus around "thin prompts, thick artifacts" as its central organizing idea. -- **Trade-offs:** The cost far exceeds the evidence. Most of the new guidance's substance is already present, and the +- **Trade-offs:** The cost far exceeds the evidence. Most of the new guidance's substance is already present. The strongest justification for radical change, the eighty percent prompt cut, is an uncorroborated self-reported figure [single-source]. - **Rests on:** (A1, A11). @@ -185,9 +195,9 @@ logged before any doc edit cites those pages. ### O5: Verify and caveat -- **What it is:** Re-verify `per-model-authoring.md` against Anthropic's current official per-model documentation - (the same source class it already cites) and refresh its last-checked date, checking its four in-corpus referrers - for stale cross-references (V7). Cross-link the existing de-duplication and code-reference rules to the new +- **What it is:** Re-verify `per-model-authoring.md` against Anthropic's current official per-model documentation, + the same source class it already cites, and refresh its last-checked date. Check its four in-corpus referrers for + stale cross-references (V7). Cross-link the existing de-duplication and code-reference rules to the new announcement as supporting citation where useful. Record the genuinely absent items, automatic memory, the `/doctor` command, and the agentic-laziness and goal-drift failure modes, as dated single-source notes or decline them until official documentation confirms them. Add no new rules. @@ -202,14 +212,14 @@ logged before any doc edit cites those pages. - **Recommendation:** O5, verify and caveat. Re-verify `per-model-authoring.md` against Anthropic's official per-model documentation and refresh its checked date, checking the four files that reference it before editing (the guidance SKILL.md, the specialization guidance, and the two portable assets). Treat the announcement's shifts - as supporting citations for rules the corpus already carries, and hold the genuinely new items (automatic memory, + as supporting citations for rules the corpus already carries. Hold the genuinely new items (automatic memory, `/doctor`, the named failure modes) to dated, caveated notes until Anthropic's official documentation confirms them. Before making any edit that cites the blog sources, repeat the directive-text screening that was logged for the best-practices page and log the result (V6). -- **Evidence basis:** The six shifts and their direction rest on Anthropic's announcement (A1) restated by derivative - press (A11) and consistent with Anthropic's separate established corpus (A2 through A8); this is internally - consistent vendor guidance, not independently verified fact, and validation (V5) requires it be carried as a - self-report. The claim that Han's corpus already encodes the established guidance rests on codebase evidence (A13, +- **Evidence basis:** The six shifts and their direction rest on Anthropic's announcement (A1), restated by + derivative press (A11), and are consistent with Anthropic's separate established corpus (A2 through A8). This is + internally consistent vendor guidance, not independently verified fact, and validation (V5) requires it be carried + as a self-report. The claim that Han's corpus already encodes the established guidance rests on codebase evidence (A13, A15, A16, A17, A18, A19) verified directly by validation (V1, V3). The datedness of `per-model-authoring.md` rests on codebase evidence (A14). No part of the recommendation rests on the uncorroborated quantitative figures. @@ -292,8 +302,8 @@ logged before any doc edit cites those pages. ### Adjustments Made -Validation reshaped the report. The pre-validation recommendation (O1, a targeted multi-file delta update) did not -survive: two of its three content additions duplicated rules the corpus already carries (V1, V3), and the urgency of +Validation reshaped the report. The pre-validation recommendation, O1, a targeted multi-file delta update, did not +survive. Two of its three content additions duplicated rules the corpus already carries (V1, V3), and the urgency of its per-model rewrite was overstated (V2, V8). O5 was added (V4) and became the recommendation. Corroboration language for the announcement was downgraded (V5), the screening precondition was added (V6), and the referrer check was added to the recommended edit (V7). The confidence rating was lowered from Medium-High to Medium. @@ -304,10 +314,10 @@ was added to the recommended edit (V7). The confidence rating was lowered from M - **Remaining Risks:** Every claim about the new guidance traces to Anthropic's own publications; no independent measurement of any quantitative figure exists. The `/doctor` command rests on secondary press only. The directive-text screening for four of the blog sources (V6) has not yet been re-run and logged. The validator did - not re-fetch the web sources; its attack tested internal consistency and fit against the repository, so if the + not re-fetch the web sources. Its attack tested internal consistency and fit against the repository, so if the announcement's content differs from how the registry characterizes it, the affected findings would need re-checking. Anthropic's official per-model pages were not fetched in this pass, so whether they changed after - 2026-07-20 is unknown; that check is the first step of the recommended update. + 2026-07-20 is unknown. That check is the first step of the recommended update. ## Sources @@ -333,7 +343,7 @@ was added to the recommended edit (V7). The confidence rating was lowered from M | A18 | Builder skills (interview and review checklists) | `han-plugin-builder/skills/skill-builder/SKILL.md` and `agent-builder/SKILL.md` | n/a | codebase | Evidence-first interviews; conformance review checklists mapping to guidance files | codebase anchor | | A19 | Code-reference and example guidance | `writing-effective-instructions.md` (lines 396-472) and `success-criteria-and-testing.md` (line 41), under `skill-building-guidance/` | n/a | codebase | Scripts for deterministic validation, canonical examples, example test suites | covers most of A1's sixth shift | -### A1: The New Rules of Context Engineering for Claude 5 Generation Models — recommendation-bearing +### A1: The New Rules of Context Engineering for Claude 5 Generation Models (recommendation-bearing) - **Link / location:** - **Retrieved:** 2026-07-27 @@ -345,7 +355,7 @@ was added to the recommended edit (V7). The confidence rating was lowered from M loss, and recommends keeping CLAUDE.md to repository gotchas while moving situational guidance into Skills. - **Evidence status:** restated by A11 without independent verification; carried as a vendor self-report per V5 -### A14: per-model-authoring.md — recommendation-bearing +### A14: per-model-authoring.md (recommendation-bearing) - **Link / location:** `han-plugin-builder/skills/guidance/references/per-model-authoring.md` - **Retrieved:** n/a @@ -353,11 +363,12 @@ was added to the recommended edit (V7). The confidence rating was lowered from M - **Summary:** States it was last checked against Anthropic's published guidance on 2026-07-20 for Sonnet 5, Opus 4.8, and Fable 5. Claims Opus 4.8 and Sonnet 5 follow instructions literally and want each behavior spelled out, while Fable 5 wants short, goal-based instructions and degrades under exhaustive checklists. Validation (V2) established - this is a behavior claim on a different axis from the announcement's density shift, and (V8) that it carries its own - same-day review provenance; the defect is datedness, not established error. + that this is a behavior claim on a different axis from the announcement's density shift. Validation (V8) + established that the file carries its own same-day review provenance. The defect is datedness, not established + error. - **Evidence status:** codebase anchor; re-verification against Anthropic's official per-model pages is the recommended action -### A15: context-hygiene.md — recommendation-bearing +### A15: context-hygiene.md (recommendation-bearing) - **Link / location:** `han-plugin-builder/skills/guidance/references/skill-building-guidance/context-hygiene.md` (lines 177-182 for the no-restating rule)