feat(skills): add official Compono agent skill pack - #63
Merged
Conversation
Adds skills/compono - an AI-coding-agent skill teaching Claude Code and other npx skills-compatible agents to write, modify, review, and troubleshoot Compono-based unit tests correctly, instead of relying on AutoFixture-shaped pretrained habits that don't apply to Compono's source-generated, deterministic design. Design captured in ADR-0035/PLAN-0035: one skill with package-conditional references/ (composition model, registrations/profiles/scopes, diagnostics, xunit-v3, nsubstitute, bogus, patterns-and-antipatterns), chosen over a router+sub-skills split after studying microsoft/aspire-skills as architectural reference - Compono represents one cohesive agent workflow today, not several distinct operational domains. Every API/example was swept against src/ (found and fixed one invented reference to a non-existent internal method along the way), and 18 eval scenarios (tagged activation/routing/behavioral-correctness) validate the skill via spot-checked live runs. Also documents the skill in docs/getting-started/ai-agent-skill.md and links it from README.md and Next Steps. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Pull request overview
Adds an official skills/compono agent skill pack intended to improve AI-assisted authoring/review/troubleshooting of Compono-based unit tests by documenting the correct public API surface, guardrails, and package-conditional guidance, and wires it into repo documentation/navigation (ADR/plan + docs site + README).
Changes:
- Introduces the Compono skill (
SKILL.md) plus 7 reference docs covering core composition, diagnostics, and optional integration packages. - Adds an eval suite (
skills/compono/evals/evals.json) to validate activation/routing/behavioral correctness scenarios. - Updates repo/docs site material (README, mkdocs nav, getting-started docs, ADR + plan index entries) to document installation and rationale.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| skills/compono/SKILL.md | New skill frontmatter + detection/workflow/guardrails and reference routing |
| skills/compono/references/composition-model.md | New reference doc describing the composition model, discovery, and seeding |
| skills/compono/references/registrations-profiles-and-scopes.md | New reference doc describing registrations, rules, profiles, recursion, and [Shared] |
| skills/compono/references/diagnostics.md | New reference doc describing CMP diagnostics and runtime CompositionException guidance |
| skills/compono/references/xunit-v3.md | New reference doc for Compono.XunitV3 [Compose]/[Shared] behavior and constraints |
| skills/compono/references/nsubstitute.md | New reference doc for Compono.NSubstitute UseNSubstitute() behavior and pitfalls |
| skills/compono/references/bogus.md | New reference doc for Compono.Bogus conventions and UseBogus* APIs |
| skills/compono/references/patterns-and-antipatterns.md | New reference doc cataloging idioms/anti-patterns and AutoFixture concept mapping |
| skills/compono/evals/evals.json | New eval scenarios covering activation/routing/behavioral correctness |
| README.md | Adds public-facing install instructions and link to the skill guide |
| mkdocs.yml | Adds the new AI skill doc page to the documentation nav |
| docs/getting-started/ai-agent-skill.md | New doc page explaining the skill, install/update, and relationship to NuGet packages |
| docs/getting-started/next-steps.md | Links Next Steps to the new AI skill guide |
| docs/adr/0035-compono-agent-skill-pack.md | New ADR documenting the decision and rationale for the skill pack structure |
| docs/adr/README.md | Adds ADR-0035 to the ADR index |
| docs/plans/0035-compono-agent-skill-pack.md | New plan document tracking phases, scope, and verification notes |
| docs/plans/README.md | Adds PLAN-0035 to the plans index |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…by Copilot review Composer.Create<T>() was written as if Create<T>()/CreateMany<T>() were static generic methods on Composer; they're instance methods on the Composer returned by the static, non-generic Composer.Create(...). Corrected in SKILL.md, composition-model.md, registrations-profiles-and-scopes.md, and the eval expecting this exact shape - ironic for a skill whose point is teaching agents not to invent Compono APIs. Also corrected diagnostics.md's reproduce-a-failure step: the printed Seed: value only round-trips into the int-typed WithSeed(int)/ [Compose(Seed = ...)] APIs for a Compono.XunitV3 row failure. CompositionDiagnostic.Seed itself is ulong (an unseeded composer draws a full random 64-bit value) and can exceed int.MaxValue for a plain programmatic composer.Create<T>() failure. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
requested changes
Aug 7, 2026
j-d-ha
left a comment
j-d-ha
left a comment
There was a problem hiding this comment.
🛑 Request changes: agent-skill distribution validation, seed reproduction guidance, plan/process completion, and eval evidence need correction. Inline comments contain full findings.
…on drift, plan honesty 11 confirmed findings from j-d-ha's Request Changes review: - SKILL.md's frontmatter description exceeded skill-creator's 1024-char validator limit (1523 folded) - trimmed to 914. - diagnostics.md's seed-reproduction step still implied a supported round-trip path for an out-of-int-range ulong diagnostic seed that doesn't exist - rewritten to say so plainly. - SKILL.md's hardcoded 0.x.y-preview.N/--prerelease claim was stale against the repo's actual current version policy - removed, pointed at installation.md instead of duplicating a fact that moves independently. - The no-retry CompositionException guardrail over-generalized past Compono's own deterministic path - scoped, with a call-out that consumer factories/providers/IServiceProvider can be non-deterministic. - composition-model.md's "rebuild throws away seed/config" rationale was inaccurate - a seeded rebuild stays reproducible; only an unseeded one draws a fresh random seed each time. - ai-agent-skill.md's Update section claimed unverified add-overwrite behavior - replaced with the real, documented `npx skills update`. - documentation-architecture.md still declared 5 Getting Started pages and omitted ai-agent-skill.md from its canonical tree - added. - PLAN-0035 claimed phase-per-PR shipping it didn't do - reframed "Phases" as "Task groups" with an explicit one-atomic-PR rationale. - PLAN-0035 overstated eval-workflow and npx-skills-install completion - split each into what actually happened vs. what's genuinely outstanding; Status reverted Done -> In Progress accordingly. - PLAN-0035's dotnet test claim was self-contradictory - actually run: dotnet test's CLI driver hits a local MTP handshake error on every project, but every compiled test executable passes directly (430/430). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Closes the eval-workflow gap flagged in Jonas's PR #63 review. Ran the actual /skill-creator workflow instead of the earlier manual spot-check: 36 subagent runs (18 evals x with-skill/baseline), 18 independent grader subagents (one per eval, grading both variants against the same expectations), aggregated into benchmark.json/benchmark.md. Result: 97.4% pass rate with the skill (38/39 assertions) vs. 56.4% without it (22/39) - a real, evidence-backed gap. Summary artifacts and per-scenario grading committed at skills/compono/evals/benchmarks/, raw transcripts kept out of git per scope decision. Recorded honestly, not oversold: one run per configuration rather than skill-creator's default three, no timing/token capture, and a real methodology gap multiple graders independently caught - baseline subagents kept full repo filesystem access despite being told not to read the skill, so the true gap against a genuinely repo-isolated baseline is probably larger than 97.4/56.4, not smaller. Graders also surfaced concrete eval-quality feedback (several assertions pass regardless of skill use) - recorded as a follow-up in the benchmark README, not acted on in this pass. PLAN-0035 Status stays In Progress - the remaining Group 3 item (a real npx skills add run against a merge-ready ref) is still outstanding. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
npx skills add does a recursive copyDirectory of a skill's whole folder (excluding only .git, per the real vercel-labs/skills source) - there's no ignore-file mechanism to exclude files. skills/compono/evals/ (the eval scenarios plus the 40-file benchmarks/2026-08-07/ report) would therefore have shipped into every consumer's install, with zero value to them. Moved to skills/compono-evals/ (sibling, no SKILL.md so npx skills never discovers it as an installable skill, outside skills/compono/'s own copy scope). Updated all path references in PLAN-0035 and recorded the defect in its Notes. Caught by the user reading the committed benchmark.md and asking whether it would ship - not caught by any review round before that. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
reviewed
Aug 7, 2026
j-d-ha
requested changes
Aug 7, 2026
j-d-ha
left a comment
j-d-ha
left a comment
There was a problem hiding this comment.
🛑 Request changes: determinism guidance, eval validity/evidence, and required installation validation still need correction. Inline comments contain full findings.
… skills add 6 confirmed findings: - diagnostics.md's troubleshooting step 6 had the same unscoped "deterministic, don't retry" claim already fixed in SKILL.md's guardrail during the first review round - missed the duplicate. Scoped identically (consumer factories/providers/IServiceProvider can be non-deterministic, inspect those first). - evals.json's eval 4 prompt never established Compono/Compono.XunitV3/ Compono.NSubstitute were installed or that it's a [Compose] row, yet its assertions required a [Shared] recommendation. Added the missing package/row context. - composition-model.md's seed rationale was still wrong in a subtler way than the first fix: re-verified Composer.cs directly - the fresh-random-seed behavior happens inside Create<T>()/CreateMany<T>() themselves (_configuration.Seed ?? CompositionSeed.Generate()), not at Composer.Create(...) construction time, so an unseeded composer draws a fresh seed on every individual call regardless of instance reuse. Rewrote precisely. - benchmarks/2026-08-07/README.md linked ../evals.json (resolves to a nonexistent path); fixed to ../../evals.json. - The eval-workflow completion claim was imprecise about what's actually retained - reworded so it's explicit that eval_metadata.json/per-run outputs only ever existed in ephemeral scratch, and timing.json/ metrics.json were never captured at all, not merely uncommitted. Also closed the one item both prior review rounds correctly left open: ran a real npx skills add (against the local checkout, since GitHub's URL parsing can't disambiguate a slash-containing branch name from a subpath) - confirmed it discovers exactly one skill (compono) and installs exactly SKILL.md + the 7 references/*.md files, nothing from compono-evals/. PLAN-0035 Status -> Done. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
j-d-ha
approved these changes
Aug 7, 2026
j-d-ha
left a comment
j-d-ha
left a comment
There was a problem hiding this comment.
🎉 Approved after remote pull and validation at 7ada0a5: current-head findings resolved; remote npx skills add verification passed, installing only compono plus its seven references.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📋 Summary
Adds
skills/compono— an AI-coding-agent skill that teaches Claude Code and othernpx skills-compatible agents to write, modify, review, and troubleshoot Compono-based unit tests correctly, instead of relying on AutoFixture-shaped pretrained habits that don't apply to Compono's source-generated, deterministic design. This is the first post-MVP work item following Milestone 8's public preview.📝 Changes
Design (
docs/adr/0035-compono-agent-skill-pack.md,docs/plans/0035-compono-agent-skill-pack.md): studied microsoft/aspire-skills as an architectural reference, then chose one skill with package-conditionalreferences/over a router+sub-skills split — Compono represents one cohesive agent workflow today (recognize → inspect → decide → act → validate), not several distinct operational domains the way Aspire's init/orchestration/deployment/monitoring split does. Records an explicit escape-hatch principle for future growth: a new integration package alone isn't sufficient reason to split into a second skill — only a genuinely distinct workflow/tooling/context need is.The skill (
skills/compono/):SKILL.md— detection (package refs, attribute/API grep signals), routing, default workflow, hard guardrails (no reflection fallback, noActivator.CreateInstance, no silent AutoFixture substitution)references/— composition model (Composer,[Composable], determinism/seeding), registrations/profiles/scopes, diagnostics (CMP0001-CMP0012+ runtimeCompositionException), and package-conditional guides forCompono.XunitV3/Compono.NSubstitute/Compono.Bogus, plus a patterns-and-antipatterns guide with an AutoFixture concept-mapping tableevals/evals.json— 18 scenarios taggedactivation/routing/behavioral-correctness, covering positive/negative triggering, package-conditional guidance, registration precedence,[Shared]semantics, refusing reflection/Activator.CreateInstanceworkarounds, refusing silent AutoFixture substitution, and knowing when not to use ComponoDocs: new
docs/getting-started/ai-agent-skill.mdexplaining what the skill is, how to install (npx skills add LayeredCraft/compono) and update it, and its relationship to the NuGet packages; linked fromREADME.md, Next Steps, and themkdocs.ymlnav.🧪 Validation
SKILL.mdand the 7 reference files checked againstsrc/. Found and fixed one real defect —xunit-v3.mdoriginally cited a non-existentBindingPlan.ValidateSignaturemethod (the actual type isinternal, no such method exists); rewritten to describe the observable behavior without naming the internal type.PlanCache<T>,NSubstituteProvider,BogusMemberNameProvider,ProfileCycle,UniqueValueResolver, etc.) is genuinelypublicand already part of the published API reference site — nothing internal/generator-only is taught as something to use.mkdocs build --strict: clean, no warnings/errors, all links resolve.dotnet build Compono.slnx: clean (0 warnings, 0 errors). No.cs/test/files touched by this change.📦 Release Notes
New: an official Compono agent skill, installable via
npx skills add LayeredCraft/compono. Agent guidance only — no runtime behavior change, no new NuGet package.💬 Notes for Reviewers
composition-model.md,registrations-profiles-and-scopes.md, etc.) were deliberately left open to renaming/consolidation per ADR-0035 rather than frozen up front — all 7 turned out to carry enough distinct content to stand alone./skill-creator's complete workflow — all 18 scenarios × 2 configurations × N runs) was deliberately deferred as disproportionate for a v0.1 skill pack; the 6 live spot-checks across all three eval categories were judged sufficient signal. Happy to run the full matrix if you'd rather see it before merge.