Skip to content

docs: align CLI and App workshops with learner-created skills and QA - #167

Draft
GeekTrainer wants to merge 2 commits into
mainfrom
geektrainer/teach-quality-skill-creation
Draft

GeekTrainer wants to merge 2 commits into
mainfrom
geektrainer/teach-quality-skill-creation

Conversation

@GeekTrainer

@GeekTrainer GeekTrainer commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

Align the CLI and App workshops around setup 0–1 and core modules 2–10, teaching learners to create, inspect, and execute a quality-checks skill and a specialist QA agent rather than depend on bundled answers. Mirror the curriculum in all five locales, repair navigation and minimal VS Code/Cloud compatibility, and align authoring guidance with the existing publisher checks; the skill creation prompt is now 184 words instead of 516 following fresh Sonnet and MAI trials.

Type of change

  • Workshop content (lesson Markdown, images)
  • Site shell (website/ Astro + Starlight wrapper)
  • Copilot configuration (.github/copilot-instructions.md, instructions, agents, skills)
  • Repo housekeeping (CI, dependabot, README, license)
  • Other:

Verification

  • cd website && npm run check:all && rm -rf dist && npm run build succeeds (target: 40 routes × 6 locales + 1 redirect = 241 built pages excluding 404; build reports 242 HTML files including 404, or note any intentional change)
  • Lychee link check passes: mkdir -p /tmp/lychee-root && ln -sfn $PWD/website/dist /tmp/lychee-root/copilot-workshops && lychee --offline --no-progress --root-dir /tmp/lychee-root 'website/dist/**/*.html'
  • External GitHub URLs that I changed have been clicked manually (lychee runs offline)

The latest shortened and localized source passed local type checks with zero errors and four existing hints, a clean build with 241 index pages / 242 HTML files, and offline lychee with 20,177 links checked, 2,838 unique links, and zero errors. The link check used an equivalent temporary root inside the worktree, cleaned afterward. All 150 source-backed App/CLI/root titles and language attributes and all 252 rendered plaintext prompt blocks matched their sources. Source scope, frozen hashes, localized alignment, canonical navigation, and whitespace checks passed. External URLs were not all manually clicked; offline checks do not certify external links.

Screenshots

Not included. No visual redesign; App UI rehearsal remains unverified.

Notes for reviewers

The shortened skill creation and functional execution were verified with both models on macOS Bash. Broader QA-agent and UI/platform acceptance remains separate and open; this PR stays a draft. These focused trials do not certify the earlier QA-agent matrix, App picker/hot-discovery behavior, native Windows/PowerShell execution, or VS Code/Cloud UI behavior. Timing estimates are deferred. Site checks are not runtime proof.

Shortened skill trial evidence

Fresh generation contexts received only the exact learner creation prompt plus genuine repository instructions, without previous generated answers or the controller's review rubric. Both used github-samples/tailspin-toys cleanup revision 525b74ef427a6d755b387c3e2c52e4d155e7e4af plus an identical five-file filtering-only fixture (manifest SHA256 894101ab7bd8c69840652cfac06fc44de529c552361b3f5c6a8d1daa81f40e4a). The fixture adds filtering implementation/tests, not skill or QA answers. The unchanged skill run prompt was executed in separate fresh contexts after inspecting the generated artifacts.

Actual model Final creation Actual checks Single-unit-file selection
claude-sonnet-5 184-word prompt; Bash-only skill and correct npm wrappers; no artifact fixes Lint clean; unit 35 tests / 3 files; full unfiltered E2E 28 tests / 4 specs; typecheck:all ran tsgo and Astro, 0 errors / warnings src/lib/games.test.ts only: 12 tests / 1 file
mai-code-1.1-flash Same exact prompt and baseline; Bash-only skill and correct npm wrappers; no artifact fixes Same complete coverage, with five explicit direct wrapper invocations returning exit 0 after a denied orchestration attempt Same requested file only: 12 tests / 1 file

The earlier 161-word candidate failed artifact acceptance with both models: both generated both shells despite the instruction not to; Sonnet documented an extra argument separator, and MAI defaulted to incomplete type checking. Required repository reads succeeded; MAI had no tool denials, and Sonnet's artifact defects preceded its later denials. Those failures are preserved separately, not silently repaired or counted as passes. Sonnet's later probe attempts included denied temporary writes and one probe created in isolated session storage; none was executed successfully. The accepted 184-word revision produced no creation-time checks or probes.

Execution was not denial-free. MAI first proposed an unnecessary outer eval loop over agent-authored constant command strings; a generic permission denial prevented execution. No untrusted payload was identified, and neither the generated instructions nor wrappers required eval. It then ran all five required commands directly and successfully. Its final prose omitted the denial; the raw evidence retains it. Sonnet recovered from four read-only inspection/cache-discovery denials and piped outputs through tail without explicit caller pipefail, so pipeline exit 0 alone is not treated as proof of native failure propagation; successful runner summaries establish the observed test results.

Separately, 24 controller-only probes on isolated script copies/stubs passed: another working directory, paths with spaces, exact npm arguments with one separator, stdout/stderr preservation, eight exit-37 propagation cases, and 16 missing/wrong-root failures before npm invocation. These are not substituted for the actual model-driven checks. Tooling was pinned Copilot CLI 1.0.84-5, project Node 24.18.0 / npm 11.16.0 (CLI bundled Node 24.20.0), with cached dependencies. No controller installation or new authentication was performed; initial app-provisioned dependency provenance was not certified. Native PowerShell was unavailable. Generated skills, feature files, and repository instructions were unchanged after execution; trial servers exited and the port lease was released.

Coordinated scope

Coordinate this content change with the separate template-removal draft github-samples/tailspin-toys#97. That cleanup currently targets geektrainer/pr-91-base at the exact github-samples/tailspin-toys#91 revision 085d9e5; it depends on github-samples/tailspin-toys#91 landing before github-samples/tailspin-toys#97 can be retargeted to main and revalidated. This repository contains no generated learner skill/agent answers, trial outputs, application code, logs, or build artifacts. The template retains its existing npm checks; learners package them into simple shell wrappers here. Filtering, skill files, QA profile, and tests stay on one feature branch through the eventual feature PR. CLI has three PR milestones; App adds its canvas milestone.

The shortening updates only lesson 5 in English and the five locales, plus directly related inspection prose that no longer mandates a scripts/ subdirectory: bundled scripts may live beneath the skill directory without an arbitrary layout requirement. The create → inspect → run flow, run prompt, and QA prompts remain unchanged. No template guidance or prebuilt solution was added to compensate for the shorter prompt.

No new redirects are added. Renamed/removed lesson URLs intentionally break existing external deep links; internal links and sidebars are migrated. The existing legacy prerequisites redirect is unchanged. Publishing behavior is unchanged: the workflow edit is an explanatory comment only, and documentation now accurately describes its existing type-check, build, and offline link-validation steps.

Frozen English prompt SHA256 values (prompt text including terminal newline, excluding fences):

Prompt SHA256
Skill creation, shortened revision 2 (184 words) fab0924e767b6a4aa0f48703054bd0dd6fc70a55d0f353ac80b3c82e1dcbfbb2
Skill run, final safety revision 299f2fe491c7789ef361ba9f77d54cefdf9ab63fa4cd0bacc74d3704490de58e
QA creation v1 043a31ad8f3ffac1e5378701627db8328f300b6a7f8368326b10badcaf0d492f
QA run v1 76101c2be3c080f484b406172a36ff7c744a29ed5444ce6808f22eebdc7fd1a1

The unchanged run safety revision requires code inspection for root/guard verification and explicitly prohibits moving, renaming, deleting, or modifying repository files to simulate failures. It preserves all four checks and semantic single-test-file selection verification without adding wrapper process management or learner failure probes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reduce creation to 184 words and align all five locales without changing run or QA prompts. Allow bundled scripts beneath the skill directory without requiring a particular subdirectory.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@GeekTrainer

GeekTrainer commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator Author

Review is now split into four focused draft PRs, registered as native stack 175 in the requested bottom-to-top order:

  1. docs: simplify workshop authoring and verification guidance #169 — Authoring and verification guidance (targets main).
  2. docs: refresh App workshop with learner-created skills and QA #174 — App workshop refresh and translations (targets docs: simplify workshop authoring and verification guidance #169).
  3. docs: align VS Code and Cloud workshop lesson structure #170 — VS Code and Cloud lesson alignment (targets docs: refresh App workshop with learner-created skills and QA #174).
  4. docs: refresh CLI workshop with learner-created skills and QA #171 — CLI workshop refresh and translations (targets docs: align VS Code and Cloud workshop lesson structure #170).

All workshop lesson content and the entire website tree exactly match the preserved complete revision at e150fd5. The final simplification in #169 adds only small Bash and PowerShell wrappers around existing npm type checks, the site build, and offline Lychee, with updated skill guidance/index. The custom comparison engine, capture hooks, parser dependency declaration, and its test framework were removed. The current full-stack tree is ffe15594ac09dde2077950e541fb97ad1382dff9 at CLI head 1dcc86f. The Bash wrapper passed against the complete curriculum; PowerShell was not installed and remains unexecuted/unparsed. Source-page, semantic, translation, and visual inspection remain separate from these existing checks. Historical runtime and UI acceptance caveats remain in the relevant PR descriptions.

During the reorder, GitHub indirectly marked old #172 merged when its CLI base acquired the App commit. With explicit approval, #174 replaced it for review; closed #172 and closed stack 173 remain as history. Nothing merged into main, which remains at 7ddcf24.

This original PR remains open as a historical reference; review the replacement drafts rather than merging this combined PR. All four replacement PRs are drafts with auto-merge disabled. Template dependencies remain github-samples/tailspin-toys#97 and github-samples/tailspin-toys#91.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant