Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 30 additions & 2 deletions .claude/skills/generate-spec/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ description: >
current working directory. Must run from inside a course folder
(courses/<name>/). Do NOT use this skill for editing an existing
spec.md or for writing product specs unrelated to a course.
Pass --env=<slug> to generate for a known build environment instead of
the agnostic learner takeaway (e.g. "/generate-spec --env=coding-agent-lab",
"generate the lab version of the spec").
argument-hint: "[--env=<slug>]"
allowed-tools: [Read, Write, Glob, Grep, Bash]
---

Expand All @@ -24,6 +28,14 @@ another engineer (or coding agent) can implement end-to-end.

## Steps

0. **Parse `--env=<slug>`, if given.** Pass it only if the user supplied one —
never synthesize a slug. With a slug, resolve
`.claude/skills/generate-spec/references/spec-generation-guide.<slug>.md`
relative to `$CLAUDE_PROJECT_DIR`. If that file does not exist, refuse with
"No environment overlay named `<slug>`." plus the list of
`spec-generation-guide.*.md` overlays that do exist. Stop. Without a flag,
every step below runs exactly as written and the output is `spec.md`.

1. **Verify cwd is a course folder.** Confirm all of these exist relative to
cwd: `materials/notebooks/`, `materials/transcripts/`, `builds/`,
`evals/`. If any are missing, refuse with: "Run /generate-spec from
Expand All @@ -35,6 +47,10 @@ another engineer (or coding agent) can implement end-to-end.
Follow it fully — it is the prescriptive contract for the spec's
content, structure, and quality. Do not restate or summarize its rules
here; defer to the guide so the two files can't drift.

**With `--env=<slug>`:** after reading the base guide in full, also read the
overlay resolved in step 0, and follow its precedence rules. The base guide
is never edited for an environment; the overlay declares what it overrides.
3. **Read every material.** Glob `materials/notebooks/**/*` and
`materials/transcripts/**/*`, then read each non-empty file. If both
directories are empty (only `.gitkeep`), refuse with: "No materials
Expand All @@ -44,8 +60,11 @@ another engineer (or coding agent) can implement end-to-end.
notebooks and helper.py on top." Stop.
4. **Write spec.md.** Produce a self-contained build-ready spec at
`./spec.md` in cwd (overwrite if it exists — this is regeneration, not
editing). What the spec must contain and how it handles learner input
are governed entirely by the guide — don't duplicate those rules here.
editing). **With `--env=<slug>`, write `./spec.<slug>.md` instead**, leaving
any existing `spec.md` alone — the two are siblings, and their diff is what
shows the environment's effect. What the spec must contain and how it
handles learner input are governed entirely by the guide — don't duplicate
those rules here.
The spec should end with a section telling the builder to
conclude with an infra/structure diagram and the phrase
"This is the infra/structure diagram of this app" — `/extract-build-log`
Expand All @@ -56,6 +75,15 @@ another engineer (or coding agent) can implement end-to-end.
5. **Report.** Tell the user the path written and that they can review,
then run /prepare-build when ready.

**With `--env=<slug>`:** also print the overlay's Environment Resolutions —
every one of the six learner-context dimensions with its resolution — so the
removals are visible rather than implied, and remind the user that
/prepare-build needs the same `--env=<slug>`.

**Without a flag:** if the course already contains a `spec.<slug>.md`, say so
and name the flag that regenerates it. Otherwise a course with an environment
variant silently gets only its agnostic spec refreshed.

## Don't

- Don't write to `builds/`, `evals/`, or `materials/`.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,228 @@
# Spec Generation Guide — `coding-agent-lab` overlay

**Applies only when `/generate-spec` is invoked with `--env=coding-agent-lab`.** Without that
flag this file is not read and the base guide governs alone.

Source of the facts below: `environments/coding-agent-lab.md` §4, §5.1, §6, §9.
Written against base guide **`8e44ecb`**.

---

## How to read this file

Read the base guide first, in full:
`.claude/skills/generate-spec/references/spec-generation-guide.md`. Then read this file.

**Precedence**

1. Where this file quotes a base passage under **OVERRIDE**, this file wins.
2. **Everywhere else the base guide wins** — including any tension this file did not anticipate.
3. An unanticipated conflict is **never resolved silently.** Follow the base rule, finish the
spec, and report the tension to the user as an *overlay gap*. It gets fixed in this file,
not judged at generation time. (This rule exists because the first lab-targeted spec was
produced by exactly such silent judgments, and none of them was recoverable afterwards.)

**Staleness check — do this before generating.** Each OVERRIDE below quotes the base text it
replaces. Confirm each quoted passage still appears verbatim in the base guide. If one does
not, this overlay was written against an older base: **stop, say which quote no longer matches,
and do not generate.**

---

## Why this mode exists

The base guide produces a takeaway a learner downloads and builds **against their own project,
on their own machine, with their own coding agent**. Six of its Decision Ledger rows exist to
absorb what the generator cannot know: the learner's project, data, goal, model/provider and
environment, plus how much scope they want.

This mode produces a spec built **inside a known lab**: the agent, the container, the model set
and the assignment are all fixed before generation starts. The difference between the two modes
is entirely in *what is unknown at generation time* — not in what a good spec contains. Every
rule about mining, contracts, acceptance criteria, the Course Context Pack, failure encoding
and the pre-build gate applies unchanged.

---

## OVERRIDE 1 — §3, the input gate

Base guide `8e44ecb` §3 reads, verbatim:

> Generation is gated on **two inputs**. Verify each is actually present before generating.

**Replaced by:** generation is gated on **two required inputs plus this overlay**. The notebook
dump and transcripts remain required and remain the *only* course evidence. This overlay is a
third input describing the build target.

This overlay is **never course evidence.** Nothing traceable to it may carry a
*course-demonstrated* label, appear in CTX-A or CTX-B, or be attributed to the course in any
way. It gets its own label (see ADD 2) and its own row in CTX-E.

---

## OVERRIDE 2 — §5.5, procedure step (f)

Base guide `8e44ecb` §5.5 **Procedure** reads, verbatim:

> (f) add the six §3 learner-context rows, each Invariant holding only the pattern's capability requirements on that dimension (empty when none) — learner rows still get a course-derived default (e.g. project defaults to the §3 example-shape target);

**Replaced by a two-phase step. Both phases are mandatory and their order is load-bearing.**

**Phase 1 — derive all six, exactly as the base guide requires.** Project, data/inputs, goal,
model/provider, environment, scope-boundary. Each gets its Invariant (the pattern's capability
requirements on that dimension, empty when the pattern demands nothing) and its course-derived
default. **Do this before reading the RUNTIME FACTS section of this file.** A dimension you have
not written down cannot be resolved, and an unresolved dimension disappears without trace — that
is the exact failure this ordering prevents.

**Phase 2 — resolve each of the six** against the Resolutions below, and emit the resolution
table into the spec (ADD 1). A dimension is resolved in one of three ways:

- **ANSWERED** — the lab fixes it. The row leaves the Ledger, and **its Invariant must land
somewhere named**: a spec section, a business rule, or another Ledger row's Invariant. An
ANSWERED dimension whose Invariant goes nowhere is a defect, not a simplification.
- **CONSTRAINED** — the lab narrows it without fixing it. The row leaves the Ledger and the
constraint becomes a business rule or a §2 entry, with at least one acceptance criterion.
- **KEPT** — the lab has no opinion. The row stays in the Ledger exactly as the base guide
derived it.

**Course-owned rows are untouched by this mode.** A `design-argued`, `design-structural`,
`realization` or `contradicted` row is never removed, merged or silently defaulted here. Where
the environment makes one of its Options impractical, mark that Option in its own cell with the
reason — never delete it, and never move the "(course default)" label.

---

## ADD 1 — the Environment Resolutions table (spec section)

Emit this table into the generated spec **immediately below the Decision Ledger**, before §1.
Lead it with one sentence naming the environment and stating that the rows below were derived
and then resolved, not skipped.

| Dimension | Resolution | Where the Invariant lives now |
|---|---|---|
| project | **ANSWERED** — the lab fixes the assignment | spec §1 Objective |
| data/inputs | **ANSWERED** — inputs are seeded into the workspace | spec §5 fixture corpus |
| goal | **ANSWERED** — the lab fixes what "working" means | spec §1 + the acceptance criteria |
| model/provider | **CONSTRAINED** — see RUNTIME FACTS | spec §2 (+ CTX-D) |
| environment | **ANSWERED** — the container | spec §2 + the Runtime section; capability invariants move to the rows that need them |
| scope-boundary | **KEPT** | stays a Ledger row |

The generated spec must also carry a short *"Adapting this beyond the lab"* note in §1, naming
where each ANSWERED or CONSTRAINED dimension now lives, so a reader taking the spec elsewhere
knows what to change.

---

## ADD 2 — a fourth provenance label (extends base §9)

- **Environment-imposed** — `[environment]`. A fact or constraint contributed by this overlay,
never by the course. Requires this mode. It must never also carry a *course-demonstrated*
label, and CTX-E must carry one row attributing these facts to the environment rather than
to any lesson.

---

## ADD 3 — checklist items (extends base §14)

Run the base §14 checklist in full, then these:

- [ ] The Environment Resolutions table is present and accounts for **all six** dimensions.
- [ ] Every ANSWERED or CONSTRAINED dimension's Invariant is exercised by ≥1 acceptance
criterion at its new home, or the table states why it cannot be.
- [ ] No course-owned row was removed by this mode; an Option the environment makes impractical
is marked in its own cell, not deleted.
- [ ] Every constraint traceable to this overlay carries `[environment]`, and none is labelled
course-demonstrated.
- [ ] CTX-E carries an environment row separate from the lesson rows.
- [ ] The provenance header pins **both** the base guide's commit and this overlay's.
- [ ] §1 carries the "Adapting this beyond the lab" note.

---

## RUNTIME FACTS

**Do not read this section until Phase 1 of OVERRIDE 2 is complete** — all six dimensions
derived and written down.

Distilled from `environments/coding-agent-lab.md`. Everything here is `[environment]`.

### The question mechanism — feeds base §6.0 unchanged

Base §6.0 already requires a structured question tool "if you have one". This environment has
one, so **this is an input to that rule, not an override of it.** The mechanism is a fenced
` ```choices ` block:

- Explanation in prose *above* the fence; the fence is **last** in the message; one option per
line; the turn ends there.
- A line `**Question N of M — <decision>**` before the fence renders a question box. Use it, so
learner and agent can both see nothing was skipped.
- A click sends the option line verbatim; a typed answer naming an option counts identically.
The generated spec must say so.
- Only the **latest** assistant message's buttons stay live. A spec must never ask anyone to go
back and change an earlier answer — re-ask as a new question.
- Never put a `choices` fence inside the spec's own examples in a way that implies the learner
brief carries one; the fence is parsed only in assistant chat messages.

### The container — feeds spec §2

Debian, non-root user, Node 22, Python 3 with venv, sqlite3, git, build tooling.
**Preinstalled:** `fastapi`, `uvicorn`, `python-dotenv`, `openai`, `anthropic`; Node `express`,
`cors`, `better-sqlite3`, `vite`, `nodemon`. **Nothing else** — no vector stores, no embedding
models, no ML frameworks.

Outbound network works, so installs and one-time model downloads succeed, but each costs the
learner wall-clock time inside the session. A spec whose stack needs anything beyond the
preinstalled set must say so in §2 and tell the build agent to batch the install early and
announce its cost. Create venvs with `--system-site-packages` so the preinstalled stack is
reusable.

Memory is finite and shared with the agent's own process. A spec whose pipeline can hold several
models resident at once must carry a business rule releasing one before loading the next.

### Model / provider — the CONSTRAINED resolution

The learner picks the *agent's* model per conversation from the platform's set; a spec must work
across all of them, or say in its own text which it assumes. **Never bake a "current best" model
name into the spec** — base §6.0's gate-time "(Recommended)" flag already owns model currency.

The container exports provider API keys and their base URLs. If the built app needs an LLM, the
spec says: read keys and base URLs from the environment, use a platform model, **never ask the
learner for a key**. If the course's pipeline needs no LLM, the spec says that plainly and adds
a rule that the ambient keys are not to be read.

### The app preview — feeds spec §4 business rules

- **One server, port 4000, bound to `0.0.0.0`**, serving both the page and the API. Nothing
else is previewable.
- **Every URL inside the page must be relative** (`api/thing`, never `/api/thing`). The preview
is reverse-proxied under a path prefix, so a root-absolute path resolves against the IDE.
This is the single most common way a working app appears broken; it belongs in the spec as a
numbered business rule with an acceptance criterion, not as a note.
- HTTP only — WebSocket upgrades are not proxied. Streaming responses work.

### The workspace — feeds spec §6 permissions

- **The workspace is the record.** A new chat starts with no memory of an earlier one, and the
grader sees only what is on disk. Decisions, resolved values and results must be written to
files, not left in chat. Base §6.0 already requires `resolved-decisions.md`; state *why* it is
load-bearing here.
- **Never author `AGENTS.md`** in the workspace — the IDE rewrites it at every boot, so guidance
placed there is silently lost. This belongs in the spec's **Never** tier.
- Every tool call is visible to the learner, and long-running servers run in the background.

---

## OUT OF SCOPE

`environments/coding-agent-lab.md` §§1, 2, 3, 7, 8, 10, 11 and 12 describe **lab packaging** —
the instruction artifact (`instructions.md`, `rubric.json`, `resources.json`), the workspace zip
and its step specs, grading, pitfalls for lab authors, and how to test a packaged lab.

**This pipeline's deliverable is the app spec, not the lab package.** Do not read those sections
as instructions and do not add their artifacts to the generated spec. In particular, that
document's §10 ("What a lab spec must decide") is a checklist for *packaging a lab*; the base
guide's §6 section template governs what this spec contains, and it wins.

If a future change makes this pipeline responsible for the lab package too, that is a different
deliverable and needs its own guide — not an extension of this one.
31 changes: 26 additions & 5 deletions .claude/skills/prepare-build/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ description: >
PROACTIVELY — treat any clear build intent as the trigger. Do NOT
use this skill to run an existing build, build a docker image, or
build features in unrelated projects.
Pass --env=<slug> to prepare a build from that environment's spec
variant (spec.<slug>.md) instead of the canonical spec.md.
argument-hint: "[--env=<slug>]"
allowed-tools: [Bash, Read]
---

Expand All @@ -27,10 +30,20 @@ Does NOT start the build — building is the agent's subsequent work.

## Steps

0. **Parse `--env=<slug>`, if given.** Pass it only if the user supplied
one — never synthesize a slug. It selects the *source* spec for this
run: with a slug the source is `spec.<slug>.md`, without one it is
`spec.md`. Call that file SOURCE_SPEC below. The slug changes nothing
else: the snapshot and working copy are always named `spec.md`, so
the build agent and both eval skills are unaffected.

1. **Verify cwd is a course folder.** Confirm all exist relative to cwd:
`spec.md`, `builds/`, `evals/`, `materials/`. If any are missing,
refuse with: "Run /prepare-build from inside a course folder —
`cd courses/<name>` first." Stop.
SOURCE_SPEC (i.e. `spec.md`, or `spec.<slug>.md` with a flag),
`builds/`, `evals/`, `materials/`. If any are missing, refuse with:
"Run /prepare-build from inside a course folder — `cd courses/<name>`
first." Stop. If SOURCE_SPEC is the only thing missing and a sibling
`spec.*.md` exists, say which variants are present and name the
`--env=<slug>` that selects one, instead of just reporting absence.
2. **Allocate the next run number.**
```bash
highest=$(ls -1 builds/ 2>/dev/null | grep -E '^run-[0-9]+$' \
Expand All @@ -40,9 +53,14 @@ Does NOT start the build — building is the agent's subsequent work.
3. **Create the run dirs and snapshot the spec.**
```bash
mkdir -p "builds/$next" "evals/$next"
cp spec.md "evals/$next/spec.md" # frozen snapshot, read by /eval-spec-vs-build
cp spec.md "builds/$next/spec.md" # working copy, read by the build agent
cp "$SOURCE_SPEC" "evals/$next/spec.md" # frozen snapshot, read by /eval-spec-vs-build
cp "$SOURCE_SPEC" "builds/$next/spec.md" # working copy, read by the build agent
```
`SOURCE_SPEC` is `spec.md`, or `spec.<slug>.md` when `--env=<slug>`
was given (step 0). **Both destinations are always named `spec.md`** —
the filename is normalised here, which is why nothing downstream
(the build agent, `/eval-spec-vs-build`, `/eval-materials-vs-build`)
needs to know an environment variant was involved.
The eval snapshot at `evals/$next/spec.md` is judged against later by
`/eval-spec-vs-build`, so the run is judged against the spec it was
actually built from even if `spec.md` is regenerated. The working
Expand Down Expand Up @@ -85,6 +103,9 @@ Does NOT start the build — building is the agent's subsequent work.
## Don't

- Don't generate or modify `spec.md` here — that's `/generate-spec`.
- Don't rename the snapshot or working copy after an `--env` run. Both
stay `spec.md`; say in the announcement which source they came from,
so the run's provenance is in the transcript.
- Don't ask the user for a run number — allocation is automatic.
- Don't reuse an existing run-NN folder. Every new build is a new run,
even with the same `spec.md`, so runs and their evals can be compared.
Expand Down
Loading