Skip to content

test: hold the README's command tables to the registry - #14

Merged
muellerei merged 1 commit into
mainfrom
test/readme-option-coverage
Sep 15, 2026
Merged

muellerei merged 1 commit into
mainfrom
test/readme-option-coverage

Conversation

@muellerei

Copy link
Copy Markdown
Owner

Follow-up to #13, which closed twenty documentation gaps. This stops them
coming back.

The defect behind the gaps

The twenty missing options were the symptom, not the cause. The cause is that
the README's command tables are a hand-maintained view of something derivable,
and nothing recomputed them:

  • --help is generated by Click and is always complete, so the gap never hurt
    anyone enough to be noticed.
  • Every later check read the table. A table looks complete when you read it;
    only a comparison shows what is not in it.

Nineteen of the twenty came in with chore: initial import, which wrote the
code and the first README in one go — the table was a selection from the start
and was read as a reference afterwards.

What the test checks

  1. Every option in the command registry appears in the README in at least one
    of its forms. Boolean flags count in either spelling, since the README lists
    the one a caller actually passes (--no-create, not --create).
  2. Every command has a row. delete-block is exempt as a documented alias of
    remove-block, sharing its row.
  3. The section counters (### Read (14)) sum to the number of commands.

--json and --help are exempt from the first check — they are on nearly
every command and documented once in prose rather than in forty rows. A
separate assertion keeps that prose honest.

It found another defect on its first run

### Edit claimed 11 commands where there are 8. The section has 11 table
rows, but insert-block occupies five of them — so the counter had been
counting rows, not commands, and would have kept drifting. Corrected to 8;
the counters now sum to 37, matching the registry.

Verified

  • Mutation-checked three ways, each turning it red: removing a documented
    option from the README, falsifying a section counter, and adding a new
    undocumented command to the registry. The third is the case this exists for.
  • A "guards the guard" assertion fails if the scan itself stops finding
    options, so a broken detector cannot pass silently.
  • Full suite green, 728 tests.

The twenty undocumented options were the symptom. The defect is that a
table is a hand-maintained view of something derivable and nothing
recomputed it: --help is generated and stays complete, so the gap never
hurt enough to be noticed, and every later check read the table - which
looks complete when you read it.

Three checks: every option in the registry appears in the README in one
of its forms, every command has a row, the section counters sum to the
number of commands.

It found a further defect on its first run: ### Edit claimed 11 where
there are 8 commands, because insert-block occupies five rows.

Mutation-checked three ways - removing a documented option, falsifying
a counter, and adding an undocumented command each turn it red.
@muellerei
muellerei merged commit 023fa6c into main Sep 15, 2026
4 checks passed
@muellerei
muellerei deleted the test/readme-option-coverage branch September 15, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant