test: hold the README's command tables to the registry - #14
Merged
Merged
Conversation
The twenty undocumented options were the symptom. The defect is that a table is a hand-maintained view of something derivable and nothing recomputed it: --help is generated and stays complete, so the gap never hurt enough to be noticed, and every later check read the table - which looks complete when you read it. Three checks: every option in the registry appears in the README in one of its forms, every command has a row, the section counters sum to the number of commands. It found a further defect on its first run: ### Edit claimed 11 where there are 8 commands, because insert-block occupies five rows. Mutation-checked three ways - removing a documented option, falsifying a counter, and adding an undocumented command each turn it red.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #13, which closed twenty documentation gaps. This stops them
coming back.
The defect behind the gaps
The twenty missing options were the symptom, not the cause. The cause is that
the README's command tables are a hand-maintained view of something derivable,
and nothing recomputed them:
--helpis generated by Click and is always complete, so the gap never hurtanyone enough to be noticed.
only a comparison shows what is not in it.
Nineteen of the twenty came in with
chore: initial import, which wrote thecode and the first README in one go — the table was a selection from the start
and was read as a reference afterwards.
What the test checks
of its forms. Boolean flags count in either spelling, since the README lists
the one a caller actually passes (
--no-create, not--create).delete-blockis exempt as a documented alias ofremove-block, sharing its row.### Read (14)) sum to the number of commands.--jsonand--helpare exempt from the first check — they are on nearlyevery command and documented once in prose rather than in forty rows. A
separate assertion keeps that prose honest.
It found another defect on its first run
### Editclaimed 11 commands where there are 8. The section has 11 tablerows, but
insert-blockoccupies five of them — so the counter had beencounting rows, not commands, and would have kept drifting. Corrected to 8;
the counters now sum to 37, matching the registry.
Verified
option from the README, falsifying a section counter, and adding a new
undocumented command to the registry. The third is the case this exists for.
options, so a broken detector cannot pass silently.