Skip to content

Pre-PR style check, doc-draft skill, and product briefs - #1693

Merged
jth-nw merged 11 commits into
devfrom
docs/pre-pr-style-check
Oct 13, 2026
Merged

jth-nw merged 11 commits into
devfrom
docs/pre-pr-style-check

Conversation

@jth-nw

@jth-nw jth-nw commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds /doc-draft, a skill that writes new Netwrix documentation from source material and lints it before the PR. It replaces the tech-writer agent. The PR also adds the pre-PR style check that /doc-draft and writers share, product briefs that give the skill context, and a page-structure guide.

What /doc-draft does

Give it a PR, a work item, specs or notes, or a request to structure a new docs directory. It reads the source, drafts into the docs tree, checks coverage, lints, and reports what it did.

Source What it reads
Pull request The diff, then the full code in a local clone (tests included, since they show edge cases), UI strings and defaults. It searches the existing docs to decide new page, new section, or edit.
GitHub issue, Jira ticket, Azure DevOps item The item, its comments and linked PRs, plus any Xchange articles it links. The code wins over the ticket on behavior. The ticket supplies what the code can't show. Ticket keys and customer names stay out of the page.
Specs, notes, a prompt The text only. It marks gaps as {/* TODO */} and never invents facts.
New product or version The documentation-section guide: required zones, order, folders, _category_.json.

Every draft:

  • starts with frontmatter that includes a description
  • gives each procedure an example, or a TODO when the source has none
  • uses short, plain sentences without dropping any fact or qualifier
  • passes a coverage check against the source (flags, defaults, limits, errors, side effects, what can't be undone) before linting
  • states a caveat the code shows on the page, not only in a TODO
  • has its internal links checked
  • is linted with npm run quality:check (Vale, Dale, AI-isms), up to 3 rounds
  • ends with a report: what it chose and why, every source read, every TODO, and any place the code disagrees with the PR, a swagger file, or an existing page

After the report it offers once to write a product brief when the product has none. It fixes docs/<product>/CLAUDE.md or the brief only where its changes made a stated fact false.

Context the skill reads

  • .claude/references/products/<product>.md: briefs for auditor, dataclassification, endpointprotector, pingcastle, threatmanager and threatprevention. Each covers what the product is, where things live in the code, code-to-UI vocabulary, and what couldn't be verified.
  • docs/<product>/CLAUDE.md for auditor, dataclassification, endpointprotector and threatprevention (new), and threatmanager (updated: it listed 2 of its 4 versions).
  • .claude/references/page-structure-guide.md: page types, new page vs section vs edit, titles, reference pages and glossaries, API docs.
  • .claude/references/lint-rules.md: a generated digest of the lint rules, so the first draft avoids them.

Pre-PR style check

Vale, Dale (a model check; 2 of 3 runs must agree) and an AI-writing check run at commit on the lines the change adds. The Stop hook runs the same check on the docs a Claude session edited, up to 3 rounds. It is skipped in CI and when dependencies aren't installed. A failed model call skips Dale and doesn't block the commit. vale-autofix.yml remains the PR-side backstop. git commit --no-verify skips the check.

How it compares

I ran /doc-draft on two Data Classification features and scored it against a code-derived fact list and the team's existing pages. These existing pages may not come from generate-issue-documentation, which I couldn't confirm.

Feature /doc-draft Team's pages
Graph Exchange sources (74 must-have facts) 43 covered, 1 wrong 14 covered, 4 wrong
Government cloud (41 must-have facts) 33 covered, 0 wrong 3 covered, 3 wrong

The team's pages named the Graph permissions, which the code doesn't state. That motivated the Xchange and Jira steps in the skill.

Review notes

  • The briefs come from code surveys and haven't been reviewed by product owners. Each lists what it couldn't verify.
  • The Jira and Xchange steps are untested. The author has no Jira access.
  • .claude/ is gitignored, so the new reference files were added with git add -f.
  • This PR also fixes two CodeQL alerts and the review findings on the check scripts, and adds js-yaml to devDependencies.

🤖 Generated with Claude Code

jth-nw and others added 5 commits October 12, 2026 12:09
Move doc linting off the PR and into the pre-commit hook. The hook checks
new pages whole and existing pages only on the lines the commit changes:
Vale, the AI-isms word list, and two model checks (an AI-isms judge and
Dale), each voted over 3 runs with verdicts cached by content. Vale must
be installed or the commit is blocked with install steps.

- scripts/quality: score CLI (staged/branch/file), Vale and Dale runners,
  judge, claude-CLI fallback when there is no API key, optional fix helper
  with validated drop/swap/reword edits, optional readability report
- Claude Code: Stop hook that checks only the docs the session edited
  (recorded by a PostToolUse hook), hooks use node instead of jq
- tech-writer agent: lints its own drafts, reads a generated lint-rules
  list, follows the documentation section guide for placement and new
  directories, takes GitHub issues, Jira tickets, and Azure DevOps items,
  and gives examples to every procedure (TODOs for concepts)
- Vale ignores bold UI labels and double-quoted product messages
- Remove vale-autofix.yml; add script-tests.yml for script changes
- Update CLAUDE.md, docs/CLAUDE.md, and CONTRIBUTING.md to match

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… persona

The pre-commit check now fails a new page without a non-empty description
in its frontmatter. Existing pages and partials are not checked. The
tech-writer agent writes a description on every page, and its opening
persona text is cut.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The drafting workflow runs in the session (or a subagent) instead of a
dedicated agent: PR, work item, specs and notes, and new-directory
structure, with the lint step and the description rule. The style
reference moves to a file beside the skill. CLAUDE.md and CONTRIBUTING.md
point to the skill.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… rules

Swap lib/judge.mjs and score.mjs for lib/consensus.mjs and check.mjs
(npm run quality:check; flags --no-model and --require-model). Add 12 Dale
rules and 7 Vale rules for AI-writing patterns, wire a quality step into the
pre-commit hook, restore vale-autofix.yml as a CI backstop, and drop phrases
from ai-isms.yml that Vale now covers.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…efs to doc-draft

- Read tests with the code, fetch Xchange links from work items, and name
  the Jira tools with a sign-in hint
- Put code-established caveats on the page, check internal links, and
  point step 3 at the new page-structure guide
- Add product briefs for auditor, dataclassification, endpointprotector,
  threatmanager, threatprevention, and pingcastle
- Add docs/CLAUDE.md files for four products and update threatmanager's

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jth-nw
jth-nw requested review from a team as code owners October 13, 2026 16:55
Comment thread scripts/doc-draft/lib/prose.mjs Fixed
Comment thread scripts/doc-draft/lib/report.mjs Fixed
Comment thread scripts/quality/fix-loop.mjs Fixed
- Offer once, after the report, to write a brief when a product has none
- Add step 10: fix a product CLAUDE.md or brief only where the draft made a
  stated fact false, and report those edits apart from the docs pages
- Lint is now step 11

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Escape backslashes before pipes in findings table cells
- Create the fix-loop patch temp file in a private mkdtemp directory

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
docusaurus.config.js imported src/plugins/llms-txt.js, which is not in the
repository, so the CI build failed with MODULE_NOT_FOUND.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Regenerate lint-rules.md so the drift check passes
- Point doc-help at doc-draft instead of the removed tech-writer agent
- Treat a failed Dale model call as skipped, and exit 3 when the check itself
  crashes; the Stop hook no longer sends a crash to Claude as findings
- Strip nested JSX tags until stable in the prose extractor

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jth-nw
jth-nw deployed to development October 13, 2026 17:18 — with GitHub Actions Active
- Skip the Stop hook in CI and when dependencies are not installed
- Give .vale-local.ini the same TokenIgnores as .vale.ini
- Run Script Tests when the Vale or Dale rule files change, and on reopen
- Start gen-lint-digest.mjs reliably on any path with fileURLToPath
- Declare js-yaml in devDependencies

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jth-nw
jth-nw deployed to development October 13, 2026 17:28 — with GitHub Actions Active
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jth-nw
jth-nw deployed to development October 13, 2026 17:46 — with GitHub Actions Active
@jth-nw
jth-nw merged commit 0973a57 into dev Oct 13, 2026
10 checks passed
@jth-nw
jth-nw deleted the docs/pre-pr-style-check branch October 13, 2026 17:48
@github-actions

Copy link
Copy Markdown
Contributor

Code Review

No security problems found. The new workflow uses pull_request (not pull_request_target) with contents: read, and nothing passes untrusted input to a shell (claude -p gets its arguments through spawn, and page text goes in on stdin). The issues below are about behavior.

1. The pre-commit quality step blocks commits on Vale suggestions
scripts/quality/lib/vale.mjs (runVale) runs vale with the repo's .vale.ini, where MinAlertLevel = suggestion. It keeps every alert on a changed line and never looks at Severity. check.mjs then fails the page on any finding. As a result, a suggestion-level alert blocks the commit, even though the existing pre-push step only blocks on warnings and errors (blockingAlerts drops suggestions). The hk.pkl comment also says pre-commit leftovers "are reported, not blocked." Fix: filter to Severity !== 'suggestion', or match whatever scripts/vale-local.mjs treats as blocking. The step also uses .vale.ini instead of .vale-local.ini, so local hooks no longer run one consistent rule set.

2. The quality and vale pre-commit steps can run at the same time
.config/hk.pkl: quality reads the index (git show :file). The vale step in the same hook runs --fix and restages files. hk runs steps in parallel unless depends is set, so quality can read the text from before the fix. It can then block a commit on findings that the fix step clears. Add depends = List("vale") to quality.

3. The pre-commit hook crashes if dependencies are missing
check.mjs imports @anthropic-ai/sdk (via lib/llm.mjs) and js-yaml (via lib/dale.mjs) at load time. A writer who pulls this change without running npm install gets an ERR_MODULE_NOT_FOUND crash, and every commit that touches docs is blocked. The Stop hook guards against this ([ -d node_modules/js-yaml ] && ...), but the pre-commit step doesn't. Either add the same guard, or load the SDK only when a model call is actually made.

4. TokenIgnores in .vale.ini turns off part of an existing CI rule
The new (\*\*[^*\n]+\*\*) ignore hides all bold text from every rule. Netwrix/PlainTextAdmonition exists to catch **NOTE:** and **CAUTION:** (two of its four tokens). With this ignore, it can't see them anymore, so CI (vale-autofix) stops flagging the most common form of a plain-text admonition. Exclude those patterns from the ignore, or move the admonition check to scope: raw. Vale didn't run on any doc in this PR's CI (No docs markdown files changed), so CI hasn't tested the new TokenIgnores line at all. That includes the “ escapes.

5. The Stop hook keeps blocking on later turns
.claude/hooks/stop-quality-check.sh: the session's edited-file list (nwx-quality-files-<session>) is never cleared. After 3 rounds, the hook deletes the counter and lets Claude stop. On the next turn, even one unrelated to docs, the counter starts at 0 again and the same unresolved findings block for 3 more rounds. Keep the counter after the 3rd round, or clear the file list. Also, $(cat "$LIST") is unquoted, so a path containing a space is split into separate arguments to todo-check.mjs.

Minor

  • script-tests.yml runs npm run test:scripts, which vale-autofix.yml already runs in its test-scripts job. PRs that touch scripts/** now run the same tests twice (both appear in this PR's checks).
  • lib/consensus.mjs verifyFindings gives each Dale finding the line number of the first line containing the quote. If an unchanged line earlier in the page has the same text, the finding gets that line number, and the inRanges filter drops it without any message.

@Denis-A-Netwrix

Copy link
Copy Markdown

It's awkward that this PR is merged without approvals

This branch was successfully deployed

1 active deployment
development — 39e1d37d Deployed Oct 13, 2026 by jth-nw via build-and-deploy #4514
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants