Skip to content

ldd: scope the R9 gate to the review and resolve doc symbols in one pass - #103

Open
buzzdan wants to merge 7 commits into
mainfrom
ldd-scoped-repo-brain-gate
Open

buzzdan wants to merge 7 commits into
mainfrom
ldd-scoped-repo-brain-gate

Conversation

@buzzdan

@buzzdan buzzdan commented Oct 18, 2026

Copy link
Copy Markdown
Owner

What

  • Scoped R9 gate. On a scoped review, ldd-detect.sh runs check-repo-brain.sh --scope <list> --words <list>.
    • The docs the diff touches (docs.txt) get every check.
    • In a doc that names a scope file, or cites a symbol the diff removes, the gate reports only those links and symbols.
    • It checks code edges of the scope's own files only, and skips Q3.
    • A code-only diff that no doc cites skips the doc checks. --all keeps the whole-repository run.
  • One-pass symbol resolution. The gate reads the repository once into a word set: git ls-files (tracked, submodules, untracked-not-ignored), or find that prunes bazel-*, node_modules and build outputs outside git. The declaration set is built only when a token needs it.
  • --gate-budget <s>. On expiry the gate's process group is stopped and its questions are counted inconclusive, never 0.
  • Scope and detect agree. ldd-scope.sh writes test-re.txt; ldd-detect.sh reads it. A --glob or --test-re that differs from the bundle is exit 2. A scope whose files are not source files of its language is exit 2, not a table of zeros.
  • comments.txt skips string literals. A comment marker counts only at the line start or after whitespace.
  • Python adapter fix. The python and generic adapters passed $(py_prune) unquoted, so ./.git/* expanded in a git repository. The gate then found no code and skipped every code↔docs check. CI in Python repos can now report violations it did not report before.

Why

On the wekapp monorepo, a scoped review of a 6-file D diff ran the R9 gate over the whole tree for about 90 minutes. The run then dropped almost every line as out of scope. The slow step was one grep -rIhoFw -f <tokens> over the whole tree. Multi-pattern fixed-string grep is slow on a large tree whichever tool runs it: 200 tokens took 2.5 min with git grep -F -w and 1 min with parallel grep. The word set takes about 17 s.

Measured on wekapp

ldd-scope.sh --glob '*.d' --test-re '(_test|_ut)\.d$' --base <base> then ldd-detect.sh:

before after
wall time ~90 min 46 s
R9 rows (gate did not finish) 0 (12 citing docs, scope lines only)
comments.txt 4 trace format strings, 0 real comments 52 real comment lines

With --gate-budget 5, detection returns in 6 s with 4 inconclusive R9 rows and no stray processes.

Checks

  • task check, task lint-core, task docs:check: green.
  • check-repo-brain_test.sh, ldd-detect_test.sh, ldd-scope_test.sh for all four plugins: green on macOS, bash 5.3. Before the rebase onto review: split a mixed diff by language and route each group to its plugin #86, the gate and detect matrices also passed on bash 3.2.
  • New cases: code-only scope skips the doc checks; a listed doc is checked in full; a citing doc reports only its scope lines; git work tree resolves tracked and untracked, never ignored; bazel-* excluded outside git; dotted tokens; glob mismatch and 0-files errors; gate budget expiry, budget met, and a failing gate under a budget; string literal is not a comment.

🤖 Generated with Claude Code

buzzdan and others added 4 commits October 18, 2026 21:56
A scoped review ran the R9 gate over the whole repository and then dropped
most of its lines. On the wekapp monorepo that took about 90 minutes for a
6-file diff. The same review now takes 48 seconds.

- check-repo-brain.sh takes --scope and --words. A scoped run checks only
  the docs in scope: the touched docs, the docs that name a scope file, and
  the docs that cite a symbol the diff removes. It checks code edges of the
  scope files only and skips Q3. With no doc in scope it skips the doc checks.
- Symbol resolution reads the repository once into a word set. In a git work
  tree it lists files through git ls-files (tracked plus untracked, never
  ignored). Elsewhere find prunes bazel-*, node_modules and other build
  outputs. A multi-pattern grep -F -w was the slow step.
- The declaration set is built only when a token needs it.
- ldd-scope.sh writes docs.txt (the touched docs) and test-re.txt (the
  --test-re pattern). ldd-detect.sh reads test-re.txt, and a --glob or
  --test-re that differs from the bundle is exit 2.
- ldd-detect.sh exits 2 when the scope has files but none matches the source
  glob, instead of an all-zero table.
- ldd-detect.sh --gate-budget <s> stops the gate as a process group on expiry
  and counts its questions "inconclusive", never 0.
- The python and generic adapters pass find(1) pruning as an array. The
  unquoted $(py_prune) expanded ./.git/* in a git repository, so the gate
  found no code and skipped every code<->docs check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… docs

On the wekapp test diff the scoped gate reported 58 lines. None came from
the diff: they were older drift in docs that only name a scope file. The
same review now reports 0 R9 lines in 42 seconds.

- A doc the diff touches gets every check. A doc that only names a scope
  file or cites a removed symbol reports only its links into a listed file
  and its backticked removed symbols.
- ldd-detect.sh drops a removed word that the scope files still carry after
  the change.
- The word set reads submodule files too (git ls-files --recurse-submodules),
  as grep -r did before.
- A word with $ (a JavaScript identifier) also offers its $-free parts, so
  ts-react tokens with $ still resolve.
- Tests: a budget the gate meets writes the same counts as no budget; a
  gate that fails under a budget fails the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a D review in wekapp, comments.txt listed trace format strings such as
DEBUG!"#CROSSCONTAINER ..." as comments. ldd-scope.sh now counts a comment
marker only at the line start or after whitespace, so a # or // inside a
string literal is not a comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved correctness and performance issues can hang detection, misreport scoped results, and repeat full-repository scans.

6 open issues
What changed in this PR

This PR narrows the shared R9 documentation gate to review scope and improves symbol resolution across all four generated plugins, building on language-grouped reviews from #86.

Changes:

  • Adds scoped documentation checks, lazy declaration scanning, and repository-word resolution.
  • Adds gate time budgets and bundle-option validation.
  • Refines comment detection, fixes Python adapter glob expansion, and expands tests and documentation.
File Description
ts-react-linter-driven-development/​skills/​pre-commit-review/​SKILL.md Documents scoped checks and timeout handling.
ts-react-linter-driven-development/​scripts/​ldd-scope.sh Carries bundle metadata; refines comment matching.
ts-react-linter-driven-development/​scripts/​ldd-scope_test.sh Tests string-marker exclusion.
ts-react-linter-driven-development/​scripts/​ldd-detect.sh Adds scoped gates, budgets, and validation.
ts-react-linter-driven-development/​scripts/​ldd-detect_test.sh Tests scope, validation, and budgets.
ts-react-linter-driven-development/​scripts/​check-repo-brain.sh Implements scoped checks and word resolution.
ts-react-linter-driven-development/​scripts/​check-repo-brain_test.sh Tests scoped checks and resolution.
ts-react-linter-driven-development/​CHANGELOG.md Records plugin changes.
python-linter-driven-development/​skills/​pre-commit-review/​SKILL.md Documents scoped checks and timeout handling.
python-linter-driven-development/​scripts/​ldd-scope.sh Carries bundle metadata; refines comment matching.
python-linter-driven-development/​scripts/​ldd-scope_test.sh Tests string-marker exclusion.
python-linter-driven-development/​scripts/​ldd-detect.sh Adds scoped gates, budgets, and validation.
python-linter-driven-development/​scripts/​ldd-detect_test.sh Tests scope, validation, and budgets.
python-linter-driven-development/​scripts/​check-repo-brain_test.sh Tests scoped checks and resolution.
python-linter-driven-development/​CHANGELOG.md Records changes and adapter fix.
linter-driven-development/​skills/​pre-commit-review/​SKILL.md Documents scoped checks and timeout handling.
linter-driven-development/​scripts/​ldd-scope.sh Carries bundle metadata; refines comment matching.
linter-driven-development/​scripts/​ldd-scope_test.sh Tests string-marker exclusion.
linter-driven-development/​scripts/​ldd-detect.sh Adds scoped gates, budgets, and validation.
linter-driven-development/​scripts/​ldd-detect_test.sh Tests scope, validation, and budgets.
linter-driven-development/​scripts/​check-repo-brain_test.sh Tests scoped checks and resolution.
linter-driven-development/​CHANGELOG.md Records changes and adapter fix.
lang/​ts-react/​passthrough/​CHANGELOG.md Supplies TypeScript/React release notes.
lang/​python/​scripts/​repo-brain-adapter.sh Preserves find patterns using quoted arrays.
lang/​python/​passthrough/​CHANGELOG.md Supplies Python release notes.
lang/​go/​passthrough/​CHANGELOG.md Supplies Go release notes.
lang/​go/​overrides/​scripts/​check-repo-brain_test.sh Adds Go-specific gate regression cases.
lang/​generic/​scripts/​repo-brain-adapter.sh Fixes Python-mode find patterns.
lang/​generic/​passthrough/​CHANGELOG.md Supplies generic-plugin release notes.
go-linter-driven-development/​skills/​pre-commit-review/​SKILL.md Documents scoped checks and timeout handling.
go-linter-driven-development/​scripts/​ldd-scope.sh Carries bundle metadata; refines comment matching.
go-linter-driven-development/​scripts/​ldd-scope_test.sh Tests string-marker exclusion.
go-linter-driven-development/​scripts/​ldd-detect.sh Adds scoped gates, budgets, and validation.
go-linter-driven-development/​scripts/​ldd-detect_test.sh Tests scope, validation, and budgets.
go-linter-driven-development/​scripts/​check-repo-brain.sh Implements scoped checks and word resolution.
go-linter-driven-development/​scripts/​check-repo-brain_test.sh Tests scoped checks and resolution.
go-linter-driven-development/​CHANGELOG.md Records plugin changes.
docs/​token-budget.md Explains scoped detection and inconclusive results.
core/​skills/​pre-commit-review/​SKILL.md Defines shared review guidance.
core/​scripts/​ldd-scope.sh Implements shared bundle and comment changes.
core/​scripts/​ldd-scope_test.sh Adds shared comment regression coverage.
core/​scripts/​ldd-detect.sh Implements shared scoped gates and budgets.
core/​scripts/​ldd-detect_test.sh Adds shared detection regression coverage.
core/​scripts/​check-repo-brain.sh Implements shared filtering and symbol resolution.
core/​scripts/​check-repo-brain_test.sh Adds shared gate regression coverage.

Review effort: Balanced

Comment thread core/scripts/ldd-detect.sh Outdated
Comment thread core/scripts/check-repo-brain.sh Outdated
Comment thread core/scripts/check-repo-brain.sh Outdated
Comment thread core/scripts/check-repo-brain.sh Outdated
Comment thread core/scripts/check-repo-brain.sh
Comment thread core/scripts/check-repo-brain.sh Outdated
buzzdan and others added 3 commits October 18, 2026 22:14
…-gate

# Conflicts:
#	go-linter-driven-development/CHANGELOG.md
#	lang/generic/passthrough/CHANGELOG.md
#	lang/go/passthrough/CHANGELOG.md
#	lang/python/passthrough/CHANGELOG.md
#	lang/ts-react/passthrough/CHANGELOG.md
#	linter-driven-development/CHANGELOG.md
#	python-linter-driven-development/CHANGELOG.md
#	ts-react-linter-driven-development/CHANGELOG.md
- ldd-detect.sh: a --gate-budget with no value is a usage error. Before, the
  option loop did not end.
- check-repo-brain.sh builds the repository word set once per gate run, not
  once per doc root. It reads files with grep, which ends every last line, so
  two files with no final newline do not fuse into one word. Binary files
  are skipped.
- A citing doc is in scope when any dot-separated part of a backticked token
  is a removed word, so `OldType.Run` is checked when the diff removes
  OldType.
- A citing doc reports a broken link only when the link's target, with its
  title and anchor stripped and . and .. folded, is a listed path. A
  same-named doc in another directory does not count.
- An index that only cites the scope checks its lines to the touched docs
  for drift, so a changed description is caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-gate

# Conflicts:
#	core/scripts/ldd-detect.sh
#	go-linter-driven-development/scripts/ldd-detect.sh
#	linter-driven-development/scripts/ldd-detect.sh
#	python-linter-driven-development/scripts/ldd-detect.sh
#	ts-react-linter-driven-development/scripts/ldd-detect.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants