From 6995807b126e10619aa9b016702cb507c457c71a Mon Sep 17 00:00:00 2001 From: Dan Mordechay Date: Sun, 18 Oct 2026 17:51:49 +0300 Subject: [PATCH 1/4] ldd-lint: a gap linter for file length and package size, one set of thresholds Add scripts/ldd-lint.sh to every plugin. The script measures lines per source file (file-length) and source files per directory (package-size), where the repository's linter does not hold them. It never edits a file. The threshold comes from the first source that applies: - the repository's linter config, counted the way that linter counts (revive file-length-limit, sonarjs/max-lines, ESLint max-lines, pylint max-module-lines); - .ldd-lint.yaml at the repository root; - the profile defaults: 450 code lines per file, a warning at 8 source files per directory, the limit at 13. The first output line names the threshold and its source. With the defaults, it also prints the setting that makes the linter hold the limit. - Three profile scalars hold the thresholds. The generator rejects a value that is not a positive integer, and a warning at or above the limit. - A core include, scripts/ldd-lint-langs.sh, is the linter table, keyed by language id. It is the one place core names linters (residue-exempt). - ldd-lint_test.sh runs 20 cases per fixture row in every plugin. The bindings' fixture rows add the source suffix, the comment forms, a test file name, the file a count leaves out, and a linter config. - The skills and routing tables use the scalars in place of 400, 450, 600, 8-12 and 13, and the package-size procedure runs the gap linter. - Remove the Go plugin's hooks/check-package-sizes.sh. It was never registered, so it never ran. Co-Authored-By: Claude Opus 5.5 --- Taskfile.yaml | 6 +- core/README.md | 30 +- core/includes/scripts/ldd-lint-langs.sh | 157 +++++ .../refactoring/file-and-package-routing.md | 18 +- core/scripts/ldd-lint.sh | 256 +++++++ core/scripts/ldd-lint_test.sh | 251 +++++++ core/skills/refactoring/reference.md | 6 +- docs/eval-fixture.md | 7 +- docs/generator.md | 5 +- docs/language-residue.md | 9 +- docs/token-budget.md | 2 +- go-linter-driven-development/CHANGELOG.md | 26 + go-linter-driven-development/README.md | 2 +- .../agents/lint-fixer.md | 2 +- .../hooks/check-package-sizes.sh | 79 --- go-linter-driven-development/hooks/hooks.json | 3 - .../scripts/ldd-detect_test.sh | 29 + .../scripts/ldd-lint.sh | 518 ++++++++++++++ .../scripts/ldd-lint_test.sh | 335 +++++++++ .../scripts/ldd-scope_test.sh | 29 + .../skills/code-designing/SKILL.md | 2 +- .../skills/refactoring/SKILL.md | 2 +- .../skills/refactoring/reference.md | 23 +- lang/generic/passthrough/CHANGELOG.md | 20 + lang/generic/passthrough/README.md | 4 +- lang/generic/profile.yaml | 7 + lang/generic/scripts/ldd-fixture.sh | 45 ++ lang/go/agents/lint-fixer/routing-table.md | 2 +- lang/go/passthrough/CHANGELOG.md | 26 + lang/go/passthrough/README.md | 2 +- .../passthrough/hooks/check-package-sizes.sh | 79 --- lang/go/passthrough/hooks/hooks.json | 3 - lang/go/profile.yaml | 7 + lang/go/scripts/ldd-fixture.sh | 29 + .../skills/code-designing/linter-triggers.md | 2 +- .../refactoring/file-and-package-routing.md | 17 +- lang/go/skills/refactoring/routing-table.md | 2 +- lang/python/passthrough/CHANGELOG.md | 20 + lang/python/profile.yaml | 7 + lang/python/scripts/ldd-fixture.sh | 29 + .../skills/code-designing/linter-triggers.md | 2 +- .../refactoring/file-and-package-routing.md | 17 +- .../skills/refactoring/routing-table.md | 2 +- .../agents/lint-fixer/routing-table.md | 2 +- lang/ts-react/passthrough/CHANGELOG.md | 20 + lang/ts-react/profile.yaml | 7 + .../ts-react/rules/R5/falsifying-questions.md | 2 +- lang/ts-react/scripts/ldd-fixture.sh | 30 + .../skills/code-designing/linter-triggers.md | 2 +- .../refactoring/file-and-package-routing.md | 17 +- .../skills/refactoring/routing-table.md | 2 +- linter-driven-development/CHANGELOG.md | 20 + linter-driven-development/README.md | 4 +- .../scripts/ldd-detect_test.sh | 45 ++ linter-driven-development/scripts/ldd-lint.sh | 634 ++++++++++++++++++ .../scripts/ldd-lint_test.sh | 391 +++++++++++ .../scripts/ldd-scope_test.sh | 45 ++ .../skills/refactoring/reference.md | 24 +- python-linter-driven-development/CHANGELOG.md | 20 + .../scripts/ldd-detect_test.sh | 29 + .../scripts/ldd-lint.sh | 521 ++++++++++++++ .../scripts/ldd-lint_test.sh | 329 +++++++++ .../scripts/ldd-scope_test.sh | 29 + .../skills/code-designing/SKILL.md | 2 +- .../skills/refactoring/SKILL.md | 2 +- .../skills/refactoring/reference.md | 23 +- .../ldd-gen/internal/binding/binding_test.go | 3 + tools/ldd-gen/internal/gen/gen_test.go | 3 + tools/ldd-gen/internal/profile/profile.go | 31 + .../ldd-gen/internal/profile/profile_test.go | 10 + tools/ldd-gen/internal/render/render_test.go | 3 + tools/ldd-gen/main_test.go | 3 + .../CHANGELOG.md | 20 + .../agents/lint-fixer.md | 2 +- .../rules/R5-vertical-slice.md | 2 +- .../scripts/ldd-detect_test.sh | 30 + .../scripts/ldd-lint.sh | 531 +++++++++++++++ .../scripts/ldd-lint_test.sh | 333 +++++++++ .../scripts/ldd-scope_test.sh | 30 + .../skills/code-designing/SKILL.md | 2 +- .../skills/refactoring/SKILL.md | 2 +- .../skills/refactoring/reference.md | 23 +- 82 files changed, 5073 insertions(+), 274 deletions(-) create mode 100644 core/includes/scripts/ldd-lint-langs.sh create mode 100755 core/scripts/ldd-lint.sh create mode 100644 core/scripts/ldd-lint_test.sh delete mode 100755 go-linter-driven-development/hooks/check-package-sizes.sh delete mode 100644 go-linter-driven-development/hooks/hooks.json create mode 100755 go-linter-driven-development/scripts/ldd-lint.sh create mode 100644 go-linter-driven-development/scripts/ldd-lint_test.sh delete mode 100755 lang/go/passthrough/hooks/check-package-sizes.sh delete mode 100644 lang/go/passthrough/hooks/hooks.json create mode 100755 linter-driven-development/scripts/ldd-lint.sh create mode 100644 linter-driven-development/scripts/ldd-lint_test.sh create mode 100755 python-linter-driven-development/scripts/ldd-lint.sh create mode 100644 python-linter-driven-development/scripts/ldd-lint_test.sh create mode 100755 ts-react-linter-driven-development/scripts/ldd-lint.sh create mode 100644 ts-react-linter-driven-development/scripts/ldd-lint_test.sh diff --git a/Taskfile.yaml b/Taskfile.yaml index 649a5b9d..da868ec5 100644 --- a/Taskfile.yaml +++ b/Taskfile.yaml @@ -31,7 +31,7 @@ tasks: || { echo "lint-core - the Residue section of core/README.md was rewritten; commit it"; exit 1; } test-gate: - desc: Run the fixture matrices of the generated scripts — the repo-brain gate, the review's scope bundle and its detection pass — for every plugin + desc: Run the fixture matrices of the generated scripts — the repo-brain gate, the review's scope bundle, its detection pass and the gap linter — for every plugin cmds: - bash go-linter-driven-development/scripts/check-repo-brain_test.sh - bash linter-driven-development/scripts/check-repo-brain_test.sh @@ -45,6 +45,10 @@ tasks: - bash linter-driven-development/scripts/ldd-detect_test.sh - bash python-linter-driven-development/scripts/ldd-detect_test.sh - bash ts-react-linter-driven-development/scripts/ldd-detect_test.sh + - bash go-linter-driven-development/scripts/ldd-lint_test.sh + - bash linter-driven-development/scripts/ldd-lint_test.sh + - bash python-linter-driven-development/scripts/ldd-lint_test.sh + - bash ts-react-linter-driven-development/scripts/ldd-lint_test.sh test-gen: desc: Unit tests and lint for the generator diff --git a/core/README.md b/core/README.md index 290e3633..368ce8a3 100644 --- a/core/README.md +++ b/core/README.md @@ -31,12 +31,16 @@ Go `text/template` with the default delimiters. Core files use only two construc - `{{.Plugin}}`, `{{.Lang}}`, `{{.CmdPrefix}}`, `{{.SrcGlob}}`, `{{.TestGlob}}`, `{{.ProjectMarker}}`, `{{.Nolint}}`, `{{.CommentPrefix}}`, `{{.DefaultTest}}`, `{{.DefaultLint}}`, `{{.DefaultLintFix}}`, `{{.Nil}}`, `{{.Task}}`, `{{.DocForm}}`, - `{{.DocComment}}`, `{{.SrcExt}}`, `{{.Unexported}}` — scalars from `profile.yaml`. - The last six are single words core prose uses as common nouns (the missing value, a - unit of concurrent work, the documentation comment form, the source-file suffix, - the visibility of a non-public symbol); which sentences use them, and which Go - idioms are includes instead, is recorded in `docs/language-residue.md`. A template that names a scalar the profile does not - define fails to render. + `{{.DocComment}}`, `{{.SrcExt}}`, `{{.Unexported}}`, `{{.FileMaxLines}}`, + `{{.PackageWarnFiles}}`, `{{.PackageMaxFiles}}` — scalars from `profile.yaml`. + Six are single words core prose uses as common nouns (the missing value, a unit of + concurrent work, the documentation comment form, the source-file suffix, the + visibility of a non-public symbol); which sentences use them, and which Go idioms + are includes instead, is recorded in `docs/language-residue.md`. The last three + are the gap linter's thresholds, positive integers with the warning below the + limit, so the prose that names a limit and the script that applies it cannot + disagree. A template that names a scalar the profile does not define fails to + render. - `{{include "rules/R1/canonical-example.md"}}` — the body of that file under `lang//`, with exactly one trailing newline removed. When the binding has no file of that name, the body comes from `core/includes/` under the same path: the @@ -128,8 +132,8 @@ unchanged. Promotion into `core/` happens when a second language needs the text: own that says so and points at the repository's test utilities; the Python binding ships a short pytest catalogue in the same shape, and the ts-react binding a Vitest, React Testing Library and MSW catalogue. -- `README.md`, `CHANGELOG.md`, `.claude-plugin/plugin.json`, `hooks/`: describe or - configure the Go plugin itself. +- `README.md`, `CHANGELOG.md`, `.claude-plugin/plugin.json`: describe or configure + the Go plugin itself. Three promoted pieces carry a note here because the Go plugin renders them differently from the other bindings: @@ -173,6 +177,16 @@ differently from the other bindings: by the language of each file, a group goes to the plugin installed for its language, else to this plugin's block, else to the generic plugin, else it is excluded and every file of it named. +- `scripts/ldd-lint.sh` and `scripts/ldd-lint_test.sh`, the gap linter and its + fixture matrix, are core templates. The linter measures lines per source file and + source files per directory, where the repository's linter does not hold them. It + reads the language table and the language block, and a third include with a core + default, `scripts/ldd-lint-langs.sh`, the linter table: per language id, the + comment syntax, the files a directory count leaves out, and where the repository's + own linter config sets a file-length limit and how that linter counts. The table is + the one place core names linters, and its `residue-exempt:` marker says so. The + matrix reads the binding's `scripts/ldd-fixture.sh` for the source suffix, the + comment forms, a test file's name and a linter config per row. - `scripts/check-repo-brain_test.sh` is a core template whose cases read one include, `scripts/repo-brain-fixture.sh`: the fixture rows (marker, code file, second package, sub-project) for every language the rendered gate's adapter serves. The diff --git a/core/includes/scripts/ldd-lint-langs.sh b/core/includes/scripts/ldd-lint-langs.sh new file mode 100644 index 00000000..9ac1a2f3 --- /dev/null +++ b/core/includes/scripts/ldd-lint-langs.sh @@ -0,0 +1,157 @@ +# ==================== linter table ==================== +# What the gap linter needs to know per language id: how a comment is written, +# which source files a directory count leaves out, and where the repository's +# own linter sets a file-length limit. The same table is rendered into every +# plugin, so two plugins measure one file the same way. +# residue-exempt: the table names every language's linter and config file on +# purpose, the same in every rendering. +# +# lint_comment_style c (// and /* */), d (also /+ +/) or hash (#) +# lint_package_skip_re ERE over a relative path: source files a +# directory count leaves out (an index module, +# a package marker, generated stubs); "" for none +# lint_file_length_config one line, from the repository root: +# covered TAB TAB TAB +# TAB +# unreadable TAB +# none +# lint_file_length_snippet the setting that makes the repository's +# linter hold the limit, or how to set it when +# no linter rule is known + +lint_comment_style() { + case "$1" in + python|ruby|shell) printf 'hash' ;; + d) printf 'd' ;; + *) printf 'c' ;; + esac +} + +lint_package_skip_re() { + case "$1" in + go) printf '%s' '(_gen|\.pb)\.go$' ;; + python) printf '%s' '(^|/)(__init__|conftest)\.py$|_pb2(_grpc)?\.py$' ;; + typescript|javascript) printf '%s' '(^|/)index\.[cm]?[jt]sx?$' ;; + esac + return 0 +} + +# _lint_window — the key's line and the lines after it, up to +# the next rule entry or 15 lines, joined by spaces; one window per match. The +# patterns reach awk as strings, so they use bracket expressions, never +# backslash escapes, which awk -v would rewrite. +_lint_window() { + awk -v key="$1" -v q="'" ' + BEGIN { + entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" + qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" + } + function flush() { if (w != "") print w; w = ""; n = 0 } + { + if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() + if ($0 ~ key) { flush(); n = 1; w = $0; next } + if (n > 0) { w = w " " $0; if (++n > 15) flush() } + } + END { flush() } + ' "$2" +} + +# _lint_number — the first match's digits +_lint_number() { + printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' +} + +# _lint_off — exit 0 when the rule in the window is switched off +_lint_off() { + printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" +} + +_lint_config_go() { + local f w max sc sb unread="" + for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do + [[ -f "$f" ]] || continue + grep -q 'file-length-limit' "$f" 2>/dev/null || continue + case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac + while IFS= read -r w; do + [[ "$w" =~ disabled:[[:space:]]*true ]] && continue + max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") + if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi + (( max > 0 )) || continue + sc=0; sb=0 + [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +_lint_config_eslint() { + local f w max sc sb + for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do + [[ -f "$f" ]] || continue + grep -q 'max-lines' "$f" 2>/dev/null || continue + # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") + printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" + return 0 + done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") + # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") + sc=0; sb=0 + [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") + done + printf 'none\n' +} + +_lint_config_python() { + local f line max unread="" + for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do + [[ -f "$f" ]] || continue + line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) + [[ -n "$line" ]] || continue + max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") + if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi + (( max > 0 )) || continue + # pylint counts every line of the module. + printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" + return 0 + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +lint_file_length_config() { + case "$1" in + go) _lint_config_go ;; + typescript|javascript) _lint_config_eslint ;; + python) _lint_config_python ;; + *) printf 'none\n' ;; + esac +} + +lint_file_length_snippet() { + case "$1" in + go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; + typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; + python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; + *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + esac +} +# ================== end linter table ================== \ No newline at end of file diff --git a/core/includes/skills/refactoring/file-and-package-routing.md b/core/includes/skills/refactoring/file-and-package-routing.md index 3740162a..081e6705 100644 --- a/core/includes/skills/refactoring/file-and-package-routing.md +++ b/core/includes/skills/refactoring/file-and-package-routing.md @@ -1,4 +1,4 @@ -**A file-length limit (the linter's, or ~450 lines when it has none):** +**A file over the file-length limit** — the repository linter's limit where it sets one, else {{.FileMaxLines}} code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: | File pattern | Action | |---|---| @@ -7,15 +7,19 @@ | Long functions, few types | Storify → extract functions (R3) | -**Package-size zones** — count the non-test, non-generated source files in one -directory: +**Package-size zones** — source files per directory: ``` -find -maxdepth 1 -type f -name '{{.SrcGlob}}' -not -name '*{{.TestGlob}}' | wc -l +bash /../../scripts/ldd-lint.sh package-size --root . ``` -≤7 green — fine. 8–12 yellow — design review *before the next file lands*. ≥13 red — -**must decompose**. Either zone: run the 3-step design review under "Package -decomposition", printed with this range (it is a *design* review — missing domain types are the +The gap linter counts the non-test, non-generated source files of each directory; +a `.ldd-lint.yaml` at the repository root sets other sizes, and the first line of +its output says which sizes applied. + +Below {{.PackageWarnFiles}} files green — fine. {{.PackageWarnFiles}} or more yellow — design review +*before the next file lands*. {{.PackageMaxFiles}} or more red — **must decompose**. Either +zone: run the 3-step design review under "Package decomposition", printed with this +range (it is a *design* review — missing domain types are the disease, file count the symptom). Invoke @code-designing to validate extracted types. \ No newline at end of file diff --git a/core/scripts/ldd-lint.sh b/core/scripts/ldd-lint.sh new file mode 100755 index 00000000..43407714 --- /dev/null +++ b/core/scripts/ldd-lint.sh @@ -0,0 +1,256 @@ +#!/usr/bin/env bash +# Gap linter of the {{.Plugin}} plugin. +# +# Measures the design triggers no linter holds, or that the repository's linter +# does not hold yet: lines per source file and source files per directory. It +# never edits a file. Where the repository's linter holds a limit, that limit +# is the one applied, counted the way that linter counts. +# +# Usage: bash scripts/ldd-lint.sh [options] +# file-length source files over the file-length limit +# package-size directories at or over the warning size or the limit +# --root the repository (default: cwd) +# --files measure the source files this list names, one +# relative path per line (a scope bundle's files.txt +# works: a "(not bundled: …)" suffix is ignored); +# default: every source file of the repository +# --tsv findings as rows: path TAB line TAB message, line 0; +# the threshold lines then go to stderr +# --glob source-file glob, for a language the table does not +# know ({{.Lang}} block below) +# --test-re test-file pattern over relative paths +# +# Thresholds, per language, first that applies — the "threshold:" line says +# which one a run used: +# 1. the repository's linter config holds the rule: its limit, counted the +# way that linter counts (file-length only; no linter counts files per +# directory); a config the table cannot read is named, then skipped +# 2. .ldd-lint.yaml at the repository root: file_max_lines, +# package_warn_files, package_max_files, each a positive integer +# 3. the plugin's defaults: {{.FileMaxLines}} code lines per file (blank and +# comment-only lines not counted); a warning at {{.PackageWarnFiles}} source files per +# directory and the limit at {{.PackageMaxFiles}}; for file-length, the line also +# prints the setting that makes the repository's linter hold the limit +# +# What is measured: the language's source files, minus test files, generated +# files and the directories the language block excludes. A directory count +# also leaves out the files the linter table names (an index module, a package +# marker, generated stubs). A file is over the limit when its count is above +# it; a directory is in the warning zone at the warning size and over the +# limit at the limit. +# +# Exit codes: 0 measured, with or without findings · 2 usage error, a --files +# list that does not exist, or a .ldd-lint.yaml value that is not a +# positive integer (inconclusive, never silently clean) +# +# Uses only POSIX-portable tools plus git when the root is a repository: +# grep, sed, awk, sort, find, head. + +set -u + +SCRIPT_NAME="ldd-lint" +DEFAULT_FILE_MAX_LINES={{.FileMaxLines}} +DEFAULT_PACKAGE_WARN_FILES={{.PackageWarnFiles}} +DEFAULT_PACKAGE_MAX_FILES={{.PackageMaxFiles}} +OVERRIDE_FILE=".ldd-lint.yaml" +ROOT="$(pwd)" +FILES_LIST="" +TSV=0 +OPT_GLOB="" +OPT_TEST_RE="" +SUBCOMMAND="" + +usage() { sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//' >&2; exit 2; } +die() { echo "$SCRIPT_NAME: $*" >&2; exit 2; } + +while (( $# > 0 )); do + case "$1" in + file-length|package-size) [[ -n "$SUBCOMMAND" ]] && die "one subcommand, got also: $1"; SUBCOMMAND="$1"; shift ;; + --root) ROOT="${2:-}"; shift 2 ;; + --files) FILES_LIST="${2:-}"; shift 2 ;; + --tsv) TSV=1; shift ;; + --glob) OPT_GLOB="${2:-}"; shift 2 ;; + --test-re) OPT_TEST_RE="${2:-}"; shift 2 ;; + -h|--help) usage ;; + *) die "unknown argument: $1 (see --help)" ;; + esac +done +[[ -n "$SUBCOMMAND" ]] || die "name a subcommand: file-length or package-size (see --help)" +[[ -d "$ROOT" ]] || die "not a directory: $ROOT" +if [[ -n "$FILES_LIST" ]]; then + [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" + FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") +fi +cd "$ROOT" || exit 2 + +{{include "scripts/ldd-langs.sh"}} + +{{include "scripts/ldd-lang.sh"}} + +{{include "scripts/ldd-lint-langs.sh"}} + +# ---------- output ---------- +# A threshold line is for the reader of the report; with --tsv the rows are +# the output and the threshold lines go to stderr. +say() { if (( TSV )); then printf '%s\n' "$*" >&2; else printf '%s\n' "$*"; fi; } +finding() { # + if (( TSV )); then printf '%s\t0\t%s\n' "$1" "$2"; else printf '%s: %s\n' "$1" "$2"; fi +} + +# ---------- the repository override ---------- +# override — the key's value in .ldd-lint.yaml, "" when absent; a value +# that is not a positive integer ends the run, because the file is the +# repository's own statement of its limits. +override() { + [[ -f "$OVERRIDE_FILE" ]] || return 0 + local v + v=$(awk -v k="$1" -v q="'" ' + { line = $0; sub(/#.*/, "", line) } + line ~ "^[[:space:]]*" k "[[:space:]]*:" { + sub("^[[:space:]]*" k "[[:space:]]*:[[:space:]]*", "", line) + gsub("[\"" q "[:space:]]", "", line) + print line; exit + }' "$OVERRIDE_FILE") + [[ -z "$v" ]] && return 0 + [[ "$v" =~ ^[0-9]+$ ]] && (( v > 0 )) || die "$OVERRIDE_FILE: $1 must be a positive integer, got: $v" + printf '%s' "$v" +} + +# ---------- the scope ---------- +# One "id TAB path" line per source file of a language this plugin reviews, +# sorted, so two runs over one tree print the same thing. +list_candidates() { + if [[ -n "$FILES_LIST" ]]; then + sed -e 's/ (not bundled:.*$//' -e 's|^\./||' "$FILES_LIST" + elif git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + git ls-files -co --exclude-standard + else + find . -type f -not -path './.git/*' | sed 's|^\./||' + fi +} + +SCOPE=$(mktemp) +trap 'rm -f "$SCOPE"' EXIT +while IFS= read -r f; do + [[ -n "$f" && -f "$f" ]] || continue + id=$(lang_of_path "$f") + [[ -n "$id" ]] || continue + case " $LANG_NATIVE " in *" $id "*) ;; *) continue ;; esac + printf '%s\t%s\n' "$id" "$f" +done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" + +# measured — exit 0 when the file counts for this language: not under +# an excluded directory, not a test, not generated. +measured() { + printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 + lang_is_test "$1" && return 1 + head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + return 0 +} + +# ---------- file-length ---------- +# COUNT_AWK prints a file's line count under the style's comment syntax: +# skipc drops comment-only lines (block comments included), skipb blank lines. +COUNT_AWK=' +function comment_only(t, rest, i) { + if (inblock) { + i = index(t, closer) + if (i == 0) return 1 + inblock = 0 + rest = substr(t, i + length(closer)); sub(/^[[:space:]]+/, "", rest) + return rest == "" || comment_only(rest) + } + if (style == "hash") return substr(t, 1, 1) == "#" + if (substr(t, 1, 2) == "//") return 1 + if (substr(t, 1, 2) == "/*" || (style == "d" && substr(t, 1, 2) == "/+")) { + closer = (substr(t, 2, 1) == "+") ? "+/" : "*/" + inblock = 1 + return comment_only(substr(t, 3)) + } + return 0 +} +{ + t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) + if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } + c = comment_only(t) + if (c && skipc) next + n++ +} +END { print n + 0 } +' + +run_file_length() { # — every measured file of that language + local id="$1" cfg status max sc sb label src unit f n o + cfg=$(lint_file_length_config "$id") + IFS=$'\t' read -r status max sc sb label <<< "$cfg" + if [[ "$status" == "unreadable" ]]; then + # an unreadable row carries the rule and file in its second field + say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" + status=none + fi + if [[ "$status" == "covered" ]]; then + src="$label" + say "threshold: $id file length — covered by $label (limit $max)" + else + sc=1; sb=1 + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" + else + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + fi + fi + unit="lines" + (( sc && sb )) && unit="code lines" + (( sc && ! sb )) && unit="lines without comments" + (( ! sc && sb )) && unit="non-blank lines" + while IFS=$'\t' read -r _ f; do + measured "$f" || continue + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") +} + +# ---------- package-size ---------- +run_package_size() { # — every directory that holds a measured file of that language + local id="$1" warn lim src ow om skip d n f + ow=$(override package_warn_files) || exit 2 + om=$(override package_max_files) || exit 2 + warn="${ow:-$DEFAULT_PACKAGE_WARN_FILES}"; lim="${om:-$DEFAULT_PACKAGE_MAX_FILES}" + (( warn < lim )) || die "$OVERRIDE_FILE: package_warn_files ($warn) must be below package_max_files ($lim)" + if [[ -n "$ow$om" ]]; then src="$OVERRIDE_FILE"; else src="profile default"; fi + say "threshold: $id package size — $src (warning at $warn source files per directory, limit $lim); no linter counts files per directory" + skip=$(lint_package_skip_re "$id") + while IFS= read -r d; do + n=0 + while IFS= read -r f; do + f="${f#./}" + [[ "$(lang_of_path "$f")" == "$id" ]] || continue + [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue + measured "$f" || continue + n=$((n + 1)) + done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + if (( n >= lim )); then + finding "$d" "$n source files (limit $lim, $src)" + elif (( n >= warn )); then + finding "$d" "$n source files (warning at $warn, $src)" + fi + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) +} + +# ---------- run ---------- +ids=$(cut -f1 "$SCOPE" | LC_ALL=C sort -u) +if [[ -z "$ids" ]]; then + say "$SCRIPT_NAME: no source file in scope" + exit 0 +fi +for id in $ids; do + lang_configure "$id" || continue + case "$SUBCOMMAND" in + file-length) run_file_length "$id" ;; + package-size) run_package_size "$id" ;; + esac +done +exit 0 diff --git a/core/scripts/ldd-lint_test.sh b/core/scripts/ldd-lint_test.sh new file mode 100644 index 00000000..412b401b --- /dev/null +++ b/core/scripts/ldd-lint_test.sh @@ -0,0 +1,251 @@ +#!/usr/bin/env bash +# Fixture matrix for ldd-lint.sh — the gap linter's own conformance test. +# +# Each case builds a throwaway tree under a temp dir, runs the script against +# it and asserts the exit code and the lines it prints. The cases are the same +# for every language; the language-specific parts — the source suffix, how a +# comment and a test file are written, the file a directory count leaves out, +# and the repository linter config that holds a file-length limit — come from +# the fixture rows spliced in below. A script whose language block serves +# several languages runs the whole matrix once per row. +# +# Usage: bash scripts/ldd-lint_test.sh [path/to/ldd-lint.sh] +# (default: the sibling ldd-lint.sh) +# Exit: 0 all cases pass · 1 any failure (details on stdout) +# +# Uses only POSIX-portable tools, like the script under test. + +set -u + +HERE=$(cd "$(dirname "$0")" && pwd) +LINT="${1:-$HERE/ldd-lint.sh}" +[[ -f "$LINT" ]] || { echo "no such script: $LINT" >&2; exit 2; } +LINT=$(cd "$(dirname "$LINT")" && pwd)/$(basename "$LINT") +MAX_LINES={{.FileMaxLines}} +WARN_FILES={{.PackageWarnFiles}} +MAX_FILES={{.PackageMaxFiles}} + +pass=0 failed=0 skipped=0 +CASE="" +OUT="" ERR="" CODE=0 +ROW="" +REPO="" +TAB=$(printf '\t') + +# ---------- harness ---------- +begin() { CASE="$1"; [[ -n "$ROW" ]] && CASE="[$ROW] $1"; REPO=$(mktemp -d); } +finish() { rm -rf "$REPO"; } + +run_lint() { # [options...] — runs the script over $REPO + local errf; errf=$(mktemp) + OUT=$(cd "$REPO" && bash "$LINT" "$@" --root . 2>"$errf"); CODE=$? + ERR=$(cat "$errf"); rm -f "$errf" +} + +ok() { pass=$((pass + 1)); echo "PASS $CASE"; } +bad() { failed=$((failed + 1)); echo "FAIL $CASE — $1"; echo " exit=$CODE"; printf '%s\n' "$OUT" "$ERR" | sed 's/^/ | /'; } +skip() { skipped=$((skipped + 1)); echo "SKIP $CASE — $1"; } + +expect_exit() { (( CODE == $1 )) || { bad "expected exit $1"; return 1; }; } +expect_has() { # — in stdout or stderr + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" || { bad "missing: $1"; return 1; } +} +expect_not() { + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" && { bad "unexpected: $1"; return 1; } + return 0 +} +expect_no_finding() { # no line names a file or directory of the tree + printf '%s\n' "$OUT" | grep -qE '^[^ ]+: [0-9]+ ' && { bad "unexpected finding"; return 1; } + return 0 +} + +# ---------- fixture builders ---------- +# write_src [ [ []]] — a source +# file with that many code lines, comment-only lines, blank lines and lines of +# one block comment, in that order. +write_src() { + local path="$REPO/$1" code="$2" com="${3:-0}" blank="${4:-0}" block="${5:-0}" i + mkdir -p "$(dirname "$path")" + { + for (( i = 1; i <= code; i++ )); do printf 'v%d = %d\n' "$i" "$i"; done + for (( i = 1; i <= com; i++ )); do printf '%s\n' "$FX_LINE_COMMENT"; done + for (( i = 1; i <= blank; i++ )); do printf '\n'; done + if (( block > 0 )); then + printf '%s\n' "$FX_BLOCK_OPEN" + for (( i = 3; i <= block; i++ )); do printf ' inside the block\n'; done + printf '%s\n' "$FX_BLOCK_CLOSE" + fi + } > "$path" +} + +# fill_dir [] — that many one-line source and test files +fill_dir() { + local i + for (( i = 1; i <= $2; i++ )); do write_src "$1/m$i$FX_EXT" 1; done + for (( i = 1; i <= ${3:-0}; i++ )); do write_src "$(fx_test_file "$1" "$i")" 1; done +} + +{{include "scripts/ldd-fixture.sh"}} + +# ========================= cases ========================= +run_row_cases() { # runs every case against the current row + +begin "a file at the limit passes and one line over is a finding" +write_src "src/at$FX_EXT" "$MAX_LINES"; write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" \ + && expect_not "src/at$FX_EXT:" && ok +finish + +begin "comment-only and blank lines are not counted by the plugin's default" +write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 20 20 +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the lines of a block comment are comment-only" +if [[ -z "$FX_BLOCK_OPEN" ]]; then skip "the language has no block comment"; finish +else + write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 0 0 30 + run_lint file-length + expect_exit 0 && expect_no_finding && ok + finish +fi + +begin "a test file over the limit is not measured" +write_src "$(fx_test_file src 1)" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "a generated file over the limit is not measured" +write_src "src/gen$FX_EXT" $((MAX_LINES + 50)) +{ printf '%s DO NOT EDIT\n' "$FX_LINE_COMMENT"; cat "$REPO/src/gen$FX_EXT"; } > "$REPO/src/gen.tmp" && mv "$REPO/src/gen.tmp" "$REPO/src/gen$FX_EXT" +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the plugin's default names the setting that makes the linter hold it" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +finish + +begin "the repository's linter config holds the limit and is counted its way" +fx_write_linter_config 100 +write_src "src/a$FX_EXT" 90 20 +run_lint file-length +expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +finish + +begin "a linter config whose limit cannot be read is named, and the next source applies" +if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish +else + write_src "src/over$FX_EXT" $((MAX_LINES + 1)) + run_lint file-length + expect_exit 0 && expect_has "its limit cannot be read here" \ + && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok + finish +fi + +begin ".ldd-lint.yaml sets the limit when the linter holds none" +printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 101 +run_lint file-length +expect_exit 0 && expect_has "set by .ldd-lint.yaml" && expect_has "src/a$FX_EXT: 101 code lines (limit 100, .ldd-lint.yaml)" && ok +finish + +begin "a .ldd-lint.yaml value that is not a positive integer is inconclusive (exit 2)" +printf 'file_max_lines: lots\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 2 && expect_has "file_max_lines must be a positive integer" && ok +finish + +begin "a directory below the warning size is not a finding" +fill_dir pkg $((WARN_FILES - 1)) 3 +write_src "pkg/$FX_SKIP_FILE" 1 +run_lint package-size +expect_exit 0 && expect_no_finding && ok +finish + +begin "a directory at the warning size is a warning" +fill_dir pkg "$WARN_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $WARN_FILES source files (warning at $WARN_FILES, profile default)" && ok +finish + +begin "a directory at the limit is over the limit" +fill_dir pkg "$MAX_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $MAX_FILES source files (limit $MAX_FILES, profile default)" && ok +finish + +begin ".ldd-lint.yaml sets the directory sizes" +printf 'package_warn_files: 2\npackage_max_files: 3\n' > "$REPO/.ldd-lint.yaml" +fill_dir small 2; fill_dir big 3 +run_lint package-size +expect_exit 0 && expect_has "small: 2 source files (warning at 2, .ldd-lint.yaml)" \ + && expect_has "big: 3 source files (limit 3, .ldd-lint.yaml)" && ok +finish + +begin "--tsv writes path, line 0 and message as rows, and the threshold to stderr" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length --tsv +expect_exit 0 \ + && { [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}$((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" ]] || bad "stdout is not the one row"; } \ + && [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}"* ]] \ + && { [[ "$ERR" == "threshold: "* ]] || bad "threshold line not on stderr"; } && [[ "$ERR" == "threshold: "* ]] && ok +finish + +begin "--files measures only the files it names" +write_src "src/named$FX_EXT" $((MAX_LINES + 1)); write_src "src/other$FX_EXT" $((MAX_LINES + 1)) +printf 'src/named%s (not bundled: %s lines)\n' "$FX_EXT" $((MAX_LINES + 1)) > "$REPO/files.txt" +run_lint file-length --files files.txt +expect_exit 0 && expect_has "src/named$FX_EXT:" && expect_not "src/other$FX_EXT:" && ok +finish + +begin "in a git repository an ignored file is not measured" +write_src "src/kept$FX_EXT" $((MAX_LINES + 1)); write_src "build/out$FX_EXT" $((MAX_LINES + 1)) +printf 'build/\n' > "$REPO/.gitignore" +(cd "$REPO" && git init -q) +run_lint file-length +expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok +finish + +begin "two runs print the same output" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" +run_lint package-size; first="$OUT" +run_lint package-size +expect_exit 0 && { [[ "$OUT" == "$first" ]] || bad "the second run printed something else"; } && [[ "$OUT" == "$first" ]] && ok +finish + +begin "a tree without a source file is measured clean" +printf 'notes\n' > "$REPO/README.md" +run_lint file-length +expect_exit 0 && expect_has "no source file in scope" && ok +finish + +} # run_row_cases + +for ROW in $FX_ROWS; do + use_row "$ROW" + run_row_cases +done +ROW="" + +# --- row-independent --- +begin "a missing subcommand is a usage error (exit 2)" +run_lint +expect_exit 2 && expect_has "name a subcommand" && ok +finish + +begin "a --files list that does not exist is a usage error (exit 2)" +run_lint file-length --files no-such-list.txt +expect_exit 2 && expect_has "no such file list" && ok +finish + +# ========================= summary ========================= +echo +echo "ldd-lint_test: $pass passed, $failed failed, $skipped skipped" +(( failed == 0 )) diff --git a/core/skills/refactoring/reference.md b/core/skills/refactoring/reference.md index 759d98fc..5fc427c2 100644 --- a/core/skills/refactoring/reference.md +++ b/core/skills/refactoring/reference.md @@ -243,9 +243,9 @@ composition: @code-designing. ### Package decomposition -Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone (≥13 non-test `{{.SrcExt}}` -files at one directory level) or yellow zone (8–12) — detection command and zone -table in `` above. +Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone ({{.PackageMaxFiles}} or more non-test +`{{.SrcExt}}` files at one directory level) or yellow zone ({{.PackageWarnFiles}} or more) — the +measuring command and the zones are in `` above. **A package-size violation is a design review, not a mechanical file split.** File count is the symptom; the disease is usually missing domain types or multiple diff --git a/docs/eval-fixture.md b/docs/eval-fixture.md index 8c0aafa8..29b6de4d 100644 --- a/docs/eval-fixture.md +++ b/docs/eval-fixture.md @@ -71,10 +71,9 @@ Ground rules, all enforced: - **House style is plain.** Standard-library testing and logging (pytest in Python), a Taskfile, no assertion library. Anything the agent introduces beyond that is a when-in-Rome finding it must raise on itself. -- **Everything lives under `internal/`, `cmd/` and `pkg/`** in go-mini because the - Go plugin's package-size hook scans only those directories; py-mini keeps - `internal/` and puts the entry point in `svc/`, since a top-level `cmd` package - would shadow the standard library module of that name. +- **Everything lives under `internal/`, `cmd/` and `pkg/`** in go-mini, the usual Go + layout; py-mini keeps `internal/` and puts the entry point in `svc/`, since a + top-level `cmd` package would shadow the standard library module of that name. ## The manifest `go/violations.yaml`, beside the fixture, is the answer key, kept outside the scaffolded tree. One diff --git a/docs/generator.md b/docs/generator.md index 150feef8..2eba3d90 100644 --- a/docs/generator.md +++ b/docs/generator.md @@ -58,9 +58,8 @@ plugin's `evals/` directory at run time, are left alone by both `check` and ## The generator `ldd-gen` is a small Go module under `tools/ldd-gen` with its own `go.mod`, so the -repository root stays free of any Go module or workspace file: the plugin's own -package-size hook stays quiet here, and Go modules checked out below the root (the -evals clone, the eval fixture) build normally. The tasks run the tool from its own +repository root stays free of any Go module or workspace file, and Go modules +checked out below the root (the evals clone, the eval fixture) build normally. The tasks run the tool from its own directory and pass the repository root with `-root`. Inside, `Profile` parses and validates the binding's scalars, `Binding` resolves includes, overrides and passthrough files, `Render` produces the plugin tree in memory, and `Compare` and `Write` sync that tree with the directory on disk. diff --git a/docs/language-residue.md b/docs/language-residue.md index 0289e96d..796cea65 100644 --- a/docs/language-residue.md +++ b/docs/language-residue.md @@ -470,10 +470,11 @@ coupling point, whether or not it looks like language: `ESCALATED:`, `LINT STATUS:`, the `Stop check` lines, `R: finding(s)`, the category headers, `🔗 CLUSTER:` — are the report contract; no binding include rewrites them. -- **Hooks and the repo-brain adapter are binding files**, not core. - `hooks/check-package-sizes.sh` counts Go files; the gate's adapter block hardcodes - the same globs and marker the profile holds. A binding that ships either writes its - own; the generic binding's adapter dispatches on the detected language. +- **The repo-brain adapter is a binding file**, not core. The gate's adapter block + hardcodes the same globs and marker the profile holds; a binding writes its own, + and the generic binding's adapter dispatches on the detected language. The gap + linter is core: what it needs per language, linter names included, sits in one + shared table keyed by language id, the way the language table does. ## How the scanner enforces this diff --git a/docs/token-budget.md b/docs/token-budget.md index bbf646b2..18b53d3f 100644 --- a/docs/token-budget.md +++ b/docs/token-budget.md @@ -346,7 +346,7 @@ count. A procedure the model paraphrases is a procedure that drifts. The plugin already ships the pattern. R9's gate is `check-repo-brain.sh`, a core template rendered per binding with the language adapter as an include and tested by -its own fixture matrix; the package-size gate is a hook. S8 adds two scripts of the +its own fixture matrix. S8 adds two scripts of the same shape under the plugin's scripts directory, and the machine-readable contract they need in the rules' question files. diff --git a/go-linter-driven-development/CHANGELOG.md b/go-linter-driven-development/CHANGELOG.md index 01649b62..bcfe2328 100644 --- a/go-linter-driven-development/CHANGELOG.md +++ b/go-linter-driven-development/CHANGELOG.md @@ -5,6 +5,32 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio ## [Unreleased] +### Added + +- **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` + measures lines per source file (`file-length`) and source files per directory + (`package-size`). Where the repository's linter config holds the file-length + limit, the script uses that limit and counts lines the way that linter counts. + Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and + otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 + source files per directory and the limit at 13. The first line of the output + names the limit that applied; with the defaults it also prints the setting that + makes the repository's linter hold the limit. `--tsv` writes findings as rows. + The script never edits a file. + +### Changed + +- **One set of thresholds.** The skills and routing tables name the same limits the + gap linter applies: the repository linter's limit where it sets one, else + 450 code lines per file and 8 and 13 files per directory. The package-size + procedure runs the gap linter in place of a hand-written `find` command. + +### Removed + +- **`hooks/check-package-sizes.sh`.** The hook was never registered, so it never + ran. The gap linter's `package-size` measures the same zones for every source + directory. + ### Fixed - **A claim that something is missing needs a listing.** A rule hunter that reports diff --git a/go-linter-driven-development/README.md b/go-linter-driven-development/README.md index 2c247160..5c15ab5a 100644 --- a/go-linter-driven-development/README.md +++ b/go-linter-driven-development/README.md @@ -26,7 +26,7 @@ go-linter-driven-development/ ├── commands/ go-ldd-analyze · autopilot · quickfix · prepare · review · status · wire-repo-brain ├── scripts/ check-repo-brain.sh — repo-brain conformance gate, installed into target repos by /wire-repo-brain │ ldd-scope.sh · ldd-detect.sh — the review's scope bundle and detection pass -└── hooks/ package-size gate +│ ldd-lint.sh — the gap linter: file length and package size where no linter holds them ``` **Five layers, one fact per fact:** diff --git a/go-linter-driven-development/agents/lint-fixer.md b/go-linter-driven-development/agents/lint-fixer.md index 3a07c071..b8d38f3e 100644 --- a/go-linter-driven-development/agents/lint-fixer.md +++ b/go-linter-driven-development/agents/lint-fixer.md @@ -60,7 +60,7 @@ this table: | `maintidx` | rules/R3-storifying.md + rules/R1-primitive-obsession.md | | `dupl` | rules/R1-primitive-obsession.md (extract shared type/logic); duplicated switches on one discriminator → rules/R11-conditional-dispatch.md | | `exhaustive` (missing enum cases) | rules/R11-conditional-dispatch.md (via @refactoring) | -| revive `file-length-limit`; package-size hook failures (`hooks/check-package-sizes.sh`) | rules/R5-vertical-slice.md | +| revive `file-length-limit`; `scripts/ldd-lint.sh file-length` and `package-size` findings | rules/R5-vertical-slice.md | | `gochecknoglobals` / `gochecknoinits` | rules/R8-no-globals.md | | `ireturn` / interface lint on single-impl interfaces | rules/R6-test-only-interfaces.md | | `go test -race` failures; `govet` `copylocks` | rules/R10-concurrency-safety.md (via @refactoring) | diff --git a/go-linter-driven-development/hooks/check-package-sizes.sh b/go-linter-driven-development/hooks/check-package-sizes.sh deleted file mode 100755 index 08a91914..00000000 --- a/go-linter-driven-development/hooks/check-package-sizes.sh +++ /dev/null @@ -1,79 +0,0 @@ -#!/usr/bin/env bash -# Portable package-size gate for the go-linter-driven-development plugin. -# -# Fires as a PostToolUse hook after Write / Edit / MultiEdit. Scans Go source -# roots under $CLAUDE_PROJECT_DIR and counts non-test, non-generated .go files -# per directory at one level deep. -# -# Thresholds (also documented in the refactoring and pre-commit-review skills): -# >=13 files = RED -> exit 2, stderr is fed back to Claude as a blocking -# error so the violation is acknowledged before more -# code lands in the oversized package. -# 8-12 files = YELLOW -> exit 0 with stdout advisory, no block. -# <=7 files = GREEN -> silent, exit 0. -# -# Guards: -# - no-op unless $CLAUDE_PROJECT_DIR/go.mod exists (not a Go project) -# - no-op if none of internal/, cmd/, pkg/ exist -# -# Uses only POSIX-portable tools: find, wc, tr, sort. No jq / python / task. - -set -u - -PROJECT_DIR="${CLAUDE_PROJECT_DIR:-$(pwd)}" - -[[ -f "$PROJECT_DIR/go.mod" ]] || exit 0 - -YELLOW_MIN=8 -RED_MIN=13 - -roots=() -for d in internal cmd pkg; do - [[ -d "$PROJECT_DIR/$d" ]] && roots+=("$PROJECT_DIR/$d") -done - -(( ${#roots[@]} == 0 )) && exit 0 - -red_lines=() -yellow_lines=() - -while IFS= read -r dir; do - [[ -z "$dir" ]] && continue - count=$(find "$dir" -maxdepth 1 -type f -name '*.go' \ - -not -name '*_test.go' \ - -not -name '*_gen.go' \ - -not -name '*.pb.go' \ - 2>/dev/null | wc -l | tr -d ' ') - - rel=${dir#$PROJECT_DIR/} - if (( count >= RED_MIN )); then - red_lines+=(" $rel: $count non-test .go files (RED zone, must decompose)") - elif (( count >= YELLOW_MIN )); then - yellow_lines+=(" $rel: $count non-test .go files (YELLOW zone, design review before next file)") - fi -done < <(find "${roots[@]}" -type d \ - -not -path '*/vendor/*' \ - -not -path '*/testdata/*' \ - 2>/dev/null | sort -u) - -if (( ${#red_lines[@]} > 0 )); then - { - echo "⛔ Package size gate — RED zone detected:" - printf '%s\n' "${red_lines[@]}" - echo "" - echo "These packages MUST be decomposed before more code lands." - echo "Apply the 3-step design review (see in the refactoring skill's reference.md):" - echo " 1. Does the package name reflect a real-world domain concept (not a role/container)?" - echo " 2. Are types well-scoped, or are there big structs hiding sub-types or primitive-obsession fields?" - echo " 3. Only after the type review, decide: sub-packages, new leaf types, or both." - } >&2 - exit 2 -fi - -if (( ${#yellow_lines[@]} > 0 )); then - echo "⚠️ Package size gate — YELLOW zone:" - printf '%s\n' "${yellow_lines[@]}" - echo "Design review recommended before the next file lands in these packages." -fi - -exit 0 diff --git a/go-linter-driven-development/hooks/hooks.json b/go-linter-driven-development/hooks/hooks.json deleted file mode 100644 index deffac97..00000000 --- a/go-linter-driven-development/hooks/hooks.json +++ /dev/null @@ -1,3 +0,0 @@ -{ - "hooks": {} -} diff --git a/go-linter-driven-development/scripts/ldd-detect_test.sh b/go-linter-driven-development/scripts/ldd-detect_test.sh index 2586b10c..8b6d5ab6 100644 --- a/go-linter-driven-development/scripts/ldd-detect_test.sh +++ b/go-linter-driven-development/scripts/ldd-detect_test.sh @@ -134,10 +134,29 @@ questions_in_rules() { # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="go" FX_NATIVE="go" use_row() { + FX_EXT='.go' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='api.pb.go' FX_GLOB='*.go' FX_SRC_FILE="services/worker.go" FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' @@ -178,6 +197,16 @@ func TestRun(t *testing.T) { GO } +fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } + +fx_write_linter_config() { # + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" +} + +fx_write_linter_config_unreadable() { + printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" +} + # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/go-linter-driven-development/scripts/ldd-lint.sh b/go-linter-driven-development/scripts/ldd-lint.sh new file mode 100755 index 00000000..d0d16c96 --- /dev/null +++ b/go-linter-driven-development/scripts/ldd-lint.sh @@ -0,0 +1,518 @@ +#!/usr/bin/env bash +# Gap linter of the go-linter-driven-development plugin. +# +# Measures the design triggers no linter holds, or that the repository's linter +# does not hold yet: lines per source file and source files per directory. It +# never edits a file. Where the repository's linter holds a limit, that limit +# is the one applied, counted the way that linter counts. +# +# Usage: bash scripts/ldd-lint.sh [options] +# file-length source files over the file-length limit +# package-size directories at or over the warning size or the limit +# --root the repository (default: cwd) +# --files measure the source files this list names, one +# relative path per line (a scope bundle's files.txt +# works: a "(not bundled: …)" suffix is ignored); +# default: every source file of the repository +# --tsv findings as rows: path TAB line TAB message, line 0; +# the threshold lines then go to stderr +# --glob source-file glob, for a language the table does not +# know (Go block below) +# --test-re test-file pattern over relative paths +# +# Thresholds, per language, first that applies — the "threshold:" line says +# which one a run used: +# 1. the repository's linter config holds the rule: its limit, counted the +# way that linter counts (file-length only; no linter counts files per +# directory); a config the table cannot read is named, then skipped +# 2. .ldd-lint.yaml at the repository root: file_max_lines, +# package_warn_files, package_max_files, each a positive integer +# 3. the plugin's defaults: 450 code lines per file (blank and +# comment-only lines not counted); a warning at 8 source files per +# directory and the limit at 13; for file-length, the line also +# prints the setting that makes the repository's linter hold the limit +# +# What is measured: the language's source files, minus test files, generated +# files and the directories the language block excludes. A directory count +# also leaves out the files the linter table names (an index module, a package +# marker, generated stubs). A file is over the limit when its count is above +# it; a directory is in the warning zone at the warning size and over the +# limit at the limit. +# +# Exit codes: 0 measured, with or without findings · 2 usage error, a --files +# list that does not exist, or a .ldd-lint.yaml value that is not a +# positive integer (inconclusive, never silently clean) +# +# Uses only POSIX-portable tools plus git when the root is a repository: +# grep, sed, awk, sort, find, head. + +set -u + +SCRIPT_NAME="ldd-lint" +DEFAULT_FILE_MAX_LINES=450 +DEFAULT_PACKAGE_WARN_FILES=8 +DEFAULT_PACKAGE_MAX_FILES=13 +OVERRIDE_FILE=".ldd-lint.yaml" +ROOT="$(pwd)" +FILES_LIST="" +TSV=0 +OPT_GLOB="" +OPT_TEST_RE="" +SUBCOMMAND="" + +usage() { sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//' >&2; exit 2; } +die() { echo "$SCRIPT_NAME: $*" >&2; exit 2; } + +while (( $# > 0 )); do + case "$1" in + file-length|package-size) [[ -n "$SUBCOMMAND" ]] && die "one subcommand, got also: $1"; SUBCOMMAND="$1"; shift ;; + --root) ROOT="${2:-}"; shift 2 ;; + --files) FILES_LIST="${2:-}"; shift 2 ;; + --tsv) TSV=1; shift ;; + --glob) OPT_GLOB="${2:-}"; shift 2 ;; + --test-re) OPT_TEST_RE="${2:-}"; shift 2 ;; + -h|--help) usage ;; + *) die "unknown argument: $1 (see --help)" ;; + esac +done +[[ -n "$SUBCOMMAND" ]] || die "name a subcommand: file-length or package-size (see --help)" +[[ -d "$ROOT" ]] || die "not a directory: $ROOT" +if [[ -n "$FILES_LIST" ]]; then + [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" + FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") +fi +cd "$ROOT" || exit 2 + +# ==================== language table ==================== +# How a file's language is told, and which plugin owns a language. The same +# table is rendered into every plugin, so two plugins that split one scope +# agree on every file. The language block below says which of these ids this +# plugin reviews itself; the rest are routed to the plugin that owns them. +# residue-exempt: the table names every language's extension and plugin on +# purpose, the same in every rendering. +# +# lang_of_path the language id of a source file, by extension: +# "custom" for a file matching --glob (then nothing +# else is a source file), "" for a file that is not +# source (a manifest, a doc, a build file) +# dedicated_plugin_for the plugin written for that language, or "" +# GENERIC_PLUGIN the plugin that reviews any language +# installed_plugin_dir where plugin is installed, from the +# installed-plugins file, or "" when it is not; +# LDD_INSTALLED_PLUGINS names another file +GENERIC_PLUGIN='linter-driven-development' + +# A .h header is C++ when the repository has C++ sources, C otherwise. +HEADER_LANG="" +header_lang() { + [[ -n "$HEADER_LANG" ]] && { printf '%s' "$HEADER_LANG"; return; } + if { git ls-files -- '*.cpp' '*.cc' '*.cxx' '*.hpp' 2>/dev/null || find . -type f \( -name '*.cpp' -o -name '*.cc' -o -name '*.cxx' -o -name '*.hpp' \) -not -path './.git/*' 2>/dev/null; } | head -1 | grep -q .; then + HEADER_LANG=cpp + else + HEADER_LANG=c + fi + printf '%s' "$HEADER_LANG" +} + +lang_of_path() { + local b="${1##*/}" + if [[ -n "${OPT_GLOB:-}" ]]; then + case "$b" in $OPT_GLOB) printf 'custom' ;; esac + return 0 + fi + case "$b" in + *.go) printf 'go' ;; + *.py) printf 'python' ;; + *.d|*.di) printf 'd' ;; + *.rs) printf 'rust' ;; + *.c) printf 'c' ;; + *.h) header_lang ;; + *.cpp|*.cc|*.cxx|*.hpp|*.hh|*.hxx) printf 'cpp' ;; + *.java) printf 'java' ;; + *.kt|*.kts) printf 'kotlin' ;; + *.rb) printf 'ruby' ;; + *.sh|*.bash) printf 'shell' ;; + *.ts|*.tsx) printf 'typescript' ;; + *.js|*.jsx|*.mjs|*.cjs) printf 'javascript' ;; + *.cs) printf 'csharp' ;; + esac + return 0 +} + +dedicated_plugin_for() { + case "$1" in + go|python) printf '%s-linter-driven-development' "$1" ;; + typescript|javascript) printf 'ts-react-linter-driven-development' ;; + esac + return 0 +} + +installed_plugin_dir() { + local f="${LDD_INSTALLED_PLUGINS:-$HOME/.claude/plugins/installed_plugins.json}" + [[ -f "$f" ]] || return 0 + awk -v key="\"$1@" ' + index($0, key) { found = 1; next } + found && /"installPath"/ { sub(/^[^:]*:[ \t]*"/, ""); sub(/".*$/, ""); print; exit } + ' "$f" +} +# ================== end language table ================== + +# ======================= language block: go ======================= +# Everything language-specific the review scripts need. The driver calls only +# the LANG_* variables and lang_* functions defined here; a build of the script +# for another language replaces this block and nothing else. Which file is +# which language is the language table above, shared by every plugin. +# +# Contract: +# LANG_NATIVE the language ids this plugin reviews itself; a group of +# any other id is routed to the plugin that owns it +# lang_configure sets the variables below for that id; "custom" is the +# --glob override, reviewed with this block's defaults; +# exit 2 for an id this block has no row for +# LANG_EXCLUDE_RE ERE over a relative path: directories never in scope +# LANG_SUPPRESS_RE ERE: a lint-suppression directive on a source line +# LANG_COMMENT_RE ERE: a source line that carries a comment +# LANG_DIRECTIVE_RE ERE: a comment line that is a directive, not prose +# LANG_GENERATED_RE ERE: a marker in a file's head that says it is generated +# lang_is_test exit 0 iff the path is a test file (--test-re overrides) +LANG_NATIVE="go custom" + +lang_configure() { + case "$1" in go|custom) ;; *) return 2 ;; esac + LANG_EXCLUDE_RE='(^|/)(vendor|\.git|testdata)/' + LANG_SUPPRESS_RE='//nolint' + LANG_COMMENT_RE='//' + LANG_DIRECTIVE_RE='//(go:|nolint| Output:|line |export |extern )' + LANG_GENERATED_RE='Code generated .* DO NOT EDIT|DO NOT EDIT|@generated' + return 0 +} + +lang_is_test() { + [[ -n "${OPT_TEST_RE:-}" ]] && { printf '%s\n' "$1" | grep -qE -- "$OPT_TEST_RE"; return; } + [[ "$1" == *_test.go ]] +} +# ===================== end language block: go ===================== + +# ==================== linter table ==================== +# What the gap linter needs to know per language id: how a comment is written, +# which source files a directory count leaves out, and where the repository's +# own linter sets a file-length limit. The same table is rendered into every +# plugin, so two plugins measure one file the same way. +# residue-exempt: the table names every language's linter and config file on +# purpose, the same in every rendering. +# +# lint_comment_style c (// and /* */), d (also /+ +/) or hash (#) +# lint_package_skip_re ERE over a relative path: source files a +# directory count leaves out (an index module, +# a package marker, generated stubs); "" for none +# lint_file_length_config one line, from the repository root: +# covered TAB TAB TAB +# TAB +# unreadable TAB +# none +# lint_file_length_snippet the setting that makes the repository's +# linter hold the limit, or how to set it when +# no linter rule is known + +lint_comment_style() { + case "$1" in + python|ruby|shell) printf 'hash' ;; + d) printf 'd' ;; + *) printf 'c' ;; + esac +} + +lint_package_skip_re() { + case "$1" in + go) printf '%s' '(_gen|\.pb)\.go$' ;; + python) printf '%s' '(^|/)(__init__|conftest)\.py$|_pb2(_grpc)?\.py$' ;; + typescript|javascript) printf '%s' '(^|/)index\.[cm]?[jt]sx?$' ;; + esac + return 0 +} + +# _lint_window — the key's line and the lines after it, up to +# the next rule entry or 15 lines, joined by spaces; one window per match. The +# patterns reach awk as strings, so they use bracket expressions, never +# backslash escapes, which awk -v would rewrite. +_lint_window() { + awk -v key="$1" -v q="'" ' + BEGIN { + entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" + qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" + } + function flush() { if (w != "") print w; w = ""; n = 0 } + { + if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() + if ($0 ~ key) { flush(); n = 1; w = $0; next } + if (n > 0) { w = w " " $0; if (++n > 15) flush() } + } + END { flush() } + ' "$2" +} + +# _lint_number — the first match's digits +_lint_number() { + printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' +} + +# _lint_off — exit 0 when the rule in the window is switched off +_lint_off() { + printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" +} + +_lint_config_go() { + local f w max sc sb unread="" + for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do + [[ -f "$f" ]] || continue + grep -q 'file-length-limit' "$f" 2>/dev/null || continue + case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac + while IFS= read -r w; do + [[ "$w" =~ disabled:[[:space:]]*true ]] && continue + max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") + if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi + (( max > 0 )) || continue + sc=0; sb=0 + [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +_lint_config_eslint() { + local f w max sc sb + for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do + [[ -f "$f" ]] || continue + grep -q 'max-lines' "$f" 2>/dev/null || continue + # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") + printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" + return 0 + done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") + # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") + sc=0; sb=0 + [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") + done + printf 'none\n' +} + +_lint_config_python() { + local f line max unread="" + for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do + [[ -f "$f" ]] || continue + line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) + [[ -n "$line" ]] || continue + max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") + if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi + (( max > 0 )) || continue + # pylint counts every line of the module. + printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" + return 0 + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +lint_file_length_config() { + case "$1" in + go) _lint_config_go ;; + typescript|javascript) _lint_config_eslint ;; + python) _lint_config_python ;; + *) printf 'none\n' ;; + esac +} + +lint_file_length_snippet() { + case "$1" in + go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; + typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; + python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; + *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + esac +} +# ================== end linter table ================== + +# ---------- output ---------- +# A threshold line is for the reader of the report; with --tsv the rows are +# the output and the threshold lines go to stderr. +say() { if (( TSV )); then printf '%s\n' "$*" >&2; else printf '%s\n' "$*"; fi; } +finding() { # + if (( TSV )); then printf '%s\t0\t%s\n' "$1" "$2"; else printf '%s: %s\n' "$1" "$2"; fi +} + +# ---------- the repository override ---------- +# override — the key's value in .ldd-lint.yaml, "" when absent; a value +# that is not a positive integer ends the run, because the file is the +# repository's own statement of its limits. +override() { + [[ -f "$OVERRIDE_FILE" ]] || return 0 + local v + v=$(awk -v k="$1" -v q="'" ' + { line = $0; sub(/#.*/, "", line) } + line ~ "^[[:space:]]*" k "[[:space:]]*:" { + sub("^[[:space:]]*" k "[[:space:]]*:[[:space:]]*", "", line) + gsub("[\"" q "[:space:]]", "", line) + print line; exit + }' "$OVERRIDE_FILE") + [[ -z "$v" ]] && return 0 + [[ "$v" =~ ^[0-9]+$ ]] && (( v > 0 )) || die "$OVERRIDE_FILE: $1 must be a positive integer, got: $v" + printf '%s' "$v" +} + +# ---------- the scope ---------- +# One "id TAB path" line per source file of a language this plugin reviews, +# sorted, so two runs over one tree print the same thing. +list_candidates() { + if [[ -n "$FILES_LIST" ]]; then + sed -e 's/ (not bundled:.*$//' -e 's|^\./||' "$FILES_LIST" + elif git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + git ls-files -co --exclude-standard + else + find . -type f -not -path './.git/*' | sed 's|^\./||' + fi +} + +SCOPE=$(mktemp) +trap 'rm -f "$SCOPE"' EXIT +while IFS= read -r f; do + [[ -n "$f" && -f "$f" ]] || continue + id=$(lang_of_path "$f") + [[ -n "$id" ]] || continue + case " $LANG_NATIVE " in *" $id "*) ;; *) continue ;; esac + printf '%s\t%s\n' "$id" "$f" +done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" + +# measured — exit 0 when the file counts for this language: not under +# an excluded directory, not a test, not generated. +measured() { + printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 + lang_is_test "$1" && return 1 + head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + return 0 +} + +# ---------- file-length ---------- +# COUNT_AWK prints a file's line count under the style's comment syntax: +# skipc drops comment-only lines (block comments included), skipb blank lines. +COUNT_AWK=' +function comment_only(t, rest, i) { + if (inblock) { + i = index(t, closer) + if (i == 0) return 1 + inblock = 0 + rest = substr(t, i + length(closer)); sub(/^[[:space:]]+/, "", rest) + return rest == "" || comment_only(rest) + } + if (style == "hash") return substr(t, 1, 1) == "#" + if (substr(t, 1, 2) == "//") return 1 + if (substr(t, 1, 2) == "/*" || (style == "d" && substr(t, 1, 2) == "/+")) { + closer = (substr(t, 2, 1) == "+") ? "+/" : "*/" + inblock = 1 + return comment_only(substr(t, 3)) + } + return 0 +} +{ + t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) + if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } + c = comment_only(t) + if (c && skipc) next + n++ +} +END { print n + 0 } +' + +run_file_length() { # — every measured file of that language + local id="$1" cfg status max sc sb label src unit f n o + cfg=$(lint_file_length_config "$id") + IFS=$'\t' read -r status max sc sb label <<< "$cfg" + if [[ "$status" == "unreadable" ]]; then + # an unreadable row carries the rule and file in its second field + say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" + status=none + fi + if [[ "$status" == "covered" ]]; then + src="$label" + say "threshold: $id file length — covered by $label (limit $max)" + else + sc=1; sb=1 + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" + else + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + fi + fi + unit="lines" + (( sc && sb )) && unit="code lines" + (( sc && ! sb )) && unit="lines without comments" + (( ! sc && sb )) && unit="non-blank lines" + while IFS=$'\t' read -r _ f; do + measured "$f" || continue + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") +} + +# ---------- package-size ---------- +run_package_size() { # — every directory that holds a measured file of that language + local id="$1" warn lim src ow om skip d n f + ow=$(override package_warn_files) || exit 2 + om=$(override package_max_files) || exit 2 + warn="${ow:-$DEFAULT_PACKAGE_WARN_FILES}"; lim="${om:-$DEFAULT_PACKAGE_MAX_FILES}" + (( warn < lim )) || die "$OVERRIDE_FILE: package_warn_files ($warn) must be below package_max_files ($lim)" + if [[ -n "$ow$om" ]]; then src="$OVERRIDE_FILE"; else src="profile default"; fi + say "threshold: $id package size — $src (warning at $warn source files per directory, limit $lim); no linter counts files per directory" + skip=$(lint_package_skip_re "$id") + while IFS= read -r d; do + n=0 + while IFS= read -r f; do + f="${f#./}" + [[ "$(lang_of_path "$f")" == "$id" ]] || continue + [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue + measured "$f" || continue + n=$((n + 1)) + done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + if (( n >= lim )); then + finding "$d" "$n source files (limit $lim, $src)" + elif (( n >= warn )); then + finding "$d" "$n source files (warning at $warn, $src)" + fi + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) +} + +# ---------- run ---------- +ids=$(cut -f1 "$SCOPE" | LC_ALL=C sort -u) +if [[ -z "$ids" ]]; then + say "$SCRIPT_NAME: no source file in scope" + exit 0 +fi +for id in $ids; do + lang_configure "$id" || continue + case "$SUBCOMMAND" in + file-length) run_file_length "$id" ;; + package-size) run_package_size "$id" ;; + esac +done +exit 0 diff --git a/go-linter-driven-development/scripts/ldd-lint_test.sh b/go-linter-driven-development/scripts/ldd-lint_test.sh new file mode 100644 index 00000000..10713ad4 --- /dev/null +++ b/go-linter-driven-development/scripts/ldd-lint_test.sh @@ -0,0 +1,335 @@ +#!/usr/bin/env bash +# Fixture matrix for ldd-lint.sh — the gap linter's own conformance test. +# +# Each case builds a throwaway tree under a temp dir, runs the script against +# it and asserts the exit code and the lines it prints. The cases are the same +# for every language; the language-specific parts — the source suffix, how a +# comment and a test file are written, the file a directory count leaves out, +# and the repository linter config that holds a file-length limit — come from +# the fixture rows spliced in below. A script whose language block serves +# several languages runs the whole matrix once per row. +# +# Usage: bash scripts/ldd-lint_test.sh [path/to/ldd-lint.sh] +# (default: the sibling ldd-lint.sh) +# Exit: 0 all cases pass · 1 any failure (details on stdout) +# +# Uses only POSIX-portable tools, like the script under test. + +set -u + +HERE=$(cd "$(dirname "$0")" && pwd) +LINT="${1:-$HERE/ldd-lint.sh}" +[[ -f "$LINT" ]] || { echo "no such script: $LINT" >&2; exit 2; } +LINT=$(cd "$(dirname "$LINT")" && pwd)/$(basename "$LINT") +MAX_LINES=450 +WARN_FILES=8 +MAX_FILES=13 + +pass=0 failed=0 skipped=0 +CASE="" +OUT="" ERR="" CODE=0 +ROW="" +REPO="" +TAB=$(printf '\t') + +# ---------- harness ---------- +begin() { CASE="$1"; [[ -n "$ROW" ]] && CASE="[$ROW] $1"; REPO=$(mktemp -d); } +finish() { rm -rf "$REPO"; } + +run_lint() { # [options...] — runs the script over $REPO + local errf; errf=$(mktemp) + OUT=$(cd "$REPO" && bash "$LINT" "$@" --root . 2>"$errf"); CODE=$? + ERR=$(cat "$errf"); rm -f "$errf" +} + +ok() { pass=$((pass + 1)); echo "PASS $CASE"; } +bad() { failed=$((failed + 1)); echo "FAIL $CASE — $1"; echo " exit=$CODE"; printf '%s\n' "$OUT" "$ERR" | sed 's/^/ | /'; } +skip() { skipped=$((skipped + 1)); echo "SKIP $CASE — $1"; } + +expect_exit() { (( CODE == $1 )) || { bad "expected exit $1"; return 1; }; } +expect_has() { # — in stdout or stderr + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" || { bad "missing: $1"; return 1; } +} +expect_not() { + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" && { bad "unexpected: $1"; return 1; } + return 0 +} +expect_no_finding() { # no line names a file or directory of the tree + printf '%s\n' "$OUT" | grep -qE '^[^ ]+: [0-9]+ ' && { bad "unexpected finding"; return 1; } + return 0 +} + +# ---------- fixture builders ---------- +# write_src [ [ []]] — a source +# file with that many code lines, comment-only lines, blank lines and lines of +# one block comment, in that order. +write_src() { + local path="$REPO/$1" code="$2" com="${3:-0}" blank="${4:-0}" block="${5:-0}" i + mkdir -p "$(dirname "$path")" + { + for (( i = 1; i <= code; i++ )); do printf 'v%d = %d\n' "$i" "$i"; done + for (( i = 1; i <= com; i++ )); do printf '%s\n' "$FX_LINE_COMMENT"; done + for (( i = 1; i <= blank; i++ )); do printf '\n'; done + if (( block > 0 )); then + printf '%s\n' "$FX_BLOCK_OPEN" + for (( i = 3; i <= block; i++ )); do printf ' inside the block\n'; done + printf '%s\n' "$FX_BLOCK_CLOSE" + fi + } > "$path" +} + +# fill_dir [] — that many one-line source and test files +fill_dir() { + local i + for (( i = 1; i <= $2; i++ )); do write_src "$1/m$i$FX_EXT" 1; done + for (( i = 1; i <= ${3:-0}; i++ )); do write_src "$(fx_test_file "$1" "$i")" 1; done +} + +# ---------- fixture rows: the Go row this script's language block serves ---------- +# The matrix runs its cases once per row. use_row sets the row's variables and +# the fx_* builders write that language's files; the cases read only these. +# +# FX_ROWS the row ids, space-separated +# FX_NATIVE of the mixed repository's languages (d, go, python, +# typescript), the ones this plugin's block reviews itself +# FX_GLOB find(1) -name pattern for the row's source files +# FX_SRC_FILE the production file under a layer directory: two sleep +# calls and one suppression directive +# FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it +# fx_write_marker write the project marker for a (sub-)project +# fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable +FX_ROWS="go" +FX_NATIVE="go" + +use_row() { + FX_EXT='.go' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='api.pb.go' + FX_GLOB='*.go' + FX_SRC_FILE="services/worker.go" + FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' +} + +fx_write_marker() { # + printf 'module example.com/%s\n\ngo 1.22\n' "$2" > "$1/go.mod" +} + +fx_write_code() { + mkdir -p "$REPO/services" + cat > "$REPO/services/worker.go" <<'GO' +package services + +import "time" + +var cache = map[string]string{} //nolint:gochecknoglobals // TODO + +// Run polls until the deadline. +func Run(deadline time.Time) { + for time.Now().Before(deadline) { + time.Sleep(time.Second) + } + time.Sleep(50 * time.Millisecond) +} +GO + cat > "$REPO/services/worker_test.go" <<'GO' +package services_test + +import ( + "testing" + "time" +) + +func TestRun(t *testing.T) { + time.Sleep(10 * time.Millisecond) +} +GO +} + +fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } + +fx_write_linter_config() { # + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" +} + +fx_write_linter_config_unreadable() { + printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" +} + +# ========================= cases ========================= +run_row_cases() { # runs every case against the current row + +begin "a file at the limit passes and one line over is a finding" +write_src "src/at$FX_EXT" "$MAX_LINES"; write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" \ + && expect_not "src/at$FX_EXT:" && ok +finish + +begin "comment-only and blank lines are not counted by the plugin's default" +write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 20 20 +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the lines of a block comment are comment-only" +if [[ -z "$FX_BLOCK_OPEN" ]]; then skip "the language has no block comment"; finish +else + write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 0 0 30 + run_lint file-length + expect_exit 0 && expect_no_finding && ok + finish +fi + +begin "a test file over the limit is not measured" +write_src "$(fx_test_file src 1)" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "a generated file over the limit is not measured" +write_src "src/gen$FX_EXT" $((MAX_LINES + 50)) +{ printf '%s DO NOT EDIT\n' "$FX_LINE_COMMENT"; cat "$REPO/src/gen$FX_EXT"; } > "$REPO/src/gen.tmp" && mv "$REPO/src/gen.tmp" "$REPO/src/gen$FX_EXT" +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the plugin's default names the setting that makes the linter hold it" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +finish + +begin "the repository's linter config holds the limit and is counted its way" +fx_write_linter_config 100 +write_src "src/a$FX_EXT" 90 20 +run_lint file-length +expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +finish + +begin "a linter config whose limit cannot be read is named, and the next source applies" +if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish +else + write_src "src/over$FX_EXT" $((MAX_LINES + 1)) + run_lint file-length + expect_exit 0 && expect_has "its limit cannot be read here" \ + && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok + finish +fi + +begin ".ldd-lint.yaml sets the limit when the linter holds none" +printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 101 +run_lint file-length +expect_exit 0 && expect_has "set by .ldd-lint.yaml" && expect_has "src/a$FX_EXT: 101 code lines (limit 100, .ldd-lint.yaml)" && ok +finish + +begin "a .ldd-lint.yaml value that is not a positive integer is inconclusive (exit 2)" +printf 'file_max_lines: lots\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 2 && expect_has "file_max_lines must be a positive integer" && ok +finish + +begin "a directory below the warning size is not a finding" +fill_dir pkg $((WARN_FILES - 1)) 3 +write_src "pkg/$FX_SKIP_FILE" 1 +run_lint package-size +expect_exit 0 && expect_no_finding && ok +finish + +begin "a directory at the warning size is a warning" +fill_dir pkg "$WARN_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $WARN_FILES source files (warning at $WARN_FILES, profile default)" && ok +finish + +begin "a directory at the limit is over the limit" +fill_dir pkg "$MAX_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $MAX_FILES source files (limit $MAX_FILES, profile default)" && ok +finish + +begin ".ldd-lint.yaml sets the directory sizes" +printf 'package_warn_files: 2\npackage_max_files: 3\n' > "$REPO/.ldd-lint.yaml" +fill_dir small 2; fill_dir big 3 +run_lint package-size +expect_exit 0 && expect_has "small: 2 source files (warning at 2, .ldd-lint.yaml)" \ + && expect_has "big: 3 source files (limit 3, .ldd-lint.yaml)" && ok +finish + +begin "--tsv writes path, line 0 and message as rows, and the threshold to stderr" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length --tsv +expect_exit 0 \ + && { [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}$((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" ]] || bad "stdout is not the one row"; } \ + && [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}"* ]] \ + && { [[ "$ERR" == "threshold: "* ]] || bad "threshold line not on stderr"; } && [[ "$ERR" == "threshold: "* ]] && ok +finish + +begin "--files measures only the files it names" +write_src "src/named$FX_EXT" $((MAX_LINES + 1)); write_src "src/other$FX_EXT" $((MAX_LINES + 1)) +printf 'src/named%s (not bundled: %s lines)\n' "$FX_EXT" $((MAX_LINES + 1)) > "$REPO/files.txt" +run_lint file-length --files files.txt +expect_exit 0 && expect_has "src/named$FX_EXT:" && expect_not "src/other$FX_EXT:" && ok +finish + +begin "in a git repository an ignored file is not measured" +write_src "src/kept$FX_EXT" $((MAX_LINES + 1)); write_src "build/out$FX_EXT" $((MAX_LINES + 1)) +printf 'build/\n' > "$REPO/.gitignore" +(cd "$REPO" && git init -q) +run_lint file-length +expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok +finish + +begin "two runs print the same output" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" +run_lint package-size; first="$OUT" +run_lint package-size +expect_exit 0 && { [[ "$OUT" == "$first" ]] || bad "the second run printed something else"; } && [[ "$OUT" == "$first" ]] && ok +finish + +begin "a tree without a source file is measured clean" +printf 'notes\n' > "$REPO/README.md" +run_lint file-length +expect_exit 0 && expect_has "no source file in scope" && ok +finish + +} # run_row_cases + +for ROW in $FX_ROWS; do + use_row "$ROW" + run_row_cases +done +ROW="" + +# --- row-independent --- +begin "a missing subcommand is a usage error (exit 2)" +run_lint +expect_exit 2 && expect_has "name a subcommand" && ok +finish + +begin "a --files list that does not exist is a usage error (exit 2)" +run_lint file-length --files no-such-list.txt +expect_exit 2 && expect_has "no such file list" && ok +finish + +# ========================= summary ========================= +echo +echo "ldd-lint_test: $pass passed, $failed failed, $skipped skipped" +(( failed == 0 )) diff --git a/go-linter-driven-development/scripts/ldd-scope_test.sh b/go-linter-driven-development/scripts/ldd-scope_test.sh index 2e3aa895..05fd0474 100755 --- a/go-linter-driven-development/scripts/ldd-scope_test.sh +++ b/go-linter-driven-development/scripts/ldd-scope_test.sh @@ -110,10 +110,29 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="go" FX_NATIVE="go" use_row() { + FX_EXT='.go' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='api.pb.go' FX_GLOB='*.go' FX_SRC_FILE="services/worker.go" FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' @@ -154,6 +173,16 @@ func TestRun(t *testing.T) { GO } +fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } + +fx_write_linter_config() { # + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" +} + +fx_write_linter_config_unreadable() { + printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" +} + # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/go-linter-driven-development/skills/code-designing/SKILL.md b/go-linter-driven-development/skills/code-designing/SKILL.md index 48ed3c98..8141e553 100644 --- a/go-linter-driven-development/skills/code-designing/SKILL.md +++ b/go-linter-driven-development/skills/code-designing/SKILL.md @@ -31,7 +31,7 @@ Backward counterpart (fixing code that already fails lint/review): @refactoring. - Linter failures that need a design decision, not a mechanical fix: - `argument-limit` (>4 params) → design an options struct (grouping data that travels together — score it per `../../rules/R1-primitive-obsession.md`) - `function-result-limit` (>3 returns) / `confusing-results` → design a named result type (same R1 scoring) - - `file-length-limit` (>450 lines) → split juicy types into their own files (juiciness per R1; file-per-type per `../../rules/R5-vertical-slice.md`); a single god type routes to @refactoring's god-object decomposition procedure first + - `file-length-limit` (the repository's limit, else the gap linter's 450 code lines) → split juicy types into their own files (juiciness per R1; file-per-type per `../../rules/R5-vertical-slice.md`); a single god type routes to @refactoring's god-object decomposition procedure first - Package-size yellow/red zone → re-model with sub-packages *before* the zone escalates (`` in @refactoring's `reference.md` (`sed -n '/^/,/^<\/package_decomposition>/p; /^### Package decomposition/,$p'`)) - A Phase 4 review CLUSTER (≥2 hunters converging on one anchor — @linter-driven-development routes it here) → **cluster-scoped mode**: skip diff --git a/go-linter-driven-development/skills/refactoring/SKILL.md b/go-linter-driven-development/skills/refactoring/SKILL.md index 6b99878b..8db263ea 100644 --- a/go-linter-driven-development/skills/refactoring/SKILL.md +++ b/go-linter-driven-development/skills/refactoring/SKILL.md @@ -45,7 +45,7 @@ table in `../../agents/lint-fixer.md` — keep them consistent.) | `maintidx` | `../../rules/R3-storifying.md` + `../../rules/R1-primitive-obsession.md` | | `dupl` | `../../rules/R1-primitive-obsession.md` (extract shared type/logic); duplicated blocks that switch on the same kind/type discriminator → `../../rules/R11-conditional-dispatch.md` | | `exhaustive` (missing enum cases) | `../../rules/R11-conditional-dispatch.md` — handle the case at the single dispatch site; a second switch appearing is the R11 violation itself | -| revive `file-length-limit`; package-size hook failures (`hooks/check-package-sizes.sh`) | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | +| revive `file-length-limit`; `scripts/ldd-lint.sh file-length` and `package-size` findings | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | | `gochecknoglobals` / `gochecknoinits` | `../../rules/R8-no-globals.md` | | `ireturn` / interface lint on single-impl interfaces | `../../rules/R6-test-only-interfaces.md` | | `go test -race` failures; `govet` `copylocks` | `../../rules/R10-concurrency-safety.md` | diff --git a/go-linter-driven-development/skills/refactoring/reference.md b/go-linter-driven-development/skills/refactoring/reference.md index 8d52cc7e..963a8f8d 100644 --- a/go-linter-driven-development/skills/refactoring/reference.md +++ b/go-linter-driven-development/skills/refactoring/reference.md @@ -66,7 +66,7 @@ decomposition): "Multi-rule procedures", below. ## File and package routing -**`file-length-limit` (>450 lines):** +**revive `file-length-limit`, or a file over 450 code lines where the repository sets no limit** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: | File pattern | Action | |---|---| @@ -75,15 +75,20 @@ decomposition): "Multi-rule procedures", below. | Long functions, few types | Storify → extract functions (R3) | -**Package-size zones** — count non-test `.go` files per directory: +**Package-size zones** — non-test `.go` files per directory, no linter counts them: ``` -find -maxdepth 1 -type f -name '*.go' -not -name '*_test.go' -not -name '*_gen.go' -not -name '*.pb.go' | wc -l +bash /../../scripts/ldd-lint.sh package-size --root . ``` -≤7 green — fine. 8–12 yellow — design review *before the next file lands*. ≥13 red — -**must decompose**. Either zone: run the 3-step design review under "Package -decomposition", printed with this range (it is a *design* review — missing domain types are the +The gap linter counts the non-test, non-generated source files of each directory; +a `.ldd-lint.yaml` at the repository root sets other sizes, and the first line of +its output says which sizes applied. + +Below 8 files green — fine. 8 or more yellow — design review +*before the next file lands*. 13 or more red — **must decompose**. Either +zone: run the 3-step design review under "Package decomposition", printed with this +range (it is a *design* review — missing domain types are the disease, file count the symptom). Invoke @code-designing to validate extracted types. @@ -265,9 +270,9 @@ composition: @code-designing. ### Package decomposition -Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone (≥13 non-test `.go` -files at one directory level) or yellow zone (8–12) — detection command and zone -table in `` above. +Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone (13 or more non-test +`.go` files at one directory level) or yellow zone (8 or more) — the +measuring command and the zones are in `` above. **A package-size violation is a design review, not a mechanical file split.** File count is the symptom; the disease is usually missing domain types or multiple diff --git a/lang/generic/passthrough/CHANGELOG.md b/lang/generic/passthrough/CHANGELOG.md index b63f092a..4cf64e87 100644 --- a/lang/generic/passthrough/CHANGELOG.md +++ b/lang/generic/passthrough/CHANGELOG.md @@ -5,6 +5,26 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio ## [Unreleased] +### Added + +- **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` + measures lines per source file (`file-length`) and source files per directory + (`package-size`). Where the repository's linter config holds the file-length + limit, the script uses that limit and counts lines the way that linter counts. + Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and + otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 + source files per directory and the limit at 13. The first line of the output + names the limit that applied; with the defaults it also prints the setting that + makes the repository's linter hold the limit. `--tsv` writes findings as rows. + The script never edits a file. + +### Changed + +- **One set of thresholds.** The skills and routing tables name the same limits the + gap linter applies: the repository linter's limit where it sets one, else + 450 code lines per file and 8 and 13 files per directory. The package-size + procedure runs the gap linter in place of a hand-written `find` command. + ### Fixed - **A claim that something is missing needs a listing.** A rule hunter that reports diff --git a/lang/generic/passthrough/README.md b/lang/generic/passthrough/README.md index 8ab046d6..bf11dbfd 100644 --- a/lang/generic/passthrough/README.md +++ b/lang/generic/passthrough/README.md @@ -107,8 +107,8 @@ linter-driven-development/ code is Go, for demonstration only; each opens with that note. The move and the reasoning hold in any language — read the Go as the shape and spell it in yours. -Not shipped here, because they are Go-specific rather than Go-illustrated: the -test-harness catalogue and the package-size hook. +Not shipped here, because it is Go-specific rather than Go-illustrated: the +test-harness catalogue. ## Mixed repositories diff --git a/lang/generic/profile.yaml b/lang/generic/profile.yaml index c7e2a13c..18e79516 100644 --- a/lang/generic/profile.yaml +++ b/lang/generic/profile.yaml @@ -26,6 +26,13 @@ doc_form: doc comment doc_comment: doc comment src_ext: "." unexported: internal +# The gap linter's thresholds, where the repository's linter holds none: code +# lines per source file (blank and comment-only lines are not counted), and +# source files per directory that start the warning zone and the limit. A +# repository's linter config, then its .ldd-lint.yaml, wins over these. +file_max_lines: "450" +package_warn_files: "8" +package_max_files: "13" # The standalone coding-rules document rendered from core/handbook/: the twelve # rules with the core pseudocode examples and this binding's house rules and # mechanics under handbook/, written outside the plugin directory. diff --git a/lang/generic/scripts/ldd-fixture.sh b/lang/generic/scripts/ldd-fixture.sh index 02639b6d..a076a206 100644 --- a/lang/generic/scripts/ldd-fixture.sh +++ b/lang/generic/scripts/ldd-fixture.sh @@ -11,17 +11,41 @@ # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="go python" FX_NATIVE="go python d typescript" use_row() { case "$1" in go) + FX_EXT='.go' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='api.pb.go' FX_GLOB='*.go' FX_SRC_FILE="services/worker.go" FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' ;; python) + FX_EXT='.py' + FX_LINE_COMMENT='# note' + FX_BLOCK_OPEN='' + FX_BLOCK_CLOSE='' + FX_SKIP_FILE='__init__.py' FX_GLOB='*.py' FX_SRC_FILE="services/worker.py" FX_SUPPRESS_LINE='CACHE: dict[str, str] = {} # noqa: PLW0603' @@ -94,3 +118,24 @@ PY ;; esac } + +fx_test_file() { + case "$ROW" in + go) printf '%s/m%s_test.go' "$1" "$2" ;; + python) printf '%s/test_m%s.py' "$1" "$2" ;; + esac +} + +fx_write_linter_config() { # + case "$ROW" in + go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" ;; + python) printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" ;; + esac +} + +fx_write_linter_config_unreadable() { + case "$ROW" in + go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; + python) printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" ;; + esac +} diff --git a/lang/go/agents/lint-fixer/routing-table.md b/lang/go/agents/lint-fixer/routing-table.md index de3c01dd..a48834c5 100644 --- a/lang/go/agents/lint-fixer/routing-table.md +++ b/lang/go/agents/lint-fixer/routing-table.md @@ -7,7 +7,7 @@ | `maintidx` | rules/R3-storifying.md + rules/R1-primitive-obsession.md | | `dupl` | rules/R1-primitive-obsession.md (extract shared type/logic); duplicated switches on one discriminator → rules/R11-conditional-dispatch.md | | `exhaustive` (missing enum cases) | rules/R11-conditional-dispatch.md (via @refactoring) | -| revive `file-length-limit`; package-size hook failures (`hooks/check-package-sizes.sh`) | rules/R5-vertical-slice.md | +| revive `file-length-limit`; `scripts/ldd-lint.sh file-length` and `package-size` findings | rules/R5-vertical-slice.md | | `gochecknoglobals` / `gochecknoinits` | rules/R8-no-globals.md | | `ireturn` / interface lint on single-impl interfaces | rules/R6-test-only-interfaces.md | | `go test -race` failures; `govet` `copylocks` | rules/R10-concurrency-safety.md (via @refactoring) | diff --git a/lang/go/passthrough/CHANGELOG.md b/lang/go/passthrough/CHANGELOG.md index 01649b62..bcfe2328 100644 --- a/lang/go/passthrough/CHANGELOG.md +++ b/lang/go/passthrough/CHANGELOG.md @@ -5,6 +5,32 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio ## [Unreleased] +### Added + +- **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` + measures lines per source file (`file-length`) and source files per directory + (`package-size`). Where the repository's linter config holds the file-length + limit, the script uses that limit and counts lines the way that linter counts. + Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and + otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 + source files per directory and the limit at 13. The first line of the output + names the limit that applied; with the defaults it also prints the setting that + makes the repository's linter hold the limit. `--tsv` writes findings as rows. + The script never edits a file. + +### Changed + +- **One set of thresholds.** The skills and routing tables name the same limits the + gap linter applies: the repository linter's limit where it sets one, else + 450 code lines per file and 8 and 13 files per directory. The package-size + procedure runs the gap linter in place of a hand-written `find` command. + +### Removed + +- **`hooks/check-package-sizes.sh`.** The hook was never registered, so it never + ran. The gap linter's `package-size` measures the same zones for every source + directory. + ### Fixed - **A claim that something is missing needs a listing.** A rule hunter that reports diff --git a/lang/go/passthrough/README.md b/lang/go/passthrough/README.md index 2c247160..5c15ab5a 100644 --- a/lang/go/passthrough/README.md +++ b/lang/go/passthrough/README.md @@ -26,7 +26,7 @@ go-linter-driven-development/ ├── commands/ go-ldd-analyze · autopilot · quickfix · prepare · review · status · wire-repo-brain ├── scripts/ check-repo-brain.sh — repo-brain conformance gate, installed into target repos by /wire-repo-brain │ ldd-scope.sh · ldd-detect.sh — the review's scope bundle and detection pass -└── hooks/ package-size gate +│ ldd-lint.sh — the gap linter: file length and package size where no linter holds them ``` **Five layers, one fact per fact:** diff --git a/lang/go/passthrough/hooks/check-package-sizes.sh b/lang/go/passthrough/hooks/check-package-sizes.sh deleted file mode 100755 index 08a91914..00000000 --- a/lang/go/passthrough/hooks/check-package-sizes.sh +++ /dev/null @@ -1,79 +0,0 @@ -#!/usr/bin/env bash -# Portable package-size gate for the go-linter-driven-development plugin. -# -# Fires as a PostToolUse hook after Write / Edit / MultiEdit. Scans Go source -# roots under $CLAUDE_PROJECT_DIR and counts non-test, non-generated .go files -# per directory at one level deep. -# -# Thresholds (also documented in the refactoring and pre-commit-review skills): -# >=13 files = RED -> exit 2, stderr is fed back to Claude as a blocking -# error so the violation is acknowledged before more -# code lands in the oversized package. -# 8-12 files = YELLOW -> exit 0 with stdout advisory, no block. -# <=7 files = GREEN -> silent, exit 0. -# -# Guards: -# - no-op unless $CLAUDE_PROJECT_DIR/go.mod exists (not a Go project) -# - no-op if none of internal/, cmd/, pkg/ exist -# -# Uses only POSIX-portable tools: find, wc, tr, sort. No jq / python / task. - -set -u - -PROJECT_DIR="${CLAUDE_PROJECT_DIR:-$(pwd)}" - -[[ -f "$PROJECT_DIR/go.mod" ]] || exit 0 - -YELLOW_MIN=8 -RED_MIN=13 - -roots=() -for d in internal cmd pkg; do - [[ -d "$PROJECT_DIR/$d" ]] && roots+=("$PROJECT_DIR/$d") -done - -(( ${#roots[@]} == 0 )) && exit 0 - -red_lines=() -yellow_lines=() - -while IFS= read -r dir; do - [[ -z "$dir" ]] && continue - count=$(find "$dir" -maxdepth 1 -type f -name '*.go' \ - -not -name '*_test.go' \ - -not -name '*_gen.go' \ - -not -name '*.pb.go' \ - 2>/dev/null | wc -l | tr -d ' ') - - rel=${dir#$PROJECT_DIR/} - if (( count >= RED_MIN )); then - red_lines+=(" $rel: $count non-test .go files (RED zone, must decompose)") - elif (( count >= YELLOW_MIN )); then - yellow_lines+=(" $rel: $count non-test .go files (YELLOW zone, design review before next file)") - fi -done < <(find "${roots[@]}" -type d \ - -not -path '*/vendor/*' \ - -not -path '*/testdata/*' \ - 2>/dev/null | sort -u) - -if (( ${#red_lines[@]} > 0 )); then - { - echo "⛔ Package size gate — RED zone detected:" - printf '%s\n' "${red_lines[@]}" - echo "" - echo "These packages MUST be decomposed before more code lands." - echo "Apply the 3-step design review (see in the refactoring skill's reference.md):" - echo " 1. Does the package name reflect a real-world domain concept (not a role/container)?" - echo " 2. Are types well-scoped, or are there big structs hiding sub-types or primitive-obsession fields?" - echo " 3. Only after the type review, decide: sub-packages, new leaf types, or both." - } >&2 - exit 2 -fi - -if (( ${#yellow_lines[@]} > 0 )); then - echo "⚠️ Package size gate — YELLOW zone:" - printf '%s\n' "${yellow_lines[@]}" - echo "Design review recommended before the next file lands in these packages." -fi - -exit 0 diff --git a/lang/go/passthrough/hooks/hooks.json b/lang/go/passthrough/hooks/hooks.json deleted file mode 100644 index deffac97..00000000 --- a/lang/go/passthrough/hooks/hooks.json +++ /dev/null @@ -1,3 +0,0 @@ -{ - "hooks": {} -} diff --git a/lang/go/profile.yaml b/lang/go/profile.yaml index 49c62c5f..e3b91d7b 100644 --- a/lang/go/profile.yaml +++ b/lang/go/profile.yaml @@ -21,6 +21,13 @@ doc_form: godoc doc_comment: godoc comment src_ext: .go unexported: unexported +# The gap linter's thresholds, where the repository's linter holds none: code +# lines per source file (blank and comment-only lines are not counted), and +# source files per directory that start the warning zone and the limit. A +# repository's linter config, then its .ldd-lint.yaml, wins over these. +file_max_lines: "450" +package_warn_files: "8" +package_max_files: "13" # The standalone coding-rules document rendered from core/handbook/ with this # binding's files under handbook/, written outside the plugin directory. handbook: coding-rules/go.md diff --git a/lang/go/scripts/ldd-fixture.sh b/lang/go/scripts/ldd-fixture.sh index 119fbb97..71fb7305 100644 --- a/lang/go/scripts/ldd-fixture.sh +++ b/lang/go/scripts/ldd-fixture.sh @@ -11,10 +11,29 @@ # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="go" FX_NATIVE="go" use_row() { + FX_EXT='.go' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='api.pb.go' FX_GLOB='*.go' FX_SRC_FILE="services/worker.go" FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' @@ -54,3 +73,13 @@ func TestRun(t *testing.T) { } GO } + +fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } + +fx_write_linter_config() { # + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" +} + +fx_write_linter_config_unreadable() { + printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" +} diff --git a/lang/go/skills/code-designing/linter-triggers.md b/lang/go/skills/code-designing/linter-triggers.md index fdec1802..cf936862 100644 --- a/lang/go/skills/code-designing/linter-triggers.md +++ b/lang/go/skills/code-designing/linter-triggers.md @@ -1,5 +1,5 @@ - Linter failures that need a design decision, not a mechanical fix: - `argument-limit` (>4 params) → design an options struct (grouping data that travels together — score it per `../../rules/R1-primitive-obsession.md`) - `function-result-limit` (>3 returns) / `confusing-results` → design a named result type (same R1 scoring) - - `file-length-limit` (>450 lines) → split juicy types into their own files (juiciness per R1; file-per-type per `../../rules/R5-vertical-slice.md`); a single god type routes to @refactoring's god-object decomposition procedure first + - `file-length-limit` (the repository's limit, else the gap linter's {{.FileMaxLines}} code lines) → split juicy types into their own files (juiciness per R1; file-per-type per `../../rules/R5-vertical-slice.md`); a single god type routes to @refactoring's god-object decomposition procedure first - Package-size yellow/red zone → re-model with sub-packages *before* the zone escalates (`` in @refactoring's `reference.md` (`sed -n '/^/,/^<\/package_decomposition>/p; /^### Package decomposition/,$p'`)) diff --git a/lang/go/skills/refactoring/file-and-package-routing.md b/lang/go/skills/refactoring/file-and-package-routing.md index 3268591c..ac2a2fc9 100644 --- a/lang/go/skills/refactoring/file-and-package-routing.md +++ b/lang/go/skills/refactoring/file-and-package-routing.md @@ -1,4 +1,4 @@ -**`file-length-limit` (>450 lines):** +**revive `file-length-limit`, or a file over {{.FileMaxLines}} code lines where the repository sets no limit** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: | File pattern | Action | |---|---| @@ -7,14 +7,19 @@ | Long functions, few types | Storify → extract functions (R3) | -**Package-size zones** — count non-test `.go` files per directory: +**Package-size zones** — non-test `.go` files per directory, no linter counts them: ``` -find -maxdepth 1 -type f -name '*.go' -not -name '*_test.go' -not -name '*_gen.go' -not -name '*.pb.go' | wc -l +bash /../../scripts/ldd-lint.sh package-size --root . ``` -≤7 green — fine. 8–12 yellow — design review *before the next file lands*. ≥13 red — -**must decompose**. Either zone: run the 3-step design review under "Package -decomposition", printed with this range (it is a *design* review — missing domain types are the +The gap linter counts the non-test, non-generated source files of each directory; +a `.ldd-lint.yaml` at the repository root sets other sizes, and the first line of +its output says which sizes applied. + +Below {{.PackageWarnFiles}} files green — fine. {{.PackageWarnFiles}} or more yellow — design review +*before the next file lands*. {{.PackageMaxFiles}} or more red — **must decompose**. Either +zone: run the 3-step design review under "Package decomposition", printed with this +range (it is a *design* review — missing domain types are the disease, file count the symptom). Invoke @code-designing to validate extracted types. diff --git a/lang/go/skills/refactoring/routing-table.md b/lang/go/skills/refactoring/routing-table.md index c97e84c3..f693f558 100644 --- a/lang/go/skills/refactoring/routing-table.md +++ b/lang/go/skills/refactoring/routing-table.md @@ -10,7 +10,7 @@ table in `../../agents/lint-fixer.md` — keep them consistent.) | `maintidx` | `../../rules/R3-storifying.md` + `../../rules/R1-primitive-obsession.md` | | `dupl` | `../../rules/R1-primitive-obsession.md` (extract shared type/logic); duplicated blocks that switch on the same kind/type discriminator → `../../rules/R11-conditional-dispatch.md` | | `exhaustive` (missing enum cases) | `../../rules/R11-conditional-dispatch.md` — handle the case at the single dispatch site; a second switch appearing is the R11 violation itself | -| revive `file-length-limit`; package-size hook failures (`hooks/check-package-sizes.sh`) | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | +| revive `file-length-limit`; `scripts/ldd-lint.sh file-length` and `package-size` findings | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | | `gochecknoglobals` / `gochecknoinits` | `../../rules/R8-no-globals.md` | | `ireturn` / interface lint on single-impl interfaces | `../../rules/R6-test-only-interfaces.md` | | `go test -race` failures; `govet` `copylocks` | `../../rules/R10-concurrency-safety.md` | diff --git a/lang/python/passthrough/CHANGELOG.md b/lang/python/passthrough/CHANGELOG.md index ad709846..884c2d4a 100644 --- a/lang/python/passthrough/CHANGELOG.md +++ b/lang/python/passthrough/CHANGELOG.md @@ -5,6 +5,26 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio ## [Unreleased] +### Added + +- **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` + measures lines per source file (`file-length`) and source files per directory + (`package-size`). Where the repository's linter config holds the file-length + limit, the script uses that limit and counts lines the way that linter counts. + Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and + otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 + source files per directory and the limit at 13. The first line of the output + names the limit that applied; with the defaults it also prints the setting that + makes the repository's linter hold the limit. `--tsv` writes findings as rows. + The script never edits a file. + +### Changed + +- **One set of thresholds.** The skills and routing tables name the same limits the + gap linter applies: the repository linter's limit where it sets one, else + 450 code lines per file and 8 and 13 files per directory. The package-size + procedure runs the gap linter in place of a hand-written `find` command. + ### Fixed - **A claim that something is missing needs a listing.** A rule hunter that reports diff --git a/lang/python/profile.yaml b/lang/python/profile.yaml index 60924f0e..e8f7802c 100644 --- a/lang/python/profile.yaml +++ b/lang/python/profile.yaml @@ -27,6 +27,13 @@ doc_form: docstring doc_comment: docstring src_ext: .py unexported: underscore-prefixed +# The gap linter's thresholds, where the repository's linter holds none: code +# lines per source file (blank and comment-only lines are not counted), and +# source files per directory that start the warning zone and the limit. A +# repository's linter config, then its .ldd-lint.yaml, wins over these. +file_max_lines: "450" +package_warn_files: "8" +package_max_files: "13" # The standalone coding-rules document rendered from core/handbook/ with this # binding's files under handbook/, written outside the plugin directory. handbook: coding-rules/python.md diff --git a/lang/python/scripts/ldd-fixture.sh b/lang/python/scripts/ldd-fixture.sh index a6b81dd1..15e28284 100644 --- a/lang/python/scripts/ldd-fixture.sh +++ b/lang/python/scripts/ldd-fixture.sh @@ -11,10 +11,29 @@ # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="python" FX_NATIVE="python" use_row() { + FX_EXT='.py' + FX_LINE_COMMENT='# note' + FX_BLOCK_OPEN='' + FX_BLOCK_CLOSE='' + FX_SKIP_FILE='__init__.py' FX_GLOB='*.py' FX_SRC_FILE="services/worker.py" FX_SUPPRESS_LINE='CACHE: dict[str, str] = {} # noqa: PLW0603' @@ -48,3 +67,13 @@ def test_run() -> None: time.sleep(0.01) PY } + +fx_test_file() { printf '%s/test_m%s.py' "$1" "$2"; } + +fx_write_linter_config() { # + printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" +} + +fx_write_linter_config_unreadable() { + printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" +} diff --git a/lang/python/skills/code-designing/linter-triggers.md b/lang/python/skills/code-designing/linter-triggers.md index 796897be..b5891e98 100644 --- a/lang/python/skills/code-designing/linter-triggers.md +++ b/lang/python/skills/code-designing/linter-triggers.md @@ -1,5 +1,5 @@ - Linter failures that need a design decision, not a mechanical fix: - `PLR0913` (too many arguments) → design a frozen dataclass of the parameters that travel together (score it per `../../rules/R1-primitive-obsession.md`); a keyword-only `*` marker tidies the call sites but leaves the clump - `PLR0911` (too many returns), or a tuple return carrying a `bool` or an error member → design a named result type, or raise (same R1 scoring) - - a module over ~450 lines (ruff has no file-length rule — count) → split juicy types into their own modules (juiciness per R1; module-per-type per `../../rules/R5-vertical-slice.md`); a single god class routes to @refactoring's god-object decomposition procedure first + - a module over the file-length limit (pylint's `max-module-lines` where set, else the gap linter's {{.FileMaxLines}} code lines; ruff has no file-length rule) → split juicy types into their own modules (juiciness per R1; module-per-type per `../../rules/R5-vertical-slice.md`); a single god class routes to @refactoring's god-object decomposition procedure first - Package-size yellow/red zone → re-model with sub-packages *before* the zone escalates (`` in @refactoring's `reference.md` (`sed -n '/^/,/^<\/package_decomposition>/p; /^### Package decomposition/,$p'`)) diff --git a/lang/python/skills/refactoring/file-and-package-routing.md b/lang/python/skills/refactoring/file-and-package-routing.md index f5d0e70c..10b430a7 100644 --- a/lang/python/skills/refactoring/file-and-package-routing.md +++ b/lang/python/skills/refactoring/file-and-package-routing.md @@ -1,4 +1,4 @@ -**A module over ~450 lines (ruff has no file-length rule — count):** +**A module over the file-length limit** — pylint's `max-module-lines` where the repository sets it (ruff has no file-length rule), else {{.FileMaxLines}} code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: | File pattern | Action | |---|---| @@ -7,14 +7,19 @@ | Long functions, few types | Storify → extract functions (R3) | -**Package-size zones** — count non-test `.py` modules per package directory: +**Package-size zones** — non-test `.py` modules per package directory, `__init__.py`, `conftest.py` and generated stubs left out: ``` -find -maxdepth 1 -type f -name '*.py' -not -name 'test_*.py' -not -name '*_test.py' -not -name 'conftest.py' -not -name '__init__.py' -not -name '*_pb2.py' -not -name '*_pb2_grpc.py' | wc -l +bash /../../scripts/ldd-lint.sh package-size --root . ``` -≤7 green — fine. 8–12 yellow — design review *before the next file lands*. ≥13 red — -**must decompose**. Either zone: run the 3-step design review under "Package -decomposition", printed with this range (it is a *design* review — missing domain types are the +The gap linter counts the non-test, non-generated source files of each directory; +a `.ldd-lint.yaml` at the repository root sets other sizes, and the first line of +its output says which sizes applied. + +Below {{.PackageWarnFiles}} files green — fine. {{.PackageWarnFiles}} or more yellow — design review +*before the next file lands*. {{.PackageMaxFiles}} or more red — **must decompose**. Either +zone: run the 3-step design review under "Package decomposition", printed with this +range (it is a *design* review — missing domain types are the disease, file count the symptom). Invoke @code-designing to validate extracted types. diff --git a/lang/python/skills/refactoring/routing-table.md b/lang/python/skills/refactoring/routing-table.md index bbda52bb..cb676240 100644 --- a/lang/python/skills/refactoring/routing-table.md +++ b/lang/python/skills/refactoring/routing-table.md @@ -14,7 +14,7 @@ findings the hunters own alone. | Duplicated code (review-only) | `../../rules/R1-primitive-obsession.md` (extract shared type/logic); duplicated `match`/if-chains on the same kind/type discriminator → `../../rules/R11-conditional-dispatch.md` | | A `match` missing enum cases (review-only; ty and mypy see it only under `assert_never`) | `../../rules/R11-conditional-dispatch.md` — handle the case at the single dispatch site and close it with `case _: assert_never(x)`; a second `match` appearing is the R11 violation itself | | `FBT001` / `FBT002` (boolean positional parameter) | Mechanical — make it keyword-only; then read the branches: sharing little → `../../rules/R11-conditional-dispatch.md`, Split Flag Argument | -| File over ~450 lines (review-only) | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | +| File over the file-length limit (`scripts/ldd-lint.sh file-length`) | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | | `PLW0603` (`global` statement); import-time side effects (review-only) | `../../rules/R8-no-globals.md` | | A `Protocol`/ABC with one implementer; a factory annotated to return the Protocol (review-only) | `../../rules/R6-test-only-interfaces.md` | | Unguarded shared state; a `daemon=True` worker; `time.sleep` in a stoppable loop (review-only) | `../../rules/R10-concurrency-safety.md` | diff --git a/lang/ts-react/agents/lint-fixer/routing-table.md b/lang/ts-react/agents/lint-fixer/routing-table.md index 728524de..ef536e7a 100644 --- a/lang/ts-react/agents/lint-fixer/routing-table.md +++ b/lang/ts-react/agents/lint-fixer/routing-table.md @@ -7,7 +7,7 @@ | `sonarjs/no-duplicate-string` on an enum-shaped literal / `no-magic-numbers` on a domain value / `sonarjs/max-union-size` | rules/R1-primitive-obsession.md ("Name enum strings" move, or a named type) | | A tuple return (`[T, boolean]`, `[Data, Error]`); a `Record`, `string[]` or `Map` of primitives crossing a function boundary or nested in a type (review-only — no ESLint rule) | rules/R1-primitive-obsession.md ("Name the Container" move — Q7 nested, Q8 flat across a boundary) | | `sonarjs/no-identical-functions` | rules/R1-primitive-obsession.md (extract shared type/logic); duplicated `switch`/if-chains on one discriminant → rules/R11-conditional-dispatch.md | -| `sonarjs/max-lines` (600) | rules/R5-vertical-slice.md | +| `sonarjs/max-lines`; `scripts/ldd-lint.sh file-length` and `package-size` findings | rules/R5-vertical-slice.md | | `react/no-multi-comp` | rules/R5-vertical-slice.md (one component per file; the page folder decides where the second goes) | | `@typescript-eslint/no-explicit-any` / `no-unsafe-*` / `no-non-null-assertion` / `no-unnecessary-condition` at a boundary | rules/R2-self-validating-types.md (parse at the boundary — a guard, not an assertion) | | `@typescript-eslint/switch-exhaustiveness-check` / `sonarjs/no-nested-switch` / `sonarjs/max-switch-cases` / `sonarjs/no-small-switch` | rules/R11-conditional-dispatch.md (via @refactoring) | diff --git a/lang/ts-react/passthrough/CHANGELOG.md b/lang/ts-react/passthrough/CHANGELOG.md index ddc0c6eb..cad904eb 100644 --- a/lang/ts-react/passthrough/CHANGELOG.md +++ b/lang/ts-react/passthrough/CHANGELOG.md @@ -5,6 +5,26 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio ## [Unreleased] +### Added + +- **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` + measures lines per source file (`file-length`) and source files per directory + (`package-size`). Where the repository's linter config holds the file-length + limit, the script uses that limit and counts lines the way that linter counts. + Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and + otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 + source files per directory and the limit at 13. The first line of the output + names the limit that applied; with the defaults it also prints the setting that + makes the repository's linter hold the limit. `--tsv` writes findings as rows. + The script never edits a file. + +### Changed + +- **One set of thresholds.** The skills and routing tables name the same limits the + gap linter applies: the repository linter's limit where it sets one, else + 450 code lines per file and 8 and 13 files per directory. The package-size + procedure runs the gap linter in place of a hand-written `find` command. + ### Fixed - **A claim that something is missing needs a listing.** A rule hunter that reports diff --git a/lang/ts-react/profile.yaml b/lang/ts-react/profile.yaml index d5950f4b..84261bdc 100644 --- a/lang/ts-react/profile.yaml +++ b/lang/ts-react/profile.yaml @@ -46,6 +46,13 @@ doc_form: JSDoc doc_comment: JSDoc comment src_ext: .ts unexported: internal +# The gap linter's thresholds, where the repository's linter holds none: code +# lines per source file (blank and comment-only lines are not counted), and +# source files per directory that start the warning zone and the limit. A +# repository's linter config, then its .ldd-lint.yaml, wins over these. +file_max_lines: "450" +package_warn_files: "8" +package_max_files: "13" # The standalone coding-rules document rendered from core/handbook/ with this # binding's files under handbook/, written outside the plugin directory. handbook: coding-rules/ts-react.md diff --git a/lang/ts-react/rules/R5/falsifying-questions.md b/lang/ts-react/rules/R5/falsifying-questions.md index d5adfd6d..26ab30b8 100644 --- a/lang/ts-react/rules/R5/falsifying-questions.md +++ b/lang/ts-react/rules/R5/falsifying-questions.md @@ -13,7 +13,7 @@ Detection: for each feature noun in the diff, `grep -rli '' --include='*.ts' --include='*.tsx' --exclude-dir=node_modules src | xargs -n1 dirname | sort -u` — count distinct layer-named directories. The inverse shape is the god file: - `sonarjs/max-lines` (600) on a `Performance.tsx` of 1,199 lines holding the page, + `sonarjs/max-lines` on a `Performance.tsx` of 1,199 lines holding the page, its hooks, its API calls and its types in one module. Violation: the same feature living in `components/`, `hooks/`, `services/` and `types/` — four top-level directories for one page — it is horizontally diff --git a/lang/ts-react/scripts/ldd-fixture.sh b/lang/ts-react/scripts/ldd-fixture.sh index 776d1fe8..12a95747 100644 --- a/lang/ts-react/scripts/ldd-fixture.sh +++ b/lang/ts-react/scripts/ldd-fixture.sh @@ -11,6 +11,20 @@ # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="typescript" FX_NATIVE="typescript" @@ -18,6 +32,11 @@ FX_NATIVE="typescript" # names for the files it creates from it, and `fresh.ts*` is no file name. # Every file the row writes is a `.ts` file, so both globs select the same set. use_row() { + FX_EXT='.ts' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='index.ts' FX_GLOB='*.ts' FX_SRC_FILE="services/worker.ts" FX_SUPPRESS_LINE='// eslint-disable-next-line import/no-mutable-exports' @@ -51,3 +70,14 @@ it('runs', async () => { }) TS } + +fx_test_file() { printf '%s/m%s.test.ts' "$1" "$2"; } + +# ESLint's own max-lines counts every line unless told to skip comments. +fx_write_linter_config() { # + printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +} + +# Every ESLint config form resolves: a rule named without a limit takes +# ESLint's documented default. +fx_write_linter_config_unreadable() { return 1; } diff --git a/lang/ts-react/skills/code-designing/linter-triggers.md b/lang/ts-react/skills/code-designing/linter-triggers.md index f6ee36c5..1c51c1ab 100644 --- a/lang/ts-react/skills/code-designing/linter-triggers.md +++ b/lang/ts-react/skills/code-designing/linter-triggers.md @@ -3,5 +3,5 @@ - a tuple return (`[T, boolean]`, `[Data, Error]`), or a `Record`, `string[]` or `Map` of primitives returned or accepted across a function boundary → the same named result type, with no ESLint rule to say so (R1, Name the Container) - `sonarjs/cognitive-complexity` or `sonarjs/max-lines-per-function` on a component, or `react/no-unstable-nested-components` → the component is the fat function: design the hook and the leaf type it should be reading (per `../../rules/R3-storifying.md`), not a smaller component nested inside it - the same `switch` on one discriminant in a second file, or `@typescript-eslint/no-explicit-any` at a boundary → design the discriminated union and its one dispatch owner (`../../rules/R11-conditional-dispatch.md`), or the parser that builds the type from `unknown` (`../../rules/R2-self-validating-types.md`) - - `sonarjs/max-lines` (600; or a module over ~450 lines where the repository sets no limit — count) → split juicy types into their own modules (juiciness per R1; module-per-type per `../../rules/R5-vertical-slice.md`); a single god component or class routes to @refactoring's god-object decomposition procedure first + - `sonarjs/max-lines` (the repository's limit, else the gap linter's {{.FileMaxLines}} code lines) → split juicy types into their own modules (juiciness per R1; module-per-type per `../../rules/R5-vertical-slice.md`); a single god component or class routes to @refactoring's god-object decomposition procedure first - Package-size yellow/red zone → re-model with sub-folders *before* the zone escalates (`` in @refactoring's `reference.md` (`sed -n '/^/,/^<\/package_decomposition>/p; /^### Package decomposition/,$p'`)) diff --git a/lang/ts-react/skills/refactoring/file-and-package-routing.md b/lang/ts-react/skills/refactoring/file-and-package-routing.md index 37d84e9b..350ef1ff 100644 --- a/lang/ts-react/skills/refactoring/file-and-package-routing.md +++ b/lang/ts-react/skills/refactoring/file-and-package-routing.md @@ -1,4 +1,4 @@ -**`sonarjs/max-lines` (600; or a module over ~450 lines where the repository sets no limit — count):** +**`sonarjs/max-lines`, or a module over {{.FileMaxLines}} code lines where the repository sets no limit** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: | File pattern | Action | |---|---| @@ -8,14 +8,19 @@ | Several components in one file (`react/no-multi-comp`) | One component per file; the page folder decides where the second lands (R4, R5) | -**Package-size zones** — count non-test `.ts`/`.tsx` modules per directory: +**Package-size zones** — non-test `.ts`/`.tsx` modules per directory, `index.ts`, declaration files and stories left out: ``` -find -maxdepth 1 -type f \( -name '*.ts' -o -name '*.tsx' \) -not -name '*.test.*' -not -name '*.spec.*' -not -name '*.stories.*' -not -name '*.d.ts' -not -name 'index.ts' | wc -l +bash /../../scripts/ldd-lint.sh package-size --root . ``` -≤7 green — fine. 8–12 yellow — design review *before the next file lands*. ≥13 red — -**must decompose**. Either zone: run the 3-step design review under "Package -decomposition", printed with this range (it is a *design* review — missing domain types are the +The gap linter counts the non-test, non-generated source files of each directory; +a `.ldd-lint.yaml` at the repository root sets other sizes, and the first line of +its output says which sizes applied. + +Below {{.PackageWarnFiles}} files green — fine. {{.PackageWarnFiles}} or more yellow — design review +*before the next file lands*. {{.PackageMaxFiles}} or more red — **must decompose**. Either +zone: run the 3-step design review under "Package decomposition", printed with this +range (it is a *design* review — missing domain types are the disease, file count the symptom). Invoke @code-designing to validate extracted types. diff --git a/lang/ts-react/skills/refactoring/routing-table.md b/lang/ts-react/skills/refactoring/routing-table.md index c3525e39..1b90b6d8 100644 --- a/lang/ts-react/skills/refactoring/routing-table.md +++ b/lang/ts-react/skills/refactoring/routing-table.md @@ -13,7 +13,7 @@ detector: those rows are review findings the hunters own alone. | A tuple return (`[T, boolean]`, `[Data, Error]`); a `Record`, `string[]` or `Map` of primitives crossing a function boundary or nested in a type (review-only) | `../../rules/R1-primitive-obsession.md` (Name the Container — Q7 nested, Q8 flat across a boundary; the receivers' lookups, filters and loops are the methods of the type that does not exist yet) | | `sonarjs/no-duplicate-string` on an enum-shaped literal; `no-magic-numbers` on a domain value; `sonarjs/max-union-size` | `../../rules/R1-primitive-obsession.md` (Name enum strings, or a named type) | | `sonarjs/no-identical-functions`; duplicated code (review-only) | `../../rules/R1-primitive-obsession.md` (extract shared type/logic); duplicated `switch`/if-chains on the same kind/type discriminator → `../../rules/R11-conditional-dispatch.md` | -| `sonarjs/max-lines` (600) | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | +| `sonarjs/max-lines`; `scripts/ldd-lint.sh file-length` and `package-size` findings | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | | `react/no-multi-comp` | `../../rules/R5-vertical-slice.md` — one component per file; the page folder decides where the second goes | | `@typescript-eslint/no-explicit-any` / `@typescript-eslint/no-unsafe-*` / `@typescript-eslint/no-non-null-assertion` / `@typescript-eslint/no-unnecessary-condition` at a boundary | `../../rules/R2-self-validating-types.md` (parse at the boundary — a guard, not an assertion) | | `@typescript-eslint/switch-exhaustiveness-check` (when configured; review-only otherwise) / `sonarjs/no-nested-switch` / `sonarjs/max-switch-cases` / `sonarjs/no-small-switch` | `../../rules/R11-conditional-dispatch.md` — handle the case at the single dispatch site and close it with `default: return assertNever(x)`; a second `switch` appearing is the R11 violation itself | diff --git a/linter-driven-development/CHANGELOG.md b/linter-driven-development/CHANGELOG.md index b63f092a..4cf64e87 100644 --- a/linter-driven-development/CHANGELOG.md +++ b/linter-driven-development/CHANGELOG.md @@ -5,6 +5,26 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio ## [Unreleased] +### Added + +- **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` + measures lines per source file (`file-length`) and source files per directory + (`package-size`). Where the repository's linter config holds the file-length + limit, the script uses that limit and counts lines the way that linter counts. + Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and + otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 + source files per directory and the limit at 13. The first line of the output + names the limit that applied; with the defaults it also prints the setting that + makes the repository's linter hold the limit. `--tsv` writes findings as rows. + The script never edits a file. + +### Changed + +- **One set of thresholds.** The skills and routing tables name the same limits the + gap linter applies: the repository linter's limit where it sets one, else + 450 code lines per file and 8 and 13 files per directory. The package-size + procedure runs the gap linter in place of a hand-written `find` command. + ### Fixed - **A claim that something is missing needs a listing.** A rule hunter that reports diff --git a/linter-driven-development/README.md b/linter-driven-development/README.md index 8ab046d6..bf11dbfd 100644 --- a/linter-driven-development/README.md +++ b/linter-driven-development/README.md @@ -107,8 +107,8 @@ linter-driven-development/ code is Go, for demonstration only; each opens with that note. The move and the reasoning hold in any language — read the Go as the shape and spell it in yours. -Not shipped here, because they are Go-specific rather than Go-illustrated: the -test-harness catalogue and the package-size hook. +Not shipped here, because it is Go-specific rather than Go-illustrated: the +test-harness catalogue. ## Mixed repositories diff --git a/linter-driven-development/scripts/ldd-detect_test.sh b/linter-driven-development/scripts/ldd-detect_test.sh index 9b7cd626..cd5b9a29 100644 --- a/linter-driven-development/scripts/ldd-detect_test.sh +++ b/linter-driven-development/scripts/ldd-detect_test.sh @@ -134,17 +134,41 @@ questions_in_rules() { # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="go python" FX_NATIVE="go python d typescript" use_row() { case "$1" in go) + FX_EXT='.go' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='api.pb.go' FX_GLOB='*.go' FX_SRC_FILE="services/worker.go" FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' ;; python) + FX_EXT='.py' + FX_LINE_COMMENT='# note' + FX_BLOCK_OPEN='' + FX_BLOCK_CLOSE='' + FX_SKIP_FILE='__init__.py' FX_GLOB='*.py' FX_SRC_FILE="services/worker.py" FX_SUPPRESS_LINE='CACHE: dict[str, str] = {} # noqa: PLW0603' @@ -218,6 +242,27 @@ PY esac } +fx_test_file() { + case "$ROW" in + go) printf '%s/m%s_test.go' "$1" "$2" ;; + python) printf '%s/test_m%s.py' "$1" "$2" ;; + esac +} + +fx_write_linter_config() { # + case "$ROW" in + go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" ;; + python) printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" ;; + esac +} + +fx_write_linter_config_unreadable() { + case "$ROW" in + go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; + python) printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" ;; + esac +} + # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/linter-driven-development/scripts/ldd-lint.sh b/linter-driven-development/scripts/ldd-lint.sh new file mode 100755 index 00000000..ccb47daf --- /dev/null +++ b/linter-driven-development/scripts/ldd-lint.sh @@ -0,0 +1,634 @@ +#!/usr/bin/env bash +# Gap linter of the linter-driven-development plugin. +# +# Measures the design triggers no linter holds, or that the repository's linter +# does not hold yet: lines per source file and source files per directory. It +# never edits a file. Where the repository's linter holds a limit, that limit +# is the one applied, counted the way that linter counts. +# +# Usage: bash scripts/ldd-lint.sh [options] +# file-length source files over the file-length limit +# package-size directories at or over the warning size or the limit +# --root the repository (default: cwd) +# --files measure the source files this list names, one +# relative path per line (a scope bundle's files.txt +# works: a "(not bundled: …)" suffix is ignored); +# default: every source file of the repository +# --tsv findings as rows: path TAB line TAB message, line 0; +# the threshold lines then go to stderr +# --glob source-file glob, for a language the table does not +# know (any-language block below) +# --test-re test-file pattern over relative paths +# +# Thresholds, per language, first that applies — the "threshold:" line says +# which one a run used: +# 1. the repository's linter config holds the rule: its limit, counted the +# way that linter counts (file-length only; no linter counts files per +# directory); a config the table cannot read is named, then skipped +# 2. .ldd-lint.yaml at the repository root: file_max_lines, +# package_warn_files, package_max_files, each a positive integer +# 3. the plugin's defaults: 450 code lines per file (blank and +# comment-only lines not counted); a warning at 8 source files per +# directory and the limit at 13; for file-length, the line also +# prints the setting that makes the repository's linter hold the limit +# +# What is measured: the language's source files, minus test files, generated +# files and the directories the language block excludes. A directory count +# also leaves out the files the linter table names (an index module, a package +# marker, generated stubs). A file is over the limit when its count is above +# it; a directory is in the warning zone at the warning size and over the +# limit at the limit. +# +# Exit codes: 0 measured, with or without findings · 2 usage error, a --files +# list that does not exist, or a .ldd-lint.yaml value that is not a +# positive integer (inconclusive, never silently clean) +# +# Uses only POSIX-portable tools plus git when the root is a repository: +# grep, sed, awk, sort, find, head. + +set -u + +SCRIPT_NAME="ldd-lint" +DEFAULT_FILE_MAX_LINES=450 +DEFAULT_PACKAGE_WARN_FILES=8 +DEFAULT_PACKAGE_MAX_FILES=13 +OVERRIDE_FILE=".ldd-lint.yaml" +ROOT="$(pwd)" +FILES_LIST="" +TSV=0 +OPT_GLOB="" +OPT_TEST_RE="" +SUBCOMMAND="" + +usage() { sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//' >&2; exit 2; } +die() { echo "$SCRIPT_NAME: $*" >&2; exit 2; } + +while (( $# > 0 )); do + case "$1" in + file-length|package-size) [[ -n "$SUBCOMMAND" ]] && die "one subcommand, got also: $1"; SUBCOMMAND="$1"; shift ;; + --root) ROOT="${2:-}"; shift 2 ;; + --files) FILES_LIST="${2:-}"; shift 2 ;; + --tsv) TSV=1; shift ;; + --glob) OPT_GLOB="${2:-}"; shift 2 ;; + --test-re) OPT_TEST_RE="${2:-}"; shift 2 ;; + -h|--help) usage ;; + *) die "unknown argument: $1 (see --help)" ;; + esac +done +[[ -n "$SUBCOMMAND" ]] || die "name a subcommand: file-length or package-size (see --help)" +[[ -d "$ROOT" ]] || die "not a directory: $ROOT" +if [[ -n "$FILES_LIST" ]]; then + [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" + FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") +fi +cd "$ROOT" || exit 2 + +# ==================== language table ==================== +# How a file's language is told, and which plugin owns a language. The same +# table is rendered into every plugin, so two plugins that split one scope +# agree on every file. The language block below says which of these ids this +# plugin reviews itself; the rest are routed to the plugin that owns them. +# residue-exempt: the table names every language's extension and plugin on +# purpose, the same in every rendering. +# +# lang_of_path the language id of a source file, by extension: +# "custom" for a file matching --glob (then nothing +# else is a source file), "" for a file that is not +# source (a manifest, a doc, a build file) +# dedicated_plugin_for the plugin written for that language, or "" +# GENERIC_PLUGIN the plugin that reviews any language +# installed_plugin_dir where plugin is installed, from the +# installed-plugins file, or "" when it is not; +# LDD_INSTALLED_PLUGINS names another file +GENERIC_PLUGIN='linter-driven-development' + +# A .h header is C++ when the repository has C++ sources, C otherwise. +HEADER_LANG="" +header_lang() { + [[ -n "$HEADER_LANG" ]] && { printf '%s' "$HEADER_LANG"; return; } + if { git ls-files -- '*.cpp' '*.cc' '*.cxx' '*.hpp' 2>/dev/null || find . -type f \( -name '*.cpp' -o -name '*.cc' -o -name '*.cxx' -o -name '*.hpp' \) -not -path './.git/*' 2>/dev/null; } | head -1 | grep -q .; then + HEADER_LANG=cpp + else + HEADER_LANG=c + fi + printf '%s' "$HEADER_LANG" +} + +lang_of_path() { + local b="${1##*/}" + if [[ -n "${OPT_GLOB:-}" ]]; then + case "$b" in $OPT_GLOB) printf 'custom' ;; esac + return 0 + fi + case "$b" in + *.go) printf 'go' ;; + *.py) printf 'python' ;; + *.d|*.di) printf 'd' ;; + *.rs) printf 'rust' ;; + *.c) printf 'c' ;; + *.h) header_lang ;; + *.cpp|*.cc|*.cxx|*.hpp|*.hh|*.hxx) printf 'cpp' ;; + *.java) printf 'java' ;; + *.kt|*.kts) printf 'kotlin' ;; + *.rb) printf 'ruby' ;; + *.sh|*.bash) printf 'shell' ;; + *.ts|*.tsx) printf 'typescript' ;; + *.js|*.jsx|*.mjs|*.cjs) printf 'javascript' ;; + *.cs) printf 'csharp' ;; + esac + return 0 +} + +dedicated_plugin_for() { + case "$1" in + go|python) printf '%s-linter-driven-development' "$1" ;; + typescript|javascript) printf 'ts-react-linter-driven-development' ;; + esac + return 0 +} + +installed_plugin_dir() { + local f="${LDD_INSTALLED_PLUGINS:-$HOME/.claude/plugins/installed_plugins.json}" + [[ -f "$f" ]] || return 0 + awk -v key="\"$1@" ' + index($0, key) { found = 1; next } + found && /"installPath"/ { sub(/^[^:]*:[ \t]*"/, ""); sub(/".*$/, ""); print; exit } + ' "$f" +} +# ================== end language table ================== + +# ==================== language block: any language ==================== +# Everything language-specific the review scripts need. The driver calls only +# the LANG_* variables and lang_* functions defined here. This is the generic +# plugin's block: one row per language the language table above names, so a +# group of any of them is reviewed here when no plugin written for that +# language is installed. Which file is which language is the table, shared by +# every plugin; this block only says how each language writes a comment, a +# suppression directive and a test file. +# +# Contract: +# LANG_NATIVE the language ids this plugin reviews itself; a group of +# any other id is routed to the plugin that owns it +# lang_configure sets the variables below for that id; "custom" is the +# --glob override: a language the table does not know, +# reviewed with the broad row at the end; +# exit 2 for an id this block has no row for +# LANG_EXCLUDE_RE ERE over a relative path: directories never in scope +# LANG_SUPPRESS_RE ERE: a lint-suppression directive on a source line +# LANG_COMMENT_RE ERE: a source line that carries a comment (bracket +# expressions, no backslashes: the driver hands it to +# awk -v, which eats `\*` under gawk) +# LANG_DIRECTIVE_RE ERE: a comment line that is a directive, not prose +# LANG_GENERATED_RE ERE: a marker in a file's head that says it is generated +# lang_is_test exit 0 iff the path is a test file (--test-re overrides) +LANG_NATIVE="go python d rust c cpp java kotlin ruby shell typescript javascript csharp custom" +LANG_ID="" + +lang_configure() { + LANG_ID="$1" + LANG_GENERATED_RE='Code generated .* DO NOT EDIT|DO NOT EDIT|@generated|automatically generated|Generated by' + case "$1" in + go) + LANG_EXCLUDE_RE='(^|/)(vendor|\.git|testdata)/' + LANG_SUPPRESS_RE='//nolint' + LANG_COMMENT_RE='//' + LANG_DIRECTIVE_RE='//(go:|nolint| Output:|line |export |extern )' + ;; + python) + LANG_EXCLUDE_RE='(^|/)(\.venv|venv|\.git|node_modules|__pycache__|\.tox|build|dist)/' + LANG_SUPPRESS_RE='#\s*(noqa|type:\s*ignore|ty:\s*ignore)' + LANG_COMMENT_RE='(#|""")' + LANG_DIRECTIVE_RE='#\s*(noqa|type:|ty:|pragma|fmt:|pylint:|ruff:|isort:|!)|>>>' + ;; + d) + # D comments: // line, /* block */ and /+ nested +/. D-Scanner's + # suppression is a comment: // @suppress(dscanner.). A file's + # inline `unittest` blocks test the production code beside them and keep + # the file a production file; a test file is one under a test directory + # or named *_test.d. + LANG_EXCLUDE_RE='(^|/)(\.git|\.dub|vendor)/' + LANG_SUPPRESS_RE='@suppress\(' + LANG_COMMENT_RE='(//|/[*]|/[+])' + LANG_DIRECTIVE_RE='@suppress\(|dfmt (on|off)|#!' + ;; + rust) + LANG_EXCLUDE_RE='(^|/)(\.git|target|vendor)/' + LANG_SUPPRESS_RE='#!?\[allow\(' + LANG_COMMENT_RE='(//|/[*])' + LANG_DIRECTIVE_RE='rustfmt::skip|clippy::' + ;; + c|cpp) + LANG_EXCLUDE_RE='(^|/)(\.git|third_party|vendor)/' + LANG_SUPPRESS_RE='NOLINT|cppcheck-suppress|#pragma (warning\s*\(\s*disable|GCC diagnostic ignored|clang diagnostic ignored)' + LANG_COMMENT_RE='(//|/[*])' + LANG_DIRECTIVE_RE='NOLINT|cppcheck-suppress|clang-format (on|off)|IWYU' + ;; + java|kotlin) + LANG_EXCLUDE_RE='(^|/)(\.git|target|build|\.gradle)/' + LANG_SUPPRESS_RE='@Suppress(Warnings)?\(|NOSONAR|noinspection|CHECKSTYLE:OFF|ktlint-disable|detekt:' + LANG_COMMENT_RE='(//|/[*])' + LANG_DIRECTIVE_RE='noinspection|CHECKSTYLE|NOSONAR|@formatter:|ktlint-|detekt:' + ;; + ruby) + LANG_EXCLUDE_RE='(^|/)(\.git|vendor/bundle|tmp)/' + LANG_SUPPRESS_RE='rubocop:disable|steep:ignore' + LANG_COMMENT_RE='(#|=begin)' + LANG_DIRECTIVE_RE='rubocop:|frozen_string_literal|encoding:|#!' + ;; + shell) + LANG_EXCLUDE_RE='(^|/)(\.git)/' + LANG_SUPPRESS_RE='shellcheck disable' + LANG_COMMENT_RE='#' + LANG_DIRECTIVE_RE='shellcheck|#!' + ;; + typescript|javascript) + LANG_EXCLUDE_RE='(^|/)(\.git|node_modules|dist)/' + LANG_SUPPRESS_RE='eslint-disable|@ts-(ignore|expect-error|nocheck)|biome-ignore' + LANG_COMMENT_RE='(//|/[*])' + LANG_DIRECTIVE_RE='eslint-|@ts-|prettier-ignore|biome-ignore|/// * c (// and /* */), d (also /+ +/) or hash (#) +# lint_package_skip_re ERE over a relative path: source files a +# directory count leaves out (an index module, +# a package marker, generated stubs); "" for none +# lint_file_length_config one line, from the repository root: +# covered TAB TAB TAB +# TAB +# unreadable TAB +# none +# lint_file_length_snippet the setting that makes the repository's +# linter hold the limit, or how to set it when +# no linter rule is known + +lint_comment_style() { + case "$1" in + python|ruby|shell) printf 'hash' ;; + d) printf 'd' ;; + *) printf 'c' ;; + esac +} + +lint_package_skip_re() { + case "$1" in + go) printf '%s' '(_gen|\.pb)\.go$' ;; + python) printf '%s' '(^|/)(__init__|conftest)\.py$|_pb2(_grpc)?\.py$' ;; + typescript|javascript) printf '%s' '(^|/)index\.[cm]?[jt]sx?$' ;; + esac + return 0 +} + +# _lint_window — the key's line and the lines after it, up to +# the next rule entry or 15 lines, joined by spaces; one window per match. The +# patterns reach awk as strings, so they use bracket expressions, never +# backslash escapes, which awk -v would rewrite. +_lint_window() { + awk -v key="$1" -v q="'" ' + BEGIN { + entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" + qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" + } + function flush() { if (w != "") print w; w = ""; n = 0 } + { + if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() + if ($0 ~ key) { flush(); n = 1; w = $0; next } + if (n > 0) { w = w " " $0; if (++n > 15) flush() } + } + END { flush() } + ' "$2" +} + +# _lint_number — the first match's digits +_lint_number() { + printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' +} + +# _lint_off — exit 0 when the rule in the window is switched off +_lint_off() { + printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" +} + +_lint_config_go() { + local f w max sc sb unread="" + for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do + [[ -f "$f" ]] || continue + grep -q 'file-length-limit' "$f" 2>/dev/null || continue + case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac + while IFS= read -r w; do + [[ "$w" =~ disabled:[[:space:]]*true ]] && continue + max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") + if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi + (( max > 0 )) || continue + sc=0; sb=0 + [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +_lint_config_eslint() { + local f w max sc sb + for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do + [[ -f "$f" ]] || continue + grep -q 'max-lines' "$f" 2>/dev/null || continue + # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") + printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" + return 0 + done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") + # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") + sc=0; sb=0 + [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") + done + printf 'none\n' +} + +_lint_config_python() { + local f line max unread="" + for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do + [[ -f "$f" ]] || continue + line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) + [[ -n "$line" ]] || continue + max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") + if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi + (( max > 0 )) || continue + # pylint counts every line of the module. + printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" + return 0 + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +lint_file_length_config() { + case "$1" in + go) _lint_config_go ;; + typescript|javascript) _lint_config_eslint ;; + python) _lint_config_python ;; + *) printf 'none\n' ;; + esac +} + +lint_file_length_snippet() { + case "$1" in + go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; + typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; + python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; + *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + esac +} +# ================== end linter table ================== + +# ---------- output ---------- +# A threshold line is for the reader of the report; with --tsv the rows are +# the output and the threshold lines go to stderr. +say() { if (( TSV )); then printf '%s\n' "$*" >&2; else printf '%s\n' "$*"; fi; } +finding() { # + if (( TSV )); then printf '%s\t0\t%s\n' "$1" "$2"; else printf '%s: %s\n' "$1" "$2"; fi +} + +# ---------- the repository override ---------- +# override — the key's value in .ldd-lint.yaml, "" when absent; a value +# that is not a positive integer ends the run, because the file is the +# repository's own statement of its limits. +override() { + [[ -f "$OVERRIDE_FILE" ]] || return 0 + local v + v=$(awk -v k="$1" -v q="'" ' + { line = $0; sub(/#.*/, "", line) } + line ~ "^[[:space:]]*" k "[[:space:]]*:" { + sub("^[[:space:]]*" k "[[:space:]]*:[[:space:]]*", "", line) + gsub("[\"" q "[:space:]]", "", line) + print line; exit + }' "$OVERRIDE_FILE") + [[ -z "$v" ]] && return 0 + [[ "$v" =~ ^[0-9]+$ ]] && (( v > 0 )) || die "$OVERRIDE_FILE: $1 must be a positive integer, got: $v" + printf '%s' "$v" +} + +# ---------- the scope ---------- +# One "id TAB path" line per source file of a language this plugin reviews, +# sorted, so two runs over one tree print the same thing. +list_candidates() { + if [[ -n "$FILES_LIST" ]]; then + sed -e 's/ (not bundled:.*$//' -e 's|^\./||' "$FILES_LIST" + elif git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + git ls-files -co --exclude-standard + else + find . -type f -not -path './.git/*' | sed 's|^\./||' + fi +} + +SCOPE=$(mktemp) +trap 'rm -f "$SCOPE"' EXIT +while IFS= read -r f; do + [[ -n "$f" && -f "$f" ]] || continue + id=$(lang_of_path "$f") + [[ -n "$id" ]] || continue + case " $LANG_NATIVE " in *" $id "*) ;; *) continue ;; esac + printf '%s\t%s\n' "$id" "$f" +done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" + +# measured — exit 0 when the file counts for this language: not under +# an excluded directory, not a test, not generated. +measured() { + printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 + lang_is_test "$1" && return 1 + head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + return 0 +} + +# ---------- file-length ---------- +# COUNT_AWK prints a file's line count under the style's comment syntax: +# skipc drops comment-only lines (block comments included), skipb blank lines. +COUNT_AWK=' +function comment_only(t, rest, i) { + if (inblock) { + i = index(t, closer) + if (i == 0) return 1 + inblock = 0 + rest = substr(t, i + length(closer)); sub(/^[[:space:]]+/, "", rest) + return rest == "" || comment_only(rest) + } + if (style == "hash") return substr(t, 1, 1) == "#" + if (substr(t, 1, 2) == "//") return 1 + if (substr(t, 1, 2) == "/*" || (style == "d" && substr(t, 1, 2) == "/+")) { + closer = (substr(t, 2, 1) == "+") ? "+/" : "*/" + inblock = 1 + return comment_only(substr(t, 3)) + } + return 0 +} +{ + t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) + if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } + c = comment_only(t) + if (c && skipc) next + n++ +} +END { print n + 0 } +' + +run_file_length() { # — every measured file of that language + local id="$1" cfg status max sc sb label src unit f n o + cfg=$(lint_file_length_config "$id") + IFS=$'\t' read -r status max sc sb label <<< "$cfg" + if [[ "$status" == "unreadable" ]]; then + # an unreadable row carries the rule and file in its second field + say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" + status=none + fi + if [[ "$status" == "covered" ]]; then + src="$label" + say "threshold: $id file length — covered by $label (limit $max)" + else + sc=1; sb=1 + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" + else + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + fi + fi + unit="lines" + (( sc && sb )) && unit="code lines" + (( sc && ! sb )) && unit="lines without comments" + (( ! sc && sb )) && unit="non-blank lines" + while IFS=$'\t' read -r _ f; do + measured "$f" || continue + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") +} + +# ---------- package-size ---------- +run_package_size() { # — every directory that holds a measured file of that language + local id="$1" warn lim src ow om skip d n f + ow=$(override package_warn_files) || exit 2 + om=$(override package_max_files) || exit 2 + warn="${ow:-$DEFAULT_PACKAGE_WARN_FILES}"; lim="${om:-$DEFAULT_PACKAGE_MAX_FILES}" + (( warn < lim )) || die "$OVERRIDE_FILE: package_warn_files ($warn) must be below package_max_files ($lim)" + if [[ -n "$ow$om" ]]; then src="$OVERRIDE_FILE"; else src="profile default"; fi + say "threshold: $id package size — $src (warning at $warn source files per directory, limit $lim); no linter counts files per directory" + skip=$(lint_package_skip_re "$id") + while IFS= read -r d; do + n=0 + while IFS= read -r f; do + f="${f#./}" + [[ "$(lang_of_path "$f")" == "$id" ]] || continue + [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue + measured "$f" || continue + n=$((n + 1)) + done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + if (( n >= lim )); then + finding "$d" "$n source files (limit $lim, $src)" + elif (( n >= warn )); then + finding "$d" "$n source files (warning at $warn, $src)" + fi + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) +} + +# ---------- run ---------- +ids=$(cut -f1 "$SCOPE" | LC_ALL=C sort -u) +if [[ -z "$ids" ]]; then + say "$SCRIPT_NAME: no source file in scope" + exit 0 +fi +for id in $ids; do + lang_configure "$id" || continue + case "$SUBCOMMAND" in + file-length) run_file_length "$id" ;; + package-size) run_package_size "$id" ;; + esac +done +exit 0 diff --git a/linter-driven-development/scripts/ldd-lint_test.sh b/linter-driven-development/scripts/ldd-lint_test.sh new file mode 100644 index 00000000..ddeceae5 --- /dev/null +++ b/linter-driven-development/scripts/ldd-lint_test.sh @@ -0,0 +1,391 @@ +#!/usr/bin/env bash +# Fixture matrix for ldd-lint.sh — the gap linter's own conformance test. +# +# Each case builds a throwaway tree under a temp dir, runs the script against +# it and asserts the exit code and the lines it prints. The cases are the same +# for every language; the language-specific parts — the source suffix, how a +# comment and a test file are written, the file a directory count leaves out, +# and the repository linter config that holds a file-length limit — come from +# the fixture rows spliced in below. A script whose language block serves +# several languages runs the whole matrix once per row. +# +# Usage: bash scripts/ldd-lint_test.sh [path/to/ldd-lint.sh] +# (default: the sibling ldd-lint.sh) +# Exit: 0 all cases pass · 1 any failure (details on stdout) +# +# Uses only POSIX-portable tools, like the script under test. + +set -u + +HERE=$(cd "$(dirname "$0")" && pwd) +LINT="${1:-$HERE/ldd-lint.sh}" +[[ -f "$LINT" ]] || { echo "no such script: $LINT" >&2; exit 2; } +LINT=$(cd "$(dirname "$LINT")" && pwd)/$(basename "$LINT") +MAX_LINES=450 +WARN_FILES=8 +MAX_FILES=13 + +pass=0 failed=0 skipped=0 +CASE="" +OUT="" ERR="" CODE=0 +ROW="" +REPO="" +TAB=$(printf '\t') + +# ---------- harness ---------- +begin() { CASE="$1"; [[ -n "$ROW" ]] && CASE="[$ROW] $1"; REPO=$(mktemp -d); } +finish() { rm -rf "$REPO"; } + +run_lint() { # [options...] — runs the script over $REPO + local errf; errf=$(mktemp) + OUT=$(cd "$REPO" && bash "$LINT" "$@" --root . 2>"$errf"); CODE=$? + ERR=$(cat "$errf"); rm -f "$errf" +} + +ok() { pass=$((pass + 1)); echo "PASS $CASE"; } +bad() { failed=$((failed + 1)); echo "FAIL $CASE — $1"; echo " exit=$CODE"; printf '%s\n' "$OUT" "$ERR" | sed 's/^/ | /'; } +skip() { skipped=$((skipped + 1)); echo "SKIP $CASE — $1"; } + +expect_exit() { (( CODE == $1 )) || { bad "expected exit $1"; return 1; }; } +expect_has() { # — in stdout or stderr + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" || { bad "missing: $1"; return 1; } +} +expect_not() { + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" && { bad "unexpected: $1"; return 1; } + return 0 +} +expect_no_finding() { # no line names a file or directory of the tree + printf '%s\n' "$OUT" | grep -qE '^[^ ]+: [0-9]+ ' && { bad "unexpected finding"; return 1; } + return 0 +} + +# ---------- fixture builders ---------- +# write_src [ [ []]] — a source +# file with that many code lines, comment-only lines, blank lines and lines of +# one block comment, in that order. +write_src() { + local path="$REPO/$1" code="$2" com="${3:-0}" blank="${4:-0}" block="${5:-0}" i + mkdir -p "$(dirname "$path")" + { + for (( i = 1; i <= code; i++ )); do printf 'v%d = %d\n' "$i" "$i"; done + for (( i = 1; i <= com; i++ )); do printf '%s\n' "$FX_LINE_COMMENT"; done + for (( i = 1; i <= blank; i++ )); do printf '\n'; done + if (( block > 0 )); then + printf '%s\n' "$FX_BLOCK_OPEN" + for (( i = 3; i <= block; i++ )); do printf ' inside the block\n'; done + printf '%s\n' "$FX_BLOCK_CLOSE" + fi + } > "$path" +} + +# fill_dir [] — that many one-line source and test files +fill_dir() { + local i + for (( i = 1; i <= $2; i++ )); do write_src "$1/m$i$FX_EXT" 1; done + for (( i = 1; i <= ${3:-0}; i++ )); do write_src "$(fx_test_file "$1" "$i")" 1; done +} + +# ---------- fixture rows: one per language the detected block supports ---------- +# The matrix runs its cases once per row. use_row sets the row's variables and +# the fx_* builders write that language's files; the cases read only these. +# +# FX_ROWS the row ids, space-separated +# FX_NATIVE of the mixed repository's languages (d, go, python, +# typescript), the ones this plugin's block reviews itself +# FX_GLOB find(1) -name pattern for the row's source files +# FX_SRC_FILE the production file under a layer directory: two sleep +# calls and one suppression directive +# FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it +# fx_write_marker write the project marker for a (sub-)project +# fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable +FX_ROWS="go python" +FX_NATIVE="go python d typescript" + +use_row() { + case "$1" in + go) + FX_EXT='.go' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='api.pb.go' + FX_GLOB='*.go' + FX_SRC_FILE="services/worker.go" + FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' + ;; + python) + FX_EXT='.py' + FX_LINE_COMMENT='# note' + FX_BLOCK_OPEN='' + FX_BLOCK_CLOSE='' + FX_SKIP_FILE='__init__.py' + FX_GLOB='*.py' + FX_SRC_FILE="services/worker.py" + FX_SUPPRESS_LINE='CACHE: dict[str, str] = {} # noqa: PLW0603' + ;; + esac +} + +fx_write_marker() { # + case "$ROW" in + go) printf 'module example.com/%s\n\ngo 1.22\n' "$2" > "$1/go.mod" ;; + python) printf '[project]\nname = "%s"\nversion = "0.1.0"\n' "$2" > "$1/pyproject.toml" ;; + esac +} + +fx_write_code() { + mkdir -p "$REPO/services" + case "$ROW" in + go) + cat > "$REPO/services/worker.go" <<'GO' +package services + +import "time" + +var cache = map[string]string{} //nolint:gochecknoglobals // TODO + +// Run polls until the deadline. +func Run(deadline time.Time) { + for time.Now().Before(deadline) { + time.Sleep(time.Second) + } + time.Sleep(50 * time.Millisecond) +} +GO + cat > "$REPO/services/worker_test.go" <<'GO' +package services_test + +import ( + "testing" + "time" +) + +func TestRun(t *testing.T) { + time.Sleep(10 * time.Millisecond) +} +GO + ;; + python) + mkdir -p "$REPO/tests" + cat > "$REPO/services/worker.py" <<'PY' +"""Polling worker.""" + +import time + +CACHE: dict[str, str] = {} # noqa: PLW0603 + + +def run(deadline: float) -> None: + """Poll until the deadline.""" + while time.monotonic() < deadline: + time.sleep(1) + time.sleep(0.05) +PY + cat > "$REPO/tests/test_worker.py" <<'PY' +import time + + +def test_run() -> None: + time.sleep(0.01) +PY + ;; + esac +} + +fx_test_file() { + case "$ROW" in + go) printf '%s/m%s_test.go' "$1" "$2" ;; + python) printf '%s/test_m%s.py' "$1" "$2" ;; + esac +} + +fx_write_linter_config() { # + case "$ROW" in + go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" ;; + python) printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" ;; + esac +} + +fx_write_linter_config_unreadable() { + case "$ROW" in + go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; + python) printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" ;; + esac +} + +# ========================= cases ========================= +run_row_cases() { # runs every case against the current row + +begin "a file at the limit passes and one line over is a finding" +write_src "src/at$FX_EXT" "$MAX_LINES"; write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" \ + && expect_not "src/at$FX_EXT:" && ok +finish + +begin "comment-only and blank lines are not counted by the plugin's default" +write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 20 20 +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the lines of a block comment are comment-only" +if [[ -z "$FX_BLOCK_OPEN" ]]; then skip "the language has no block comment"; finish +else + write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 0 0 30 + run_lint file-length + expect_exit 0 && expect_no_finding && ok + finish +fi + +begin "a test file over the limit is not measured" +write_src "$(fx_test_file src 1)" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "a generated file over the limit is not measured" +write_src "src/gen$FX_EXT" $((MAX_LINES + 50)) +{ printf '%s DO NOT EDIT\n' "$FX_LINE_COMMENT"; cat "$REPO/src/gen$FX_EXT"; } > "$REPO/src/gen.tmp" && mv "$REPO/src/gen.tmp" "$REPO/src/gen$FX_EXT" +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the plugin's default names the setting that makes the linter hold it" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +finish + +begin "the repository's linter config holds the limit and is counted its way" +fx_write_linter_config 100 +write_src "src/a$FX_EXT" 90 20 +run_lint file-length +expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +finish + +begin "a linter config whose limit cannot be read is named, and the next source applies" +if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish +else + write_src "src/over$FX_EXT" $((MAX_LINES + 1)) + run_lint file-length + expect_exit 0 && expect_has "its limit cannot be read here" \ + && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok + finish +fi + +begin ".ldd-lint.yaml sets the limit when the linter holds none" +printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 101 +run_lint file-length +expect_exit 0 && expect_has "set by .ldd-lint.yaml" && expect_has "src/a$FX_EXT: 101 code lines (limit 100, .ldd-lint.yaml)" && ok +finish + +begin "a .ldd-lint.yaml value that is not a positive integer is inconclusive (exit 2)" +printf 'file_max_lines: lots\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 2 && expect_has "file_max_lines must be a positive integer" && ok +finish + +begin "a directory below the warning size is not a finding" +fill_dir pkg $((WARN_FILES - 1)) 3 +write_src "pkg/$FX_SKIP_FILE" 1 +run_lint package-size +expect_exit 0 && expect_no_finding && ok +finish + +begin "a directory at the warning size is a warning" +fill_dir pkg "$WARN_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $WARN_FILES source files (warning at $WARN_FILES, profile default)" && ok +finish + +begin "a directory at the limit is over the limit" +fill_dir pkg "$MAX_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $MAX_FILES source files (limit $MAX_FILES, profile default)" && ok +finish + +begin ".ldd-lint.yaml sets the directory sizes" +printf 'package_warn_files: 2\npackage_max_files: 3\n' > "$REPO/.ldd-lint.yaml" +fill_dir small 2; fill_dir big 3 +run_lint package-size +expect_exit 0 && expect_has "small: 2 source files (warning at 2, .ldd-lint.yaml)" \ + && expect_has "big: 3 source files (limit 3, .ldd-lint.yaml)" && ok +finish + +begin "--tsv writes path, line 0 and message as rows, and the threshold to stderr" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length --tsv +expect_exit 0 \ + && { [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}$((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" ]] || bad "stdout is not the one row"; } \ + && [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}"* ]] \ + && { [[ "$ERR" == "threshold: "* ]] || bad "threshold line not on stderr"; } && [[ "$ERR" == "threshold: "* ]] && ok +finish + +begin "--files measures only the files it names" +write_src "src/named$FX_EXT" $((MAX_LINES + 1)); write_src "src/other$FX_EXT" $((MAX_LINES + 1)) +printf 'src/named%s (not bundled: %s lines)\n' "$FX_EXT" $((MAX_LINES + 1)) > "$REPO/files.txt" +run_lint file-length --files files.txt +expect_exit 0 && expect_has "src/named$FX_EXT:" && expect_not "src/other$FX_EXT:" && ok +finish + +begin "in a git repository an ignored file is not measured" +write_src "src/kept$FX_EXT" $((MAX_LINES + 1)); write_src "build/out$FX_EXT" $((MAX_LINES + 1)) +printf 'build/\n' > "$REPO/.gitignore" +(cd "$REPO" && git init -q) +run_lint file-length +expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok +finish + +begin "two runs print the same output" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" +run_lint package-size; first="$OUT" +run_lint package-size +expect_exit 0 && { [[ "$OUT" == "$first" ]] || bad "the second run printed something else"; } && [[ "$OUT" == "$first" ]] && ok +finish + +begin "a tree without a source file is measured clean" +printf 'notes\n' > "$REPO/README.md" +run_lint file-length +expect_exit 0 && expect_has "no source file in scope" && ok +finish + +} # run_row_cases + +for ROW in $FX_ROWS; do + use_row "$ROW" + run_row_cases +done +ROW="" + +# --- row-independent --- +begin "a missing subcommand is a usage error (exit 2)" +run_lint +expect_exit 2 && expect_has "name a subcommand" && ok +finish + +begin "a --files list that does not exist is a usage error (exit 2)" +run_lint file-length --files no-such-list.txt +expect_exit 2 && expect_has "no such file list" && ok +finish + +# ========================= summary ========================= +echo +echo "ldd-lint_test: $pass passed, $failed failed, $skipped skipped" +(( failed == 0 )) diff --git a/linter-driven-development/scripts/ldd-scope_test.sh b/linter-driven-development/scripts/ldd-scope_test.sh index 8c0d4b75..1543e338 100755 --- a/linter-driven-development/scripts/ldd-scope_test.sh +++ b/linter-driven-development/scripts/ldd-scope_test.sh @@ -110,17 +110,41 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="go python" FX_NATIVE="go python d typescript" use_row() { case "$1" in go) + FX_EXT='.go' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='api.pb.go' FX_GLOB='*.go' FX_SRC_FILE="services/worker.go" FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' ;; python) + FX_EXT='.py' + FX_LINE_COMMENT='# note' + FX_BLOCK_OPEN='' + FX_BLOCK_CLOSE='' + FX_SKIP_FILE='__init__.py' FX_GLOB='*.py' FX_SRC_FILE="services/worker.py" FX_SUPPRESS_LINE='CACHE: dict[str, str] = {} # noqa: PLW0603' @@ -194,6 +218,27 @@ PY esac } +fx_test_file() { + case "$ROW" in + go) printf '%s/m%s_test.go' "$1" "$2" ;; + python) printf '%s/test_m%s.py' "$1" "$2" ;; + esac +} + +fx_write_linter_config() { # + case "$ROW" in + go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" ;; + python) printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" ;; + esac +} + +fx_write_linter_config_unreadable() { + case "$ROW" in + go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; + python) printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" ;; + esac +} + # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/linter-driven-development/skills/refactoring/reference.md b/linter-driven-development/skills/refactoring/reference.md index b0c4b3ce..55de1a54 100644 --- a/linter-driven-development/skills/refactoring/reference.md +++ b/linter-driven-development/skills/refactoring/reference.md @@ -65,7 +65,7 @@ decomposition): "Multi-rule procedures", below. ## File and package routing -**A file-length limit (the linter's, or ~450 lines when it has none):** +**A file over the file-length limit** — the repository linter's limit where it sets one, else 450 code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: | File pattern | Action | |---|---| @@ -74,16 +74,20 @@ decomposition): "Multi-rule procedures", below. | Long functions, few types | Storify → extract functions (R3) | -**Package-size zones** — count the non-test, non-generated source files in one -directory: +**Package-size zones** — source files per directory: ``` -find -maxdepth 1 -type f -name 'detected-language source' -not -name '*' | wc -l +bash /../../scripts/ldd-lint.sh package-size --root . ``` -≤7 green — fine. 8–12 yellow — design review *before the next file lands*. ≥13 red — -**must decompose**. Either zone: run the 3-step design review under "Package -decomposition", printed with this range (it is a *design* review — missing domain types are the +The gap linter counts the non-test, non-generated source files of each directory; +a `.ldd-lint.yaml` at the repository root sets other sizes, and the first line of +its output says which sizes applied. + +Below 8 files green — fine. 8 or more yellow — design review +*before the next file lands*. 13 or more red — **must decompose**. Either +zone: run the 3-step design review under "Package decomposition", printed with this +range (it is a *design* review — missing domain types are the disease, file count the symptom). Invoke @code-designing to validate extracted types. @@ -265,9 +269,9 @@ composition: @code-designing. ### Package decomposition -Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone (≥13 non-test `.` -files at one directory level) or yellow zone (8–12) — detection command and zone -table in `` above. +Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone (13 or more non-test +`.` files at one directory level) or yellow zone (8 or more) — the +measuring command and the zones are in `` above. **A package-size violation is a design review, not a mechanical file split.** File count is the symptom; the disease is usually missing domain types or multiple diff --git a/python-linter-driven-development/CHANGELOG.md b/python-linter-driven-development/CHANGELOG.md index ad709846..884c2d4a 100644 --- a/python-linter-driven-development/CHANGELOG.md +++ b/python-linter-driven-development/CHANGELOG.md @@ -5,6 +5,26 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio ## [Unreleased] +### Added + +- **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` + measures lines per source file (`file-length`) and source files per directory + (`package-size`). Where the repository's linter config holds the file-length + limit, the script uses that limit and counts lines the way that linter counts. + Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and + otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 + source files per directory and the limit at 13. The first line of the output + names the limit that applied; with the defaults it also prints the setting that + makes the repository's linter hold the limit. `--tsv` writes findings as rows. + The script never edits a file. + +### Changed + +- **One set of thresholds.** The skills and routing tables name the same limits the + gap linter applies: the repository linter's limit where it sets one, else + 450 code lines per file and 8 and 13 files per directory. The package-size + procedure runs the gap linter in place of a hand-written `find` command. + ### Fixed - **A claim that something is missing needs a listing.** A rule hunter that reports diff --git a/python-linter-driven-development/scripts/ldd-detect_test.sh b/python-linter-driven-development/scripts/ldd-detect_test.sh index 600da1fb..f4ab9071 100644 --- a/python-linter-driven-development/scripts/ldd-detect_test.sh +++ b/python-linter-driven-development/scripts/ldd-detect_test.sh @@ -134,10 +134,29 @@ questions_in_rules() { # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="python" FX_NATIVE="python" use_row() { + FX_EXT='.py' + FX_LINE_COMMENT='# note' + FX_BLOCK_OPEN='' + FX_BLOCK_CLOSE='' + FX_SKIP_FILE='__init__.py' FX_GLOB='*.py' FX_SRC_FILE="services/worker.py" FX_SUPPRESS_LINE='CACHE: dict[str, str] = {} # noqa: PLW0603' @@ -172,6 +191,16 @@ def test_run() -> None: PY } +fx_test_file() { printf '%s/test_m%s.py' "$1" "$2"; } + +fx_write_linter_config() { # + printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" +} + +fx_write_linter_config_unreadable() { + printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" +} + # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/python-linter-driven-development/scripts/ldd-lint.sh b/python-linter-driven-development/scripts/ldd-lint.sh new file mode 100755 index 00000000..84956d45 --- /dev/null +++ b/python-linter-driven-development/scripts/ldd-lint.sh @@ -0,0 +1,521 @@ +#!/usr/bin/env bash +# Gap linter of the python-linter-driven-development plugin. +# +# Measures the design triggers no linter holds, or that the repository's linter +# does not hold yet: lines per source file and source files per directory. It +# never edits a file. Where the repository's linter holds a limit, that limit +# is the one applied, counted the way that linter counts. +# +# Usage: bash scripts/ldd-lint.sh [options] +# file-length source files over the file-length limit +# package-size directories at or over the warning size or the limit +# --root the repository (default: cwd) +# --files measure the source files this list names, one +# relative path per line (a scope bundle's files.txt +# works: a "(not bundled: …)" suffix is ignored); +# default: every source file of the repository +# --tsv findings as rows: path TAB line TAB message, line 0; +# the threshold lines then go to stderr +# --glob source-file glob, for a language the table does not +# know (Python block below) +# --test-re test-file pattern over relative paths +# +# Thresholds, per language, first that applies — the "threshold:" line says +# which one a run used: +# 1. the repository's linter config holds the rule: its limit, counted the +# way that linter counts (file-length only; no linter counts files per +# directory); a config the table cannot read is named, then skipped +# 2. .ldd-lint.yaml at the repository root: file_max_lines, +# package_warn_files, package_max_files, each a positive integer +# 3. the plugin's defaults: 450 code lines per file (blank and +# comment-only lines not counted); a warning at 8 source files per +# directory and the limit at 13; for file-length, the line also +# prints the setting that makes the repository's linter hold the limit +# +# What is measured: the language's source files, minus test files, generated +# files and the directories the language block excludes. A directory count +# also leaves out the files the linter table names (an index module, a package +# marker, generated stubs). A file is over the limit when its count is above +# it; a directory is in the warning zone at the warning size and over the +# limit at the limit. +# +# Exit codes: 0 measured, with or without findings · 2 usage error, a --files +# list that does not exist, or a .ldd-lint.yaml value that is not a +# positive integer (inconclusive, never silently clean) +# +# Uses only POSIX-portable tools plus git when the root is a repository: +# grep, sed, awk, sort, find, head. + +set -u + +SCRIPT_NAME="ldd-lint" +DEFAULT_FILE_MAX_LINES=450 +DEFAULT_PACKAGE_WARN_FILES=8 +DEFAULT_PACKAGE_MAX_FILES=13 +OVERRIDE_FILE=".ldd-lint.yaml" +ROOT="$(pwd)" +FILES_LIST="" +TSV=0 +OPT_GLOB="" +OPT_TEST_RE="" +SUBCOMMAND="" + +usage() { sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//' >&2; exit 2; } +die() { echo "$SCRIPT_NAME: $*" >&2; exit 2; } + +while (( $# > 0 )); do + case "$1" in + file-length|package-size) [[ -n "$SUBCOMMAND" ]] && die "one subcommand, got also: $1"; SUBCOMMAND="$1"; shift ;; + --root) ROOT="${2:-}"; shift 2 ;; + --files) FILES_LIST="${2:-}"; shift 2 ;; + --tsv) TSV=1; shift ;; + --glob) OPT_GLOB="${2:-}"; shift 2 ;; + --test-re) OPT_TEST_RE="${2:-}"; shift 2 ;; + -h|--help) usage ;; + *) die "unknown argument: $1 (see --help)" ;; + esac +done +[[ -n "$SUBCOMMAND" ]] || die "name a subcommand: file-length or package-size (see --help)" +[[ -d "$ROOT" ]] || die "not a directory: $ROOT" +if [[ -n "$FILES_LIST" ]]; then + [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" + FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") +fi +cd "$ROOT" || exit 2 + +# ==================== language table ==================== +# How a file's language is told, and which plugin owns a language. The same +# table is rendered into every plugin, so two plugins that split one scope +# agree on every file. The language block below says which of these ids this +# plugin reviews itself; the rest are routed to the plugin that owns them. +# residue-exempt: the table names every language's extension and plugin on +# purpose, the same in every rendering. +# +# lang_of_path the language id of a source file, by extension: +# "custom" for a file matching --glob (then nothing +# else is a source file), "" for a file that is not +# source (a manifest, a doc, a build file) +# dedicated_plugin_for the plugin written for that language, or "" +# GENERIC_PLUGIN the plugin that reviews any language +# installed_plugin_dir where plugin is installed, from the +# installed-plugins file, or "" when it is not; +# LDD_INSTALLED_PLUGINS names another file +GENERIC_PLUGIN='linter-driven-development' + +# A .h header is C++ when the repository has C++ sources, C otherwise. +HEADER_LANG="" +header_lang() { + [[ -n "$HEADER_LANG" ]] && { printf '%s' "$HEADER_LANG"; return; } + if { git ls-files -- '*.cpp' '*.cc' '*.cxx' '*.hpp' 2>/dev/null || find . -type f \( -name '*.cpp' -o -name '*.cc' -o -name '*.cxx' -o -name '*.hpp' \) -not -path './.git/*' 2>/dev/null; } | head -1 | grep -q .; then + HEADER_LANG=cpp + else + HEADER_LANG=c + fi + printf '%s' "$HEADER_LANG" +} + +lang_of_path() { + local b="${1##*/}" + if [[ -n "${OPT_GLOB:-}" ]]; then + case "$b" in $OPT_GLOB) printf 'custom' ;; esac + return 0 + fi + case "$b" in + *.go) printf 'go' ;; + *.py) printf 'python' ;; + *.d|*.di) printf 'd' ;; + *.rs) printf 'rust' ;; + *.c) printf 'c' ;; + *.h) header_lang ;; + *.cpp|*.cc|*.cxx|*.hpp|*.hh|*.hxx) printf 'cpp' ;; + *.java) printf 'java' ;; + *.kt|*.kts) printf 'kotlin' ;; + *.rb) printf 'ruby' ;; + *.sh|*.bash) printf 'shell' ;; + *.ts|*.tsx) printf 'typescript' ;; + *.js|*.jsx|*.mjs|*.cjs) printf 'javascript' ;; + *.cs) printf 'csharp' ;; + esac + return 0 +} + +dedicated_plugin_for() { + case "$1" in + go|python) printf '%s-linter-driven-development' "$1" ;; + typescript|javascript) printf 'ts-react-linter-driven-development' ;; + esac + return 0 +} + +installed_plugin_dir() { + local f="${LDD_INSTALLED_PLUGINS:-$HOME/.claude/plugins/installed_plugins.json}" + [[ -f "$f" ]] || return 0 + awk -v key="\"$1@" ' + index($0, key) { found = 1; next } + found && /"installPath"/ { sub(/^[^:]*:[ \t]*"/, ""); sub(/".*$/, ""); print; exit } + ' "$f" +} +# ================== end language table ================== + +# ===================== language block: python ===================== +# Everything language-specific the review scripts need. The driver calls only +# the LANG_* variables and lang_* functions defined here; a build of the script +# for another language replaces this block and nothing else. Which file is +# which language is the language table above, shared by every plugin. +# +# Contract: +# LANG_NATIVE the language ids this plugin reviews itself; a group of +# any other id is routed to the plugin that owns it +# lang_configure sets the variables below for that id; "custom" is the +# --glob override, reviewed with this block's defaults; +# exit 2 for an id this block has no row for +# LANG_EXCLUDE_RE ERE over a relative path: directories never in scope +# LANG_SUPPRESS_RE ERE: a lint-suppression directive on a source line +# LANG_COMMENT_RE ERE: a source line that carries a comment +# LANG_DIRECTIVE_RE ERE: a comment line that is a directive, not prose +# LANG_GENERATED_RE ERE: a marker in a file's head that says it is generated +# lang_is_test exit 0 iff the path is a test file (--test-re overrides) +LANG_NATIVE="python custom" + +lang_configure() { + case "$1" in python|custom) ;; *) return 2 ;; esac + LANG_EXCLUDE_RE='(^|/)(\.venv|venv|\.git|node_modules|__pycache__|\.tox|build|dist)/' + LANG_SUPPRESS_RE='#\s*(noqa|type:\s*ignore|ty:\s*ignore)' + LANG_COMMENT_RE='(#|""")' + LANG_DIRECTIVE_RE='#\s*(noqa|type:|ty:|pragma|fmt:|pylint:|ruff:|isort:|!)|>>>' + LANG_GENERATED_RE='Generated by|DO NOT EDIT|@generated|automatically generated' + return 0 +} + +lang_is_test() { + [[ -n "${OPT_TEST_RE:-}" ]] && { printf '%s\n' "$1" | grep -qE -- "$OPT_TEST_RE"; return; } + case "$1" in + *_test.py|test_*.py|*/test_*.py|conftest.py|*/conftest.py|tests/*|*/tests/*) return 0 ;; + esac + return 1 +} +# =================== end language block: python =================== + +# ==================== linter table ==================== +# What the gap linter needs to know per language id: how a comment is written, +# which source files a directory count leaves out, and where the repository's +# own linter sets a file-length limit. The same table is rendered into every +# plugin, so two plugins measure one file the same way. +# residue-exempt: the table names every language's linter and config file on +# purpose, the same in every rendering. +# +# lint_comment_style c (// and /* */), d (also /+ +/) or hash (#) +# lint_package_skip_re ERE over a relative path: source files a +# directory count leaves out (an index module, +# a package marker, generated stubs); "" for none +# lint_file_length_config one line, from the repository root: +# covered TAB TAB TAB +# TAB +# unreadable TAB +# none +# lint_file_length_snippet the setting that makes the repository's +# linter hold the limit, or how to set it when +# no linter rule is known + +lint_comment_style() { + case "$1" in + python|ruby|shell) printf 'hash' ;; + d) printf 'd' ;; + *) printf 'c' ;; + esac +} + +lint_package_skip_re() { + case "$1" in + go) printf '%s' '(_gen|\.pb)\.go$' ;; + python) printf '%s' '(^|/)(__init__|conftest)\.py$|_pb2(_grpc)?\.py$' ;; + typescript|javascript) printf '%s' '(^|/)index\.[cm]?[jt]sx?$' ;; + esac + return 0 +} + +# _lint_window — the key's line and the lines after it, up to +# the next rule entry or 15 lines, joined by spaces; one window per match. The +# patterns reach awk as strings, so they use bracket expressions, never +# backslash escapes, which awk -v would rewrite. +_lint_window() { + awk -v key="$1" -v q="'" ' + BEGIN { + entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" + qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" + } + function flush() { if (w != "") print w; w = ""; n = 0 } + { + if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() + if ($0 ~ key) { flush(); n = 1; w = $0; next } + if (n > 0) { w = w " " $0; if (++n > 15) flush() } + } + END { flush() } + ' "$2" +} + +# _lint_number — the first match's digits +_lint_number() { + printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' +} + +# _lint_off — exit 0 when the rule in the window is switched off +_lint_off() { + printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" +} + +_lint_config_go() { + local f w max sc sb unread="" + for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do + [[ -f "$f" ]] || continue + grep -q 'file-length-limit' "$f" 2>/dev/null || continue + case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac + while IFS= read -r w; do + [[ "$w" =~ disabled:[[:space:]]*true ]] && continue + max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") + if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi + (( max > 0 )) || continue + sc=0; sb=0 + [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +_lint_config_eslint() { + local f w max sc sb + for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do + [[ -f "$f" ]] || continue + grep -q 'max-lines' "$f" 2>/dev/null || continue + # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") + printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" + return 0 + done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") + # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") + sc=0; sb=0 + [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") + done + printf 'none\n' +} + +_lint_config_python() { + local f line max unread="" + for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do + [[ -f "$f" ]] || continue + line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) + [[ -n "$line" ]] || continue + max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") + if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi + (( max > 0 )) || continue + # pylint counts every line of the module. + printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" + return 0 + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +lint_file_length_config() { + case "$1" in + go) _lint_config_go ;; + typescript|javascript) _lint_config_eslint ;; + python) _lint_config_python ;; + *) printf 'none\n' ;; + esac +} + +lint_file_length_snippet() { + case "$1" in + go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; + typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; + python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; + *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + esac +} +# ================== end linter table ================== + +# ---------- output ---------- +# A threshold line is for the reader of the report; with --tsv the rows are +# the output and the threshold lines go to stderr. +say() { if (( TSV )); then printf '%s\n' "$*" >&2; else printf '%s\n' "$*"; fi; } +finding() { # + if (( TSV )); then printf '%s\t0\t%s\n' "$1" "$2"; else printf '%s: %s\n' "$1" "$2"; fi +} + +# ---------- the repository override ---------- +# override — the key's value in .ldd-lint.yaml, "" when absent; a value +# that is not a positive integer ends the run, because the file is the +# repository's own statement of its limits. +override() { + [[ -f "$OVERRIDE_FILE" ]] || return 0 + local v + v=$(awk -v k="$1" -v q="'" ' + { line = $0; sub(/#.*/, "", line) } + line ~ "^[[:space:]]*" k "[[:space:]]*:" { + sub("^[[:space:]]*" k "[[:space:]]*:[[:space:]]*", "", line) + gsub("[\"" q "[:space:]]", "", line) + print line; exit + }' "$OVERRIDE_FILE") + [[ -z "$v" ]] && return 0 + [[ "$v" =~ ^[0-9]+$ ]] && (( v > 0 )) || die "$OVERRIDE_FILE: $1 must be a positive integer, got: $v" + printf '%s' "$v" +} + +# ---------- the scope ---------- +# One "id TAB path" line per source file of a language this plugin reviews, +# sorted, so two runs over one tree print the same thing. +list_candidates() { + if [[ -n "$FILES_LIST" ]]; then + sed -e 's/ (not bundled:.*$//' -e 's|^\./||' "$FILES_LIST" + elif git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + git ls-files -co --exclude-standard + else + find . -type f -not -path './.git/*' | sed 's|^\./||' + fi +} + +SCOPE=$(mktemp) +trap 'rm -f "$SCOPE"' EXIT +while IFS= read -r f; do + [[ -n "$f" && -f "$f" ]] || continue + id=$(lang_of_path "$f") + [[ -n "$id" ]] || continue + case " $LANG_NATIVE " in *" $id "*) ;; *) continue ;; esac + printf '%s\t%s\n' "$id" "$f" +done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" + +# measured — exit 0 when the file counts for this language: not under +# an excluded directory, not a test, not generated. +measured() { + printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 + lang_is_test "$1" && return 1 + head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + return 0 +} + +# ---------- file-length ---------- +# COUNT_AWK prints a file's line count under the style's comment syntax: +# skipc drops comment-only lines (block comments included), skipb blank lines. +COUNT_AWK=' +function comment_only(t, rest, i) { + if (inblock) { + i = index(t, closer) + if (i == 0) return 1 + inblock = 0 + rest = substr(t, i + length(closer)); sub(/^[[:space:]]+/, "", rest) + return rest == "" || comment_only(rest) + } + if (style == "hash") return substr(t, 1, 1) == "#" + if (substr(t, 1, 2) == "//") return 1 + if (substr(t, 1, 2) == "/*" || (style == "d" && substr(t, 1, 2) == "/+")) { + closer = (substr(t, 2, 1) == "+") ? "+/" : "*/" + inblock = 1 + return comment_only(substr(t, 3)) + } + return 0 +} +{ + t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) + if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } + c = comment_only(t) + if (c && skipc) next + n++ +} +END { print n + 0 } +' + +run_file_length() { # — every measured file of that language + local id="$1" cfg status max sc sb label src unit f n o + cfg=$(lint_file_length_config "$id") + IFS=$'\t' read -r status max sc sb label <<< "$cfg" + if [[ "$status" == "unreadable" ]]; then + # an unreadable row carries the rule and file in its second field + say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" + status=none + fi + if [[ "$status" == "covered" ]]; then + src="$label" + say "threshold: $id file length — covered by $label (limit $max)" + else + sc=1; sb=1 + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" + else + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + fi + fi + unit="lines" + (( sc && sb )) && unit="code lines" + (( sc && ! sb )) && unit="lines without comments" + (( ! sc && sb )) && unit="non-blank lines" + while IFS=$'\t' read -r _ f; do + measured "$f" || continue + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") +} + +# ---------- package-size ---------- +run_package_size() { # — every directory that holds a measured file of that language + local id="$1" warn lim src ow om skip d n f + ow=$(override package_warn_files) || exit 2 + om=$(override package_max_files) || exit 2 + warn="${ow:-$DEFAULT_PACKAGE_WARN_FILES}"; lim="${om:-$DEFAULT_PACKAGE_MAX_FILES}" + (( warn < lim )) || die "$OVERRIDE_FILE: package_warn_files ($warn) must be below package_max_files ($lim)" + if [[ -n "$ow$om" ]]; then src="$OVERRIDE_FILE"; else src="profile default"; fi + say "threshold: $id package size — $src (warning at $warn source files per directory, limit $lim); no linter counts files per directory" + skip=$(lint_package_skip_re "$id") + while IFS= read -r d; do + n=0 + while IFS= read -r f; do + f="${f#./}" + [[ "$(lang_of_path "$f")" == "$id" ]] || continue + [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue + measured "$f" || continue + n=$((n + 1)) + done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + if (( n >= lim )); then + finding "$d" "$n source files (limit $lim, $src)" + elif (( n >= warn )); then + finding "$d" "$n source files (warning at $warn, $src)" + fi + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) +} + +# ---------- run ---------- +ids=$(cut -f1 "$SCOPE" | LC_ALL=C sort -u) +if [[ -z "$ids" ]]; then + say "$SCRIPT_NAME: no source file in scope" + exit 0 +fi +for id in $ids; do + lang_configure "$id" || continue + case "$SUBCOMMAND" in + file-length) run_file_length "$id" ;; + package-size) run_package_size "$id" ;; + esac +done +exit 0 diff --git a/python-linter-driven-development/scripts/ldd-lint_test.sh b/python-linter-driven-development/scripts/ldd-lint_test.sh new file mode 100644 index 00000000..7890e850 --- /dev/null +++ b/python-linter-driven-development/scripts/ldd-lint_test.sh @@ -0,0 +1,329 @@ +#!/usr/bin/env bash +# Fixture matrix for ldd-lint.sh — the gap linter's own conformance test. +# +# Each case builds a throwaway tree under a temp dir, runs the script against +# it and asserts the exit code and the lines it prints. The cases are the same +# for every language; the language-specific parts — the source suffix, how a +# comment and a test file are written, the file a directory count leaves out, +# and the repository linter config that holds a file-length limit — come from +# the fixture rows spliced in below. A script whose language block serves +# several languages runs the whole matrix once per row. +# +# Usage: bash scripts/ldd-lint_test.sh [path/to/ldd-lint.sh] +# (default: the sibling ldd-lint.sh) +# Exit: 0 all cases pass · 1 any failure (details on stdout) +# +# Uses only POSIX-portable tools, like the script under test. + +set -u + +HERE=$(cd "$(dirname "$0")" && pwd) +LINT="${1:-$HERE/ldd-lint.sh}" +[[ -f "$LINT" ]] || { echo "no such script: $LINT" >&2; exit 2; } +LINT=$(cd "$(dirname "$LINT")" && pwd)/$(basename "$LINT") +MAX_LINES=450 +WARN_FILES=8 +MAX_FILES=13 + +pass=0 failed=0 skipped=0 +CASE="" +OUT="" ERR="" CODE=0 +ROW="" +REPO="" +TAB=$(printf '\t') + +# ---------- harness ---------- +begin() { CASE="$1"; [[ -n "$ROW" ]] && CASE="[$ROW] $1"; REPO=$(mktemp -d); } +finish() { rm -rf "$REPO"; } + +run_lint() { # [options...] — runs the script over $REPO + local errf; errf=$(mktemp) + OUT=$(cd "$REPO" && bash "$LINT" "$@" --root . 2>"$errf"); CODE=$? + ERR=$(cat "$errf"); rm -f "$errf" +} + +ok() { pass=$((pass + 1)); echo "PASS $CASE"; } +bad() { failed=$((failed + 1)); echo "FAIL $CASE — $1"; echo " exit=$CODE"; printf '%s\n' "$OUT" "$ERR" | sed 's/^/ | /'; } +skip() { skipped=$((skipped + 1)); echo "SKIP $CASE — $1"; } + +expect_exit() { (( CODE == $1 )) || { bad "expected exit $1"; return 1; }; } +expect_has() { # — in stdout or stderr + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" || { bad "missing: $1"; return 1; } +} +expect_not() { + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" && { bad "unexpected: $1"; return 1; } + return 0 +} +expect_no_finding() { # no line names a file or directory of the tree + printf '%s\n' "$OUT" | grep -qE '^[^ ]+: [0-9]+ ' && { bad "unexpected finding"; return 1; } + return 0 +} + +# ---------- fixture builders ---------- +# write_src [ [ []]] — a source +# file with that many code lines, comment-only lines, blank lines and lines of +# one block comment, in that order. +write_src() { + local path="$REPO/$1" code="$2" com="${3:-0}" blank="${4:-0}" block="${5:-0}" i + mkdir -p "$(dirname "$path")" + { + for (( i = 1; i <= code; i++ )); do printf 'v%d = %d\n' "$i" "$i"; done + for (( i = 1; i <= com; i++ )); do printf '%s\n' "$FX_LINE_COMMENT"; done + for (( i = 1; i <= blank; i++ )); do printf '\n'; done + if (( block > 0 )); then + printf '%s\n' "$FX_BLOCK_OPEN" + for (( i = 3; i <= block; i++ )); do printf ' inside the block\n'; done + printf '%s\n' "$FX_BLOCK_CLOSE" + fi + } > "$path" +} + +# fill_dir [] — that many one-line source and test files +fill_dir() { + local i + for (( i = 1; i <= $2; i++ )); do write_src "$1/m$i$FX_EXT" 1; done + for (( i = 1; i <= ${3:-0}; i++ )); do write_src "$(fx_test_file "$1" "$i")" 1; done +} + +# ---------- fixture rows: the Python row this script's language block serves ---------- +# The matrix runs its cases once per row. use_row sets the row's variables and +# the fx_* builders write that language's files; the cases read only these. +# +# FX_ROWS the row ids, space-separated +# FX_NATIVE of the mixed repository's languages (d, go, python, +# typescript), the ones this plugin's block reviews itself +# FX_GLOB find(1) -name pattern for the row's source files +# FX_SRC_FILE the production file under a layer directory: two sleep +# calls and one suppression directive +# FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it +# fx_write_marker write the project marker for a (sub-)project +# fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable +FX_ROWS="python" +FX_NATIVE="python" + +use_row() { + FX_EXT='.py' + FX_LINE_COMMENT='# note' + FX_BLOCK_OPEN='' + FX_BLOCK_CLOSE='' + FX_SKIP_FILE='__init__.py' + FX_GLOB='*.py' + FX_SRC_FILE="services/worker.py" + FX_SUPPRESS_LINE='CACHE: dict[str, str] = {} # noqa: PLW0603' +} + +fx_write_marker() { # + printf '[project]\nname = "%s"\nversion = "0.1.0"\n' "$2" > "$1/pyproject.toml" +} + +fx_write_code() { + mkdir -p "$REPO/services" "$REPO/tests" + cat > "$REPO/services/worker.py" <<'PY' +"""Polling worker.""" + +import time + +CACHE: dict[str, str] = {} # noqa: PLW0603 + + +def run(deadline: float) -> None: + """Poll until the deadline.""" + while time.monotonic() < deadline: + time.sleep(1) + time.sleep(0.05) +PY + cat > "$REPO/tests/test_worker.py" <<'PY' +import time + + +def test_run() -> None: + time.sleep(0.01) +PY +} + +fx_test_file() { printf '%s/test_m%s.py' "$1" "$2"; } + +fx_write_linter_config() { # + printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" +} + +fx_write_linter_config_unreadable() { + printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" +} + +# ========================= cases ========================= +run_row_cases() { # runs every case against the current row + +begin "a file at the limit passes and one line over is a finding" +write_src "src/at$FX_EXT" "$MAX_LINES"; write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" \ + && expect_not "src/at$FX_EXT:" && ok +finish + +begin "comment-only and blank lines are not counted by the plugin's default" +write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 20 20 +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the lines of a block comment are comment-only" +if [[ -z "$FX_BLOCK_OPEN" ]]; then skip "the language has no block comment"; finish +else + write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 0 0 30 + run_lint file-length + expect_exit 0 && expect_no_finding && ok + finish +fi + +begin "a test file over the limit is not measured" +write_src "$(fx_test_file src 1)" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "a generated file over the limit is not measured" +write_src "src/gen$FX_EXT" $((MAX_LINES + 50)) +{ printf '%s DO NOT EDIT\n' "$FX_LINE_COMMENT"; cat "$REPO/src/gen$FX_EXT"; } > "$REPO/src/gen.tmp" && mv "$REPO/src/gen.tmp" "$REPO/src/gen$FX_EXT" +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the plugin's default names the setting that makes the linter hold it" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +finish + +begin "the repository's linter config holds the limit and is counted its way" +fx_write_linter_config 100 +write_src "src/a$FX_EXT" 90 20 +run_lint file-length +expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +finish + +begin "a linter config whose limit cannot be read is named, and the next source applies" +if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish +else + write_src "src/over$FX_EXT" $((MAX_LINES + 1)) + run_lint file-length + expect_exit 0 && expect_has "its limit cannot be read here" \ + && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok + finish +fi + +begin ".ldd-lint.yaml sets the limit when the linter holds none" +printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 101 +run_lint file-length +expect_exit 0 && expect_has "set by .ldd-lint.yaml" && expect_has "src/a$FX_EXT: 101 code lines (limit 100, .ldd-lint.yaml)" && ok +finish + +begin "a .ldd-lint.yaml value that is not a positive integer is inconclusive (exit 2)" +printf 'file_max_lines: lots\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 2 && expect_has "file_max_lines must be a positive integer" && ok +finish + +begin "a directory below the warning size is not a finding" +fill_dir pkg $((WARN_FILES - 1)) 3 +write_src "pkg/$FX_SKIP_FILE" 1 +run_lint package-size +expect_exit 0 && expect_no_finding && ok +finish + +begin "a directory at the warning size is a warning" +fill_dir pkg "$WARN_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $WARN_FILES source files (warning at $WARN_FILES, profile default)" && ok +finish + +begin "a directory at the limit is over the limit" +fill_dir pkg "$MAX_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $MAX_FILES source files (limit $MAX_FILES, profile default)" && ok +finish + +begin ".ldd-lint.yaml sets the directory sizes" +printf 'package_warn_files: 2\npackage_max_files: 3\n' > "$REPO/.ldd-lint.yaml" +fill_dir small 2; fill_dir big 3 +run_lint package-size +expect_exit 0 && expect_has "small: 2 source files (warning at 2, .ldd-lint.yaml)" \ + && expect_has "big: 3 source files (limit 3, .ldd-lint.yaml)" && ok +finish + +begin "--tsv writes path, line 0 and message as rows, and the threshold to stderr" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length --tsv +expect_exit 0 \ + && { [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}$((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" ]] || bad "stdout is not the one row"; } \ + && [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}"* ]] \ + && { [[ "$ERR" == "threshold: "* ]] || bad "threshold line not on stderr"; } && [[ "$ERR" == "threshold: "* ]] && ok +finish + +begin "--files measures only the files it names" +write_src "src/named$FX_EXT" $((MAX_LINES + 1)); write_src "src/other$FX_EXT" $((MAX_LINES + 1)) +printf 'src/named%s (not bundled: %s lines)\n' "$FX_EXT" $((MAX_LINES + 1)) > "$REPO/files.txt" +run_lint file-length --files files.txt +expect_exit 0 && expect_has "src/named$FX_EXT:" && expect_not "src/other$FX_EXT:" && ok +finish + +begin "in a git repository an ignored file is not measured" +write_src "src/kept$FX_EXT" $((MAX_LINES + 1)); write_src "build/out$FX_EXT" $((MAX_LINES + 1)) +printf 'build/\n' > "$REPO/.gitignore" +(cd "$REPO" && git init -q) +run_lint file-length +expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok +finish + +begin "two runs print the same output" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" +run_lint package-size; first="$OUT" +run_lint package-size +expect_exit 0 && { [[ "$OUT" == "$first" ]] || bad "the second run printed something else"; } && [[ "$OUT" == "$first" ]] && ok +finish + +begin "a tree without a source file is measured clean" +printf 'notes\n' > "$REPO/README.md" +run_lint file-length +expect_exit 0 && expect_has "no source file in scope" && ok +finish + +} # run_row_cases + +for ROW in $FX_ROWS; do + use_row "$ROW" + run_row_cases +done +ROW="" + +# --- row-independent --- +begin "a missing subcommand is a usage error (exit 2)" +run_lint +expect_exit 2 && expect_has "name a subcommand" && ok +finish + +begin "a --files list that does not exist is a usage error (exit 2)" +run_lint file-length --files no-such-list.txt +expect_exit 2 && expect_has "no such file list" && ok +finish + +# ========================= summary ========================= +echo +echo "ldd-lint_test: $pass passed, $failed failed, $skipped skipped" +(( failed == 0 )) diff --git a/python-linter-driven-development/scripts/ldd-scope_test.sh b/python-linter-driven-development/scripts/ldd-scope_test.sh index f2c01906..c489d907 100755 --- a/python-linter-driven-development/scripts/ldd-scope_test.sh +++ b/python-linter-driven-development/scripts/ldd-scope_test.sh @@ -110,10 +110,29 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="python" FX_NATIVE="python" use_row() { + FX_EXT='.py' + FX_LINE_COMMENT='# note' + FX_BLOCK_OPEN='' + FX_BLOCK_CLOSE='' + FX_SKIP_FILE='__init__.py' FX_GLOB='*.py' FX_SRC_FILE="services/worker.py" FX_SUPPRESS_LINE='CACHE: dict[str, str] = {} # noqa: PLW0603' @@ -148,6 +167,16 @@ def test_run() -> None: PY } +fx_test_file() { printf '%s/test_m%s.py' "$1" "$2"; } + +fx_write_linter_config() { # + printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" +} + +fx_write_linter_config_unreadable() { + printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" +} + # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/python-linter-driven-development/skills/code-designing/SKILL.md b/python-linter-driven-development/skills/code-designing/SKILL.md index 48eca3c5..d431a8b3 100644 --- a/python-linter-driven-development/skills/code-designing/SKILL.md +++ b/python-linter-driven-development/skills/code-designing/SKILL.md @@ -31,7 +31,7 @@ Backward counterpart (fixing code that already fails lint/review): @refactoring. - Linter failures that need a design decision, not a mechanical fix: - `PLR0913` (too many arguments) → design a frozen dataclass of the parameters that travel together (score it per `../../rules/R1-primitive-obsession.md`); a keyword-only `*` marker tidies the call sites but leaves the clump - `PLR0911` (too many returns), or a tuple return carrying a `bool` or an error member → design a named result type, or raise (same R1 scoring) - - a module over ~450 lines (ruff has no file-length rule — count) → split juicy types into their own modules (juiciness per R1; module-per-type per `../../rules/R5-vertical-slice.md`); a single god class routes to @refactoring's god-object decomposition procedure first + - a module over the file-length limit (pylint's `max-module-lines` where set, else the gap linter's 450 code lines; ruff has no file-length rule) → split juicy types into their own modules (juiciness per R1; module-per-type per `../../rules/R5-vertical-slice.md`); a single god class routes to @refactoring's god-object decomposition procedure first - Package-size yellow/red zone → re-model with sub-packages *before* the zone escalates (`` in @refactoring's `reference.md` (`sed -n '/^/,/^<\/package_decomposition>/p; /^### Package decomposition/,$p'`)) - A Phase 4 review CLUSTER (≥2 hunters converging on one anchor — @linter-driven-development routes it here) → **cluster-scoped mode**: skip diff --git a/python-linter-driven-development/skills/refactoring/SKILL.md b/python-linter-driven-development/skills/refactoring/SKILL.md index b3eb8b0a..63e69cd7 100644 --- a/python-linter-driven-development/skills/refactoring/SKILL.md +++ b/python-linter-driven-development/skills/refactoring/SKILL.md @@ -49,7 +49,7 @@ findings the hunters own alone. | Duplicated code (review-only) | `../../rules/R1-primitive-obsession.md` (extract shared type/logic); duplicated `match`/if-chains on the same kind/type discriminator → `../../rules/R11-conditional-dispatch.md` | | A `match` missing enum cases (review-only; ty and mypy see it only under `assert_never`) | `../../rules/R11-conditional-dispatch.md` — handle the case at the single dispatch site and close it with `case _: assert_never(x)`; a second `match` appearing is the R11 violation itself | | `FBT001` / `FBT002` (boolean positional parameter) | Mechanical — make it keyword-only; then read the branches: sharing little → `../../rules/R11-conditional-dispatch.md`, Split Flag Argument | -| File over ~450 lines (review-only) | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | +| File over the file-length limit (`scripts/ldd-lint.sh file-length`) | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | | `PLW0603` (`global` statement); import-time side effects (review-only) | `../../rules/R8-no-globals.md` | | A `Protocol`/ABC with one implementer; a factory annotated to return the Protocol (review-only) | `../../rules/R6-test-only-interfaces.md` | | Unguarded shared state; a `daemon=True` worker; `time.sleep` in a stoppable loop (review-only) | `../../rules/R10-concurrency-safety.md` | diff --git a/python-linter-driven-development/skills/refactoring/reference.md b/python-linter-driven-development/skills/refactoring/reference.md index d0385b65..925e3a99 100644 --- a/python-linter-driven-development/skills/refactoring/reference.md +++ b/python-linter-driven-development/skills/refactoring/reference.md @@ -70,7 +70,7 @@ decomposition): "Multi-rule procedures", below. ## File and package routing -**A module over ~450 lines (ruff has no file-length rule — count):** +**A module over the file-length limit** — pylint's `max-module-lines` where the repository sets it (ruff has no file-length rule), else 450 code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: | File pattern | Action | |---|---| @@ -79,15 +79,20 @@ decomposition): "Multi-rule procedures", below. | Long functions, few types | Storify → extract functions (R3) | -**Package-size zones** — count non-test `.py` modules per package directory: +**Package-size zones** — non-test `.py` modules per package directory, `__init__.py`, `conftest.py` and generated stubs left out: ``` -find -maxdepth 1 -type f -name '*.py' -not -name 'test_*.py' -not -name '*_test.py' -not -name 'conftest.py' -not -name '__init__.py' -not -name '*_pb2.py' -not -name '*_pb2_grpc.py' | wc -l +bash /../../scripts/ldd-lint.sh package-size --root . ``` -≤7 green — fine. 8–12 yellow — design review *before the next file lands*. ≥13 red — -**must decompose**. Either zone: run the 3-step design review under "Package -decomposition", printed with this range (it is a *design* review — missing domain types are the +The gap linter counts the non-test, non-generated source files of each directory; +a `.ldd-lint.yaml` at the repository root sets other sizes, and the first line of +its output says which sizes applied. + +Below 8 files green — fine. 8 or more yellow — design review +*before the next file lands*. 13 or more red — **must decompose**. Either +zone: run the 3-step design review under "Package decomposition", printed with this +range (it is a *design* review — missing domain types are the disease, file count the symptom). Invoke @code-designing to validate extracted types. @@ -269,9 +274,9 @@ composition: @code-designing. ### Package decomposition -Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone (≥13 non-test `.py` -files at one directory level) or yellow zone (8–12) — detection command and zone -table in `` above. +Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone (13 or more non-test +`.py` files at one directory level) or yellow zone (8 or more) — the +measuring command and the zones are in `` above. **A package-size violation is a design review, not a mechanical file split.** File count is the symptom; the disease is usually missing domain types or multiple diff --git a/tools/ldd-gen/internal/binding/binding_test.go b/tools/ldd-gen/internal/binding/binding_test.go index 29686d71..85e2573e 100644 --- a/tools/ldd-gen/internal/binding/binding_test.go +++ b/tools/ldd-gen/internal/binding/binding_test.go @@ -28,6 +28,9 @@ doc_form: docstring doc_comment: doc comment src_ext: .p unexported: hidden +file_max_lines: "450" +package_warn_files: "8" +package_max_files: "13" ` func langFS() fstest.MapFS { diff --git a/tools/ldd-gen/internal/gen/gen_test.go b/tools/ldd-gen/internal/gen/gen_test.go index 3109b254..5d190a9f 100644 --- a/tools/ldd-gen/internal/gen/gen_test.go +++ b/tools/ldd-gen/internal/gen/gen_test.go @@ -30,6 +30,9 @@ doc_form: docstring doc_comment: doc comment src_ext: .p unexported: hidden +file_max_lines: "450" +package_warn_files: "8" +package_max_files: "13" ignore: ["evals/*"] ` diff --git a/tools/ldd-gen/internal/profile/profile.go b/tools/ldd-gen/internal/profile/profile.go index 97cd5199..242bd24b 100644 --- a/tools/ldd-gen/internal/profile/profile.go +++ b/tools/ldd-gen/internal/profile/profile.go @@ -10,6 +10,7 @@ import ( "path" "reflect" "regexp" + "strconv" "strings" "gopkg.in/yaml.v3" @@ -40,6 +41,13 @@ type Vars struct { DocComment string `yaml:"doc_comment"` SrcExt string `yaml:"src_ext"` Unexported string `yaml:"unexported"` + // The thresholds the gap linter applies where the repository's linter + // holds none: code lines per source file, and source files per directory + // that start the warning zone and the limit. Each is a positive integer, + // and the warning starts below the limit. + FileMaxLines string `yaml:"file_max_lines"` + PackageWarnFiles string `yaml:"package_warn_files"` + PackageMaxFiles string `yaml:"package_max_files"` } // Profile is a parsed profile.yaml. Plugin doubles as the output directory @@ -175,6 +183,29 @@ func (v Vars) validate() error { if len(missing) > 0 { return errors.New("profile: missing " + strings.Join(missing, ", ")) } + return v.validateThresholds() +} + +// validateThresholds accepts positive integers only, with the package +// warning below the package limit: the scripts compare them as numbers, and +// a warning zone that starts at or above the limit is empty. +func (v Vars) validateThresholds() error { + values := map[string]string{ + "file_max_lines": v.FileMaxLines, + "package_warn_files": v.PackageWarnFiles, + "package_max_files": v.PackageMaxFiles, + } + numbers := make(map[string]int, len(values)) + for _, key := range []string{"file_max_lines", "package_warn_files", "package_max_files"} { + n, err := strconv.Atoi(values[key]) + if err != nil || n <= 0 { + return fmt.Errorf("profile: %s %q must be a positive integer", key, values[key]) + } + numbers[key] = n + } + if numbers["package_warn_files"] >= numbers["package_max_files"] { + return fmt.Errorf("profile: package_warn_files %d must be below package_max_files %d", numbers["package_warn_files"], numbers["package_max_files"]) + } return nil } diff --git a/tools/ldd-gen/internal/profile/profile_test.go b/tools/ldd-gen/internal/profile/profile_test.go index 5809bf06..a3ae7687 100644 --- a/tools/ldd-gen/internal/profile/profile_test.go +++ b/tools/ldd-gen/internal/profile/profile_test.go @@ -27,6 +27,9 @@ doc_form: godoc doc_comment: godoc comment src_ext: .go unexported: unexported +file_max_lines: "450" +package_warn_files: "8" +package_max_files: "13" ignore: - evals/* - .DS_Store @@ -83,6 +86,9 @@ func TestParse_Errors(t *testing.T) { withPrefix := func(prefix string) string { return strings.Replace(full, "cmd_prefix: go-ldd", "cmd_prefix: "+prefix, 1) } + withScalar := func(line, replacement string) string { + return strings.Replace(full, line, replacement, 1) + } cases := []struct { name string input string @@ -109,6 +115,10 @@ func TestParse_Errors(t *testing.T) { {name: "include_fallback without under", input: full + "include_fallback: {from: go}\n", want: "include_fallback.under"}, {name: "include_fallback.under without slash", input: full + "include_fallback: {from: go, under: examples}\n", want: "include_fallback.under"}, {name: "include_fallback.under escapes", input: full + "include_fallback: {from: go, under: ../examples/}\n", want: "include_fallback.under"}, + {name: "file_max_lines not a number", input: withScalar(`file_max_lines: "450"`, `file_max_lines: "lots"`), want: `file_max_lines "lots" must be a positive integer`}, + {name: "file_max_lines zero", input: withScalar(`file_max_lines: "450"`, `file_max_lines: "0"`), want: "file_max_lines"}, + {name: "package_max_files negative", input: withScalar(`package_max_files: "13"`, `package_max_files: "-1"`), want: "package_max_files"}, + {name: "package warning at the limit", input: withScalar(`package_warn_files: "8"`, `package_warn_files: "13"`), want: "package_warn_files 13 must be below package_max_files 13"}, {name: "include_fallback.under absolute", input: full + "include_fallback: {from: go, under: /examples/}\n", want: "include_fallback.under"}, } for _, tc := range cases { diff --git a/tools/ldd-gen/internal/render/render_test.go b/tools/ldd-gen/internal/render/render_test.go index a64f3aeb..b6a9621d 100644 --- a/tools/ldd-gen/internal/render/render_test.go +++ b/tools/ldd-gen/internal/render/render_test.go @@ -33,6 +33,9 @@ doc_form: docstring doc_comment: doc comment src_ext: .p unexported: hidden +file_max_lines: "450" +package_warn_files: "8" +package_max_files: "13" ignore: [".DS_Store", "__pycache__"] ` diff --git a/tools/ldd-gen/main_test.go b/tools/ldd-gen/main_test.go index 08c4e880..28ae8a41 100644 --- a/tools/ldd-gen/main_test.go +++ b/tools/ldd-gen/main_test.go @@ -27,6 +27,9 @@ doc_form: docstring doc_comment: doc comment src_ext: .p unexported: hidden +file_max_lines: "450" +package_warn_files: "8" +package_max_files: "13" ` func write(t *testing.T, root, rel, content string) { diff --git a/ts-react-linter-driven-development/CHANGELOG.md b/ts-react-linter-driven-development/CHANGELOG.md index ddc0c6eb..cad904eb 100644 --- a/ts-react-linter-driven-development/CHANGELOG.md +++ b/ts-react-linter-driven-development/CHANGELOG.md @@ -5,6 +5,26 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio ## [Unreleased] +### Added + +- **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` + measures lines per source file (`file-length`) and source files per directory + (`package-size`). Where the repository's linter config holds the file-length + limit, the script uses that limit and counts lines the way that linter counts. + Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and + otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 + source files per directory and the limit at 13. The first line of the output + names the limit that applied; with the defaults it also prints the setting that + makes the repository's linter hold the limit. `--tsv` writes findings as rows. + The script never edits a file. + +### Changed + +- **One set of thresholds.** The skills and routing tables name the same limits the + gap linter applies: the repository linter's limit where it sets one, else + 450 code lines per file and 8 and 13 files per directory. The package-size + procedure runs the gap linter in place of a hand-written `find` command. + ### Fixed - **A claim that something is missing needs a listing.** A rule hunter that reports diff --git a/ts-react-linter-driven-development/agents/lint-fixer.md b/ts-react-linter-driven-development/agents/lint-fixer.md index 602965d0..2669020c 100644 --- a/ts-react-linter-driven-development/agents/lint-fixer.md +++ b/ts-react-linter-driven-development/agents/lint-fixer.md @@ -75,7 +75,7 @@ this table: | `sonarjs/no-duplicate-string` on an enum-shaped literal / `no-magic-numbers` on a domain value / `sonarjs/max-union-size` | rules/R1-primitive-obsession.md ("Name enum strings" move, or a named type) | | A tuple return (`[T, boolean]`, `[Data, Error]`); a `Record`, `string[]` or `Map` of primitives crossing a function boundary or nested in a type (review-only — no ESLint rule) | rules/R1-primitive-obsession.md ("Name the Container" move — Q7 nested, Q8 flat across a boundary) | | `sonarjs/no-identical-functions` | rules/R1-primitive-obsession.md (extract shared type/logic); duplicated `switch`/if-chains on one discriminant → rules/R11-conditional-dispatch.md | -| `sonarjs/max-lines` (600) | rules/R5-vertical-slice.md | +| `sonarjs/max-lines`; `scripts/ldd-lint.sh file-length` and `package-size` findings | rules/R5-vertical-slice.md | | `react/no-multi-comp` | rules/R5-vertical-slice.md (one component per file; the page folder decides where the second goes) | | `@typescript-eslint/no-explicit-any` / `no-unsafe-*` / `no-non-null-assertion` / `no-unnecessary-condition` at a boundary | rules/R2-self-validating-types.md (parse at the boundary — a guard, not an assertion) | | `@typescript-eslint/switch-exhaustiveness-check` / `sonarjs/no-nested-switch` / `sonarjs/max-switch-cases` / `sonarjs/no-small-switch` | rules/R11-conditional-dispatch.md (via @refactoring) | diff --git a/ts-react-linter-driven-development/rules/R5-vertical-slice.md b/ts-react-linter-driven-development/rules/R5-vertical-slice.md index 4c9d43fc..43b08235 100644 --- a/ts-react-linter-driven-development/rules/R5-vertical-slice.md +++ b/ts-react-linter-driven-development/rules/R5-vertical-slice.md @@ -142,7 +142,7 @@ Answer each with evidence (`file:line`, command output) — never a bare verdict Detection: for each feature noun in the diff, `grep -rli '' --include='*.ts' --include='*.tsx' --exclude-dir=node_modules src | xargs -n1 dirname | sort -u` — count distinct layer-named directories. The inverse shape is the god file: - `sonarjs/max-lines` (600) on a `Performance.tsx` of 1,199 lines holding the page, + `sonarjs/max-lines` on a `Performance.tsx` of 1,199 lines holding the page, its hooks, its API calls and its types in one module. Violation: the same feature living in `components/`, `hooks/`, `services/` and `types/` — four top-level directories for one page — it is horizontally diff --git a/ts-react-linter-driven-development/scripts/ldd-detect_test.sh b/ts-react-linter-driven-development/scripts/ldd-detect_test.sh index 5fe08220..375f923b 100644 --- a/ts-react-linter-driven-development/scripts/ldd-detect_test.sh +++ b/ts-react-linter-driven-development/scripts/ldd-detect_test.sh @@ -134,6 +134,20 @@ questions_in_rules() { # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="typescript" FX_NATIVE="typescript" @@ -141,6 +155,11 @@ FX_NATIVE="typescript" # names for the files it creates from it, and `fresh.ts*` is no file name. # Every file the row writes is a `.ts` file, so both globs select the same set. use_row() { + FX_EXT='.ts' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='index.ts' FX_GLOB='*.ts' FX_SRC_FILE="services/worker.ts" FX_SUPPRESS_LINE='// eslint-disable-next-line import/no-mutable-exports' @@ -175,6 +194,17 @@ it('runs', async () => { TS } +fx_test_file() { printf '%s/m%s.test.ts' "$1" "$2"; } + +# ESLint's own max-lines counts every line unless told to skip comments. +fx_write_linter_config() { # + printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +} + +# Every ESLint config form resolves: a rule named without a limit takes +# ESLint's documented default. +fx_write_linter_config_unreadable() { return 1; } + # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/ts-react-linter-driven-development/scripts/ldd-lint.sh b/ts-react-linter-driven-development/scripts/ldd-lint.sh new file mode 100755 index 00000000..a26ea48b --- /dev/null +++ b/ts-react-linter-driven-development/scripts/ldd-lint.sh @@ -0,0 +1,531 @@ +#!/usr/bin/env bash +# Gap linter of the ts-react-linter-driven-development plugin. +# +# Measures the design triggers no linter holds, or that the repository's linter +# does not hold yet: lines per source file and source files per directory. It +# never edits a file. Where the repository's linter holds a limit, that limit +# is the one applied, counted the way that linter counts. +# +# Usage: bash scripts/ldd-lint.sh [options] +# file-length source files over the file-length limit +# package-size directories at or over the warning size or the limit +# --root the repository (default: cwd) +# --files measure the source files this list names, one +# relative path per line (a scope bundle's files.txt +# works: a "(not bundled: …)" suffix is ignored); +# default: every source file of the repository +# --tsv findings as rows: path TAB line TAB message, line 0; +# the threshold lines then go to stderr +# --glob source-file glob, for a language the table does not +# know (TypeScript + React block below) +# --test-re test-file pattern over relative paths +# +# Thresholds, per language, first that applies — the "threshold:" line says +# which one a run used: +# 1. the repository's linter config holds the rule: its limit, counted the +# way that linter counts (file-length only; no linter counts files per +# directory); a config the table cannot read is named, then skipped +# 2. .ldd-lint.yaml at the repository root: file_max_lines, +# package_warn_files, package_max_files, each a positive integer +# 3. the plugin's defaults: 450 code lines per file (blank and +# comment-only lines not counted); a warning at 8 source files per +# directory and the limit at 13; for file-length, the line also +# prints the setting that makes the repository's linter hold the limit +# +# What is measured: the language's source files, minus test files, generated +# files and the directories the language block excludes. A directory count +# also leaves out the files the linter table names (an index module, a package +# marker, generated stubs). A file is over the limit when its count is above +# it; a directory is in the warning zone at the warning size and over the +# limit at the limit. +# +# Exit codes: 0 measured, with or without findings · 2 usage error, a --files +# list that does not exist, or a .ldd-lint.yaml value that is not a +# positive integer (inconclusive, never silently clean) +# +# Uses only POSIX-portable tools plus git when the root is a repository: +# grep, sed, awk, sort, find, head. + +set -u + +SCRIPT_NAME="ldd-lint" +DEFAULT_FILE_MAX_LINES=450 +DEFAULT_PACKAGE_WARN_FILES=8 +DEFAULT_PACKAGE_MAX_FILES=13 +OVERRIDE_FILE=".ldd-lint.yaml" +ROOT="$(pwd)" +FILES_LIST="" +TSV=0 +OPT_GLOB="" +OPT_TEST_RE="" +SUBCOMMAND="" + +usage() { sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//' >&2; exit 2; } +die() { echo "$SCRIPT_NAME: $*" >&2; exit 2; } + +while (( $# > 0 )); do + case "$1" in + file-length|package-size) [[ -n "$SUBCOMMAND" ]] && die "one subcommand, got also: $1"; SUBCOMMAND="$1"; shift ;; + --root) ROOT="${2:-}"; shift 2 ;; + --files) FILES_LIST="${2:-}"; shift 2 ;; + --tsv) TSV=1; shift ;; + --glob) OPT_GLOB="${2:-}"; shift 2 ;; + --test-re) OPT_TEST_RE="${2:-}"; shift 2 ;; + -h|--help) usage ;; + *) die "unknown argument: $1 (see --help)" ;; + esac +done +[[ -n "$SUBCOMMAND" ]] || die "name a subcommand: file-length or package-size (see --help)" +[[ -d "$ROOT" ]] || die "not a directory: $ROOT" +if [[ -n "$FILES_LIST" ]]; then + [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" + FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") +fi +cd "$ROOT" || exit 2 + +# ==================== language table ==================== +# How a file's language is told, and which plugin owns a language. The same +# table is rendered into every plugin, so two plugins that split one scope +# agree on every file. The language block below says which of these ids this +# plugin reviews itself; the rest are routed to the plugin that owns them. +# residue-exempt: the table names every language's extension and plugin on +# purpose, the same in every rendering. +# +# lang_of_path the language id of a source file, by extension: +# "custom" for a file matching --glob (then nothing +# else is a source file), "" for a file that is not +# source (a manifest, a doc, a build file) +# dedicated_plugin_for the plugin written for that language, or "" +# GENERIC_PLUGIN the plugin that reviews any language +# installed_plugin_dir where plugin is installed, from the +# installed-plugins file, or "" when it is not; +# LDD_INSTALLED_PLUGINS names another file +GENERIC_PLUGIN='linter-driven-development' + +# A .h header is C++ when the repository has C++ sources, C otherwise. +HEADER_LANG="" +header_lang() { + [[ -n "$HEADER_LANG" ]] && { printf '%s' "$HEADER_LANG"; return; } + if { git ls-files -- '*.cpp' '*.cc' '*.cxx' '*.hpp' 2>/dev/null || find . -type f \( -name '*.cpp' -o -name '*.cc' -o -name '*.cxx' -o -name '*.hpp' \) -not -path './.git/*' 2>/dev/null; } | head -1 | grep -q .; then + HEADER_LANG=cpp + else + HEADER_LANG=c + fi + printf '%s' "$HEADER_LANG" +} + +lang_of_path() { + local b="${1##*/}" + if [[ -n "${OPT_GLOB:-}" ]]; then + case "$b" in $OPT_GLOB) printf 'custom' ;; esac + return 0 + fi + case "$b" in + *.go) printf 'go' ;; + *.py) printf 'python' ;; + *.d|*.di) printf 'd' ;; + *.rs) printf 'rust' ;; + *.c) printf 'c' ;; + *.h) header_lang ;; + *.cpp|*.cc|*.cxx|*.hpp|*.hh|*.hxx) printf 'cpp' ;; + *.java) printf 'java' ;; + *.kt|*.kts) printf 'kotlin' ;; + *.rb) printf 'ruby' ;; + *.sh|*.bash) printf 'shell' ;; + *.ts|*.tsx) printf 'typescript' ;; + *.js|*.jsx|*.mjs|*.cjs) printf 'javascript' ;; + *.cs) printf 'csharp' ;; + esac + return 0 +} + +dedicated_plugin_for() { + case "$1" in + go|python) printf '%s-linter-driven-development' "$1" ;; + typescript|javascript) printf 'ts-react-linter-driven-development' ;; + esac + return 0 +} + +installed_plugin_dir() { + local f="${LDD_INSTALLED_PLUGINS:-$HOME/.claude/plugins/installed_plugins.json}" + [[ -f "$f" ]] || return 0 + awk -v key="\"$1@" ' + index($0, key) { found = 1; next } + found && /"installPath"/ { sub(/^[^:]*:[ \t]*"/, ""); sub(/".*$/, ""); print; exit } + ' "$f" +} +# ================== end language table ================== + +# ===================== language block: ts-react ===================== +# Everything language-specific the review scripts need. The driver calls only +# the LANG_* variables and lang_* functions defined here; a build of the script +# for another language replaces this block and nothing else. Which file is +# which language is the language table above, shared by every plugin. +# +# Contract: +# LANG_NATIVE the language ids this plugin reviews itself; a group of +# any other id is routed to the plugin that owns it +# lang_configure sets the variables below for that id; "custom" is the +# --glob override, reviewed with this block's defaults; +# exit 2 for an id this block has no row for +# LANG_EXCLUDE_RE ERE over a relative path: directories never in scope +# LANG_SUPPRESS_RE ERE: a lint-suppression directive on a source line +# LANG_COMMENT_RE ERE: a source line that carries a comment +# (bracket expressions, no backslashes: the driver hands it to +# awk -v, which eats `\*` and `\s` under gawk) +# LANG_DIRECTIVE_RE ERE: a comment line that is a directive, not prose +# LANG_GENERATED_RE ERE: a marker in a file's head that says it is generated +# lang_is_test exit 0 iff the path is a test file (--test-re overrides) +# +# The table sends `.ts` and `.tsx` here as typescript and `.js`, `.jsx`, `.mjs` +# and `.cjs` as javascript; both are reviewed with the same row. Declaration +# files, the `*.generated.ts(x)` modules a code generator writes, and build +# output are dropped by the exclude pattern. +LANG_NATIVE="typescript javascript custom" + +lang_configure() { + case "$1" in typescript|javascript|custom) ;; *) return 2 ;; esac + LANG_EXCLUDE_RE='(^|/)(node_modules|dist|build|\.next|coverage|\.git|storybook-static)/|\.d\.ts$|\.generated\.[jt]sx?$' + LANG_SUPPRESS_RE='eslint-disable(-next-line|-line)?|@ts-(expect-error|ignore|nocheck)|prettier-ignore' + LANG_COMMENT_RE='//|/[*]|^[[:space:]]*[*]' + LANG_DIRECTIVE_RE='eslint-(disable|enable)|@ts-(expect-error|ignore|nocheck|check)|prettier-ignore|/// c (// and /* */), d (also /+ +/) or hash (#) +# lint_package_skip_re ERE over a relative path: source files a +# directory count leaves out (an index module, +# a package marker, generated stubs); "" for none +# lint_file_length_config one line, from the repository root: +# covered TAB TAB TAB +# TAB +# unreadable TAB +# none +# lint_file_length_snippet the setting that makes the repository's +# linter hold the limit, or how to set it when +# no linter rule is known + +lint_comment_style() { + case "$1" in + python|ruby|shell) printf 'hash' ;; + d) printf 'd' ;; + *) printf 'c' ;; + esac +} + +lint_package_skip_re() { + case "$1" in + go) printf '%s' '(_gen|\.pb)\.go$' ;; + python) printf '%s' '(^|/)(__init__|conftest)\.py$|_pb2(_grpc)?\.py$' ;; + typescript|javascript) printf '%s' '(^|/)index\.[cm]?[jt]sx?$' ;; + esac + return 0 +} + +# _lint_window — the key's line and the lines after it, up to +# the next rule entry or 15 lines, joined by spaces; one window per match. The +# patterns reach awk as strings, so they use bracket expressions, never +# backslash escapes, which awk -v would rewrite. +_lint_window() { + awk -v key="$1" -v q="'" ' + BEGIN { + entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" + qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" + } + function flush() { if (w != "") print w; w = ""; n = 0 } + { + if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() + if ($0 ~ key) { flush(); n = 1; w = $0; next } + if (n > 0) { w = w " " $0; if (++n > 15) flush() } + } + END { flush() } + ' "$2" +} + +# _lint_number — the first match's digits +_lint_number() { + printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' +} + +# _lint_off — exit 0 when the rule in the window is switched off +_lint_off() { + printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" +} + +_lint_config_go() { + local f w max sc sb unread="" + for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do + [[ -f "$f" ]] || continue + grep -q 'file-length-limit' "$f" 2>/dev/null || continue + case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac + while IFS= read -r w; do + [[ "$w" =~ disabled:[[:space:]]*true ]] && continue + max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") + if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi + (( max > 0 )) || continue + sc=0; sb=0 + [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +_lint_config_eslint() { + local f w max sc sb + for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do + [[ -f "$f" ]] || continue + grep -q 'max-lines' "$f" 2>/dev/null || continue + # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") + printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" + return 0 + done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") + # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. + while IFS= read -r w; do + w="${w%%]*}" + _lint_off "$w" && continue + max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") + [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") + sc=0; sb=0 + [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 + [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 + printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" + return 0 + done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") + done + printf 'none\n' +} + +_lint_config_python() { + local f line max unread="" + for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do + [[ -f "$f" ]] || continue + line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) + [[ -n "$line" ]] || continue + max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") + if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi + (( max > 0 )) || continue + # pylint counts every line of the module. + printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" + return 0 + done + [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } + printf 'none\n' +} + +lint_file_length_config() { + case "$1" in + go) _lint_config_go ;; + typescript|javascript) _lint_config_eslint ;; + python) _lint_config_python ;; + *) printf 'none\n' ;; + esac +} + +lint_file_length_snippet() { + case "$1" in + go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; + typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; + python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; + *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + esac +} +# ================== end linter table ================== + +# ---------- output ---------- +# A threshold line is for the reader of the report; with --tsv the rows are +# the output and the threshold lines go to stderr. +say() { if (( TSV )); then printf '%s\n' "$*" >&2; else printf '%s\n' "$*"; fi; } +finding() { # + if (( TSV )); then printf '%s\t0\t%s\n' "$1" "$2"; else printf '%s: %s\n' "$1" "$2"; fi +} + +# ---------- the repository override ---------- +# override — the key's value in .ldd-lint.yaml, "" when absent; a value +# that is not a positive integer ends the run, because the file is the +# repository's own statement of its limits. +override() { + [[ -f "$OVERRIDE_FILE" ]] || return 0 + local v + v=$(awk -v k="$1" -v q="'" ' + { line = $0; sub(/#.*/, "", line) } + line ~ "^[[:space:]]*" k "[[:space:]]*:" { + sub("^[[:space:]]*" k "[[:space:]]*:[[:space:]]*", "", line) + gsub("[\"" q "[:space:]]", "", line) + print line; exit + }' "$OVERRIDE_FILE") + [[ -z "$v" ]] && return 0 + [[ "$v" =~ ^[0-9]+$ ]] && (( v > 0 )) || die "$OVERRIDE_FILE: $1 must be a positive integer, got: $v" + printf '%s' "$v" +} + +# ---------- the scope ---------- +# One "id TAB path" line per source file of a language this plugin reviews, +# sorted, so two runs over one tree print the same thing. +list_candidates() { + if [[ -n "$FILES_LIST" ]]; then + sed -e 's/ (not bundled:.*$//' -e 's|^\./||' "$FILES_LIST" + elif git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + git ls-files -co --exclude-standard + else + find . -type f -not -path './.git/*' | sed 's|^\./||' + fi +} + +SCOPE=$(mktemp) +trap 'rm -f "$SCOPE"' EXIT +while IFS= read -r f; do + [[ -n "$f" && -f "$f" ]] || continue + id=$(lang_of_path "$f") + [[ -n "$id" ]] || continue + case " $LANG_NATIVE " in *" $id "*) ;; *) continue ;; esac + printf '%s\t%s\n' "$id" "$f" +done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" + +# measured — exit 0 when the file counts for this language: not under +# an excluded directory, not a test, not generated. +measured() { + printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 + lang_is_test "$1" && return 1 + head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + return 0 +} + +# ---------- file-length ---------- +# COUNT_AWK prints a file's line count under the style's comment syntax: +# skipc drops comment-only lines (block comments included), skipb blank lines. +COUNT_AWK=' +function comment_only(t, rest, i) { + if (inblock) { + i = index(t, closer) + if (i == 0) return 1 + inblock = 0 + rest = substr(t, i + length(closer)); sub(/^[[:space:]]+/, "", rest) + return rest == "" || comment_only(rest) + } + if (style == "hash") return substr(t, 1, 1) == "#" + if (substr(t, 1, 2) == "//") return 1 + if (substr(t, 1, 2) == "/*" || (style == "d" && substr(t, 1, 2) == "/+")) { + closer = (substr(t, 2, 1) == "+") ? "+/" : "*/" + inblock = 1 + return comment_only(substr(t, 3)) + } + return 0 +} +{ + t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) + if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } + c = comment_only(t) + if (c && skipc) next + n++ +} +END { print n + 0 } +' + +run_file_length() { # — every measured file of that language + local id="$1" cfg status max sc sb label src unit f n o + cfg=$(lint_file_length_config "$id") + IFS=$'\t' read -r status max sc sb label <<< "$cfg" + if [[ "$status" == "unreadable" ]]; then + # an unreadable row carries the rule and file in its second field + say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" + status=none + fi + if [[ "$status" == "covered" ]]; then + src="$label" + say "threshold: $id file length — covered by $label (limit $max)" + else + sc=1; sb=1 + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" + else + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + fi + fi + unit="lines" + (( sc && sb )) && unit="code lines" + (( sc && ! sb )) && unit="lines without comments" + (( ! sc && sb )) && unit="non-blank lines" + while IFS=$'\t' read -r _ f; do + measured "$f" || continue + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") +} + +# ---------- package-size ---------- +run_package_size() { # — every directory that holds a measured file of that language + local id="$1" warn lim src ow om skip d n f + ow=$(override package_warn_files) || exit 2 + om=$(override package_max_files) || exit 2 + warn="${ow:-$DEFAULT_PACKAGE_WARN_FILES}"; lim="${om:-$DEFAULT_PACKAGE_MAX_FILES}" + (( warn < lim )) || die "$OVERRIDE_FILE: package_warn_files ($warn) must be below package_max_files ($lim)" + if [[ -n "$ow$om" ]]; then src="$OVERRIDE_FILE"; else src="profile default"; fi + say "threshold: $id package size — $src (warning at $warn source files per directory, limit $lim); no linter counts files per directory" + skip=$(lint_package_skip_re "$id") + while IFS= read -r d; do + n=0 + while IFS= read -r f; do + f="${f#./}" + [[ "$(lang_of_path "$f")" == "$id" ]] || continue + [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue + measured "$f" || continue + n=$((n + 1)) + done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + if (( n >= lim )); then + finding "$d" "$n source files (limit $lim, $src)" + elif (( n >= warn )); then + finding "$d" "$n source files (warning at $warn, $src)" + fi + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) +} + +# ---------- run ---------- +ids=$(cut -f1 "$SCOPE" | LC_ALL=C sort -u) +if [[ -z "$ids" ]]; then + say "$SCRIPT_NAME: no source file in scope" + exit 0 +fi +for id in $ids; do + lang_configure "$id" || continue + case "$SUBCOMMAND" in + file-length) run_file_length "$id" ;; + package-size) run_package_size "$id" ;; + esac +done +exit 0 diff --git a/ts-react-linter-driven-development/scripts/ldd-lint_test.sh b/ts-react-linter-driven-development/scripts/ldd-lint_test.sh new file mode 100644 index 00000000..36b35bac --- /dev/null +++ b/ts-react-linter-driven-development/scripts/ldd-lint_test.sh @@ -0,0 +1,333 @@ +#!/usr/bin/env bash +# Fixture matrix for ldd-lint.sh — the gap linter's own conformance test. +# +# Each case builds a throwaway tree under a temp dir, runs the script against +# it and asserts the exit code and the lines it prints. The cases are the same +# for every language; the language-specific parts — the source suffix, how a +# comment and a test file are written, the file a directory count leaves out, +# and the repository linter config that holds a file-length limit — come from +# the fixture rows spliced in below. A script whose language block serves +# several languages runs the whole matrix once per row. +# +# Usage: bash scripts/ldd-lint_test.sh [path/to/ldd-lint.sh] +# (default: the sibling ldd-lint.sh) +# Exit: 0 all cases pass · 1 any failure (details on stdout) +# +# Uses only POSIX-portable tools, like the script under test. + +set -u + +HERE=$(cd "$(dirname "$0")" && pwd) +LINT="${1:-$HERE/ldd-lint.sh}" +[[ -f "$LINT" ]] || { echo "no such script: $LINT" >&2; exit 2; } +LINT=$(cd "$(dirname "$LINT")" && pwd)/$(basename "$LINT") +MAX_LINES=450 +WARN_FILES=8 +MAX_FILES=13 + +pass=0 failed=0 skipped=0 +CASE="" +OUT="" ERR="" CODE=0 +ROW="" +REPO="" +TAB=$(printf '\t') + +# ---------- harness ---------- +begin() { CASE="$1"; [[ -n "$ROW" ]] && CASE="[$ROW] $1"; REPO=$(mktemp -d); } +finish() { rm -rf "$REPO"; } + +run_lint() { # [options...] — runs the script over $REPO + local errf; errf=$(mktemp) + OUT=$(cd "$REPO" && bash "$LINT" "$@" --root . 2>"$errf"); CODE=$? + ERR=$(cat "$errf"); rm -f "$errf" +} + +ok() { pass=$((pass + 1)); echo "PASS $CASE"; } +bad() { failed=$((failed + 1)); echo "FAIL $CASE — $1"; echo " exit=$CODE"; printf '%s\n' "$OUT" "$ERR" | sed 's/^/ | /'; } +skip() { skipped=$((skipped + 1)); echo "SKIP $CASE — $1"; } + +expect_exit() { (( CODE == $1 )) || { bad "expected exit $1"; return 1; }; } +expect_has() { # — in stdout or stderr + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" || { bad "missing: $1"; return 1; } +} +expect_not() { + printf '%s\n%s\n' "$OUT" "$ERR" | grep -qF -- "$1" && { bad "unexpected: $1"; return 1; } + return 0 +} +expect_no_finding() { # no line names a file or directory of the tree + printf '%s\n' "$OUT" | grep -qE '^[^ ]+: [0-9]+ ' && { bad "unexpected finding"; return 1; } + return 0 +} + +# ---------- fixture builders ---------- +# write_src [ [ []]] — a source +# file with that many code lines, comment-only lines, blank lines and lines of +# one block comment, in that order. +write_src() { + local path="$REPO/$1" code="$2" com="${3:-0}" blank="${4:-0}" block="${5:-0}" i + mkdir -p "$(dirname "$path")" + { + for (( i = 1; i <= code; i++ )); do printf 'v%d = %d\n' "$i" "$i"; done + for (( i = 1; i <= com; i++ )); do printf '%s\n' "$FX_LINE_COMMENT"; done + for (( i = 1; i <= blank; i++ )); do printf '\n'; done + if (( block > 0 )); then + printf '%s\n' "$FX_BLOCK_OPEN" + for (( i = 3; i <= block; i++ )); do printf ' inside the block\n'; done + printf '%s\n' "$FX_BLOCK_CLOSE" + fi + } > "$path" +} + +# fill_dir [] — that many one-line source and test files +fill_dir() { + local i + for (( i = 1; i <= $2; i++ )); do write_src "$1/m$i$FX_EXT" 1; done + for (( i = 1; i <= ${3:-0}; i++ )); do write_src "$(fx_test_file "$1" "$i")" 1; done +} + +# ---------- fixture rows: the TypeScript row this script's language block serves ---------- +# The matrix runs its cases once per row. use_row sets the row's variables and +# the fx_* builders write that language's files; the cases read only these. +# +# FX_ROWS the row ids, space-separated +# FX_NATIVE of the mixed repository's languages (d, go, python, +# typescript), the ones this plugin's block reviews itself +# FX_GLOB find(1) -name pattern for the row's source files +# FX_SRC_FILE the production file under a layer directory: two sleep +# calls and one suppression directive +# FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it +# fx_write_marker write the project marker for a (sub-)project +# fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable +FX_ROWS="typescript" +FX_NATIVE="typescript" + +# The row's glob is `*.ts`, not the block's `*.ts*`: the matrix derives file +# names for the files it creates from it, and `fresh.ts*` is no file name. +# Every file the row writes is a `.ts` file, so both globs select the same set. +use_row() { + FX_EXT='.ts' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='index.ts' + FX_GLOB='*.ts' + FX_SRC_FILE="services/worker.ts" + FX_SUPPRESS_LINE='// eslint-disable-next-line import/no-mutable-exports' +} + +fx_write_marker() { # + printf '{"name": "%s", "private": true}\n' "$2" > "$1/package.json" +} + +fx_write_code() { + mkdir -p "$REPO/services" + cat > "$REPO/services/worker.ts" <<'TS' +// Polling worker. + +// eslint-disable-next-line import/no-mutable-exports +export let cache: Map = new Map() + +// Polls until the deadline. +export async function run(deadline: number): Promise { + while (Date.now() < deadline) { + await new Promise((resolve) => setTimeout(resolve, 1000)) + } + await new Promise((resolve) => setTimeout(resolve, 50)) +} +TS + cat > "$REPO/services/worker.test.ts" <<'TS' +import { it } from 'vitest' + +it('runs', async () => { + await new Promise((resolve) => setTimeout(resolve, 10)) +}) +TS +} + +fx_test_file() { printf '%s/m%s.test.ts' "$1" "$2"; } + +# ESLint's own max-lines counts every line unless told to skip comments. +fx_write_linter_config() { # + printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +} + +# Every ESLint config form resolves: a rule named without a limit takes +# ESLint's documented default. +fx_write_linter_config_unreadable() { return 1; } + +# ========================= cases ========================= +run_row_cases() { # runs every case against the current row + +begin "a file at the limit passes and one line over is a finding" +write_src "src/at$FX_EXT" "$MAX_LINES"; write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" \ + && expect_not "src/at$FX_EXT:" && ok +finish + +begin "comment-only and blank lines are not counted by the plugin's default" +write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 20 20 +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the lines of a block comment are comment-only" +if [[ -z "$FX_BLOCK_OPEN" ]]; then skip "the language has no block comment"; finish +else + write_src "src/long$FX_EXT" $((MAX_LINES - 10)) 0 0 30 + run_lint file-length + expect_exit 0 && expect_no_finding && ok + finish +fi + +begin "a test file over the limit is not measured" +write_src "$(fx_test_file src 1)" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "a generated file over the limit is not measured" +write_src "src/gen$FX_EXT" $((MAX_LINES + 50)) +{ printf '%s DO NOT EDIT\n' "$FX_LINE_COMMENT"; cat "$REPO/src/gen$FX_EXT"; } > "$REPO/src/gen.tmp" && mv "$REPO/src/gen.tmp" "$REPO/src/gen$FX_EXT" +run_lint file-length +expect_exit 0 && expect_no_finding && ok +finish + +begin "the plugin's default names the setting that makes the linter hold it" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +finish + +begin "the repository's linter config holds the limit and is counted its way" +fx_write_linter_config 100 +write_src "src/a$FX_EXT" 90 20 +run_lint file-length +expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +finish + +begin "a linter config whose limit cannot be read is named, and the next source applies" +if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish +else + write_src "src/over$FX_EXT" $((MAX_LINES + 1)) + run_lint file-length + expect_exit 0 && expect_has "its limit cannot be read here" \ + && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok + finish +fi + +begin ".ldd-lint.yaml sets the limit when the linter holds none" +printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 101 +run_lint file-length +expect_exit 0 && expect_has "set by .ldd-lint.yaml" && expect_has "src/a$FX_EXT: 101 code lines (limit 100, .ldd-lint.yaml)" && ok +finish + +begin "a .ldd-lint.yaml value that is not a positive integer is inconclusive (exit 2)" +printf 'file_max_lines: lots\n' > "$REPO/.ldd-lint.yaml" +write_src "src/a$FX_EXT" 1 +run_lint file-length +expect_exit 2 && expect_has "file_max_lines must be a positive integer" && ok +finish + +begin "a directory below the warning size is not a finding" +fill_dir pkg $((WARN_FILES - 1)) 3 +write_src "pkg/$FX_SKIP_FILE" 1 +run_lint package-size +expect_exit 0 && expect_no_finding && ok +finish + +begin "a directory at the warning size is a warning" +fill_dir pkg "$WARN_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $WARN_FILES source files (warning at $WARN_FILES, profile default)" && ok +finish + +begin "a directory at the limit is over the limit" +fill_dir pkg "$MAX_FILES" +run_lint package-size +expect_exit 0 && expect_has "pkg: $MAX_FILES source files (limit $MAX_FILES, profile default)" && ok +finish + +begin ".ldd-lint.yaml sets the directory sizes" +printf 'package_warn_files: 2\npackage_max_files: 3\n' > "$REPO/.ldd-lint.yaml" +fill_dir small 2; fill_dir big 3 +run_lint package-size +expect_exit 0 && expect_has "small: 2 source files (warning at 2, .ldd-lint.yaml)" \ + && expect_has "big: 3 source files (limit 3, .ldd-lint.yaml)" && ok +finish + +begin "--tsv writes path, line 0 and message as rows, and the threshold to stderr" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length --tsv +expect_exit 0 \ + && { [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}$((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" ]] || bad "stdout is not the one row"; } \ + && [[ "$OUT" == "src/over$FX_EXT${TAB}0${TAB}"* ]] \ + && { [[ "$ERR" == "threshold: "* ]] || bad "threshold line not on stderr"; } && [[ "$ERR" == "threshold: "* ]] && ok +finish + +begin "--files measures only the files it names" +write_src "src/named$FX_EXT" $((MAX_LINES + 1)); write_src "src/other$FX_EXT" $((MAX_LINES + 1)) +printf 'src/named%s (not bundled: %s lines)\n' "$FX_EXT" $((MAX_LINES + 1)) > "$REPO/files.txt" +run_lint file-length --files files.txt +expect_exit 0 && expect_has "src/named$FX_EXT:" && expect_not "src/other$FX_EXT:" && ok +finish + +begin "in a git repository an ignored file is not measured" +write_src "src/kept$FX_EXT" $((MAX_LINES + 1)); write_src "build/out$FX_EXT" $((MAX_LINES + 1)) +printf 'build/\n' > "$REPO/.gitignore" +(cd "$REPO" && git init -q) +run_lint file-length +expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok +finish + +begin "two runs print the same output" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" +run_lint package-size; first="$OUT" +run_lint package-size +expect_exit 0 && { [[ "$OUT" == "$first" ]] || bad "the second run printed something else"; } && [[ "$OUT" == "$first" ]] && ok +finish + +begin "a tree without a source file is measured clean" +printf 'notes\n' > "$REPO/README.md" +run_lint file-length +expect_exit 0 && expect_has "no source file in scope" && ok +finish + +} # run_row_cases + +for ROW in $FX_ROWS; do + use_row "$ROW" + run_row_cases +done +ROW="" + +# --- row-independent --- +begin "a missing subcommand is a usage error (exit 2)" +run_lint +expect_exit 2 && expect_has "name a subcommand" && ok +finish + +begin "a --files list that does not exist is a usage error (exit 2)" +run_lint file-length --files no-such-list.txt +expect_exit 2 && expect_has "no such file list" && ok +finish + +# ========================= summary ========================= +echo +echo "ldd-lint_test: $pass passed, $failed failed, $skipped skipped" +(( failed == 0 )) diff --git a/ts-react-linter-driven-development/scripts/ldd-scope_test.sh b/ts-react-linter-driven-development/scripts/ldd-scope_test.sh index d68e1606..262304ce 100755 --- a/ts-react-linter-driven-development/scripts/ldd-scope_test.sh +++ b/ts-react-linter-driven-development/scripts/ldd-scope_test.sh @@ -110,6 +110,20 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # FX_SUPPRESS_LINE the suppression directive's line, as the diff adds it # fx_write_marker write the project marker for a (sub-)project # fx_write_code write $FX_SRC_FILE and a test file with a sleep +# +# The gap linter's matrix (ldd-lint_test.sh) reads these as well: +# FX_EXT the source suffix of the files the lint cases write +# FX_LINE_COMMENT a comment-only line +# FX_BLOCK_OPEN the line that opens a block comment, "" when the +# FX_BLOCK_CLOSE language has none, and the line that closes it +# FX_SKIP_FILE a source file name a directory count leaves out +# fx_test_file a test file's path in +# fx_write_linter_config the repository linter config that holds +# the file-length limit at , counting +# every line +# fx_write_linter_config_unreadable a config that names the rule with a limit +# the linter table cannot read; exit 1 when +# every config form is readable FX_ROWS="typescript" FX_NATIVE="typescript" @@ -117,6 +131,11 @@ FX_NATIVE="typescript" # names for the files it creates from it, and `fresh.ts*` is no file name. # Every file the row writes is a `.ts` file, so both globs select the same set. use_row() { + FX_EXT='.ts' + FX_LINE_COMMENT='// note' + FX_BLOCK_OPEN='/*' + FX_BLOCK_CLOSE='*/' + FX_SKIP_FILE='index.ts' FX_GLOB='*.ts' FX_SRC_FILE="services/worker.ts" FX_SUPPRESS_LINE='// eslint-disable-next-line import/no-mutable-exports' @@ -151,6 +170,17 @@ it('runs', async () => { TS } +fx_test_file() { printf '%s/m%s.test.ts' "$1" "$2"; } + +# ESLint's own max-lines counts every line unless told to skip comments. +fx_write_linter_config() { # + printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +} + +# Every ESLint config form resolves: a rule named without a limit takes +# ESLint's documented default. +fx_write_linter_config_unreadable() { return 1; } + # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/ts-react-linter-driven-development/skills/code-designing/SKILL.md b/ts-react-linter-driven-development/skills/code-designing/SKILL.md index fcf2fd1c..3c604076 100644 --- a/ts-react-linter-driven-development/skills/code-designing/SKILL.md +++ b/ts-react-linter-driven-development/skills/code-designing/SKILL.md @@ -33,7 +33,7 @@ Backward counterpart (fixing code that already fails lint/review): @refactoring. - a tuple return (`[T, boolean]`, `[Data, Error]`), or a `Record`, `string[]` or `Map` of primitives returned or accepted across a function boundary → the same named result type, with no ESLint rule to say so (R1, Name the Container) - `sonarjs/cognitive-complexity` or `sonarjs/max-lines-per-function` on a component, or `react/no-unstable-nested-components` → the component is the fat function: design the hook and the leaf type it should be reading (per `../../rules/R3-storifying.md`), not a smaller component nested inside it - the same `switch` on one discriminant in a second file, or `@typescript-eslint/no-explicit-any` at a boundary → design the discriminated union and its one dispatch owner (`../../rules/R11-conditional-dispatch.md`), or the parser that builds the type from `unknown` (`../../rules/R2-self-validating-types.md`) - - `sonarjs/max-lines` (600; or a module over ~450 lines where the repository sets no limit — count) → split juicy types into their own modules (juiciness per R1; module-per-type per `../../rules/R5-vertical-slice.md`); a single god component or class routes to @refactoring's god-object decomposition procedure first + - `sonarjs/max-lines` (the repository's limit, else the gap linter's 450 code lines) → split juicy types into their own modules (juiciness per R1; module-per-type per `../../rules/R5-vertical-slice.md`); a single god component or class routes to @refactoring's god-object decomposition procedure first - Package-size yellow/red zone → re-model with sub-folders *before* the zone escalates (`` in @refactoring's `reference.md` (`sed -n '/^/,/^<\/package_decomposition>/p; /^### Package decomposition/,$p'`)) - A Phase 4 review CLUSTER (≥2 hunters converging on one anchor — @linter-driven-development routes it here) → **cluster-scoped mode**: skip diff --git a/ts-react-linter-driven-development/skills/refactoring/SKILL.md b/ts-react-linter-driven-development/skills/refactoring/SKILL.md index 675f93f1..4c5e6ed4 100644 --- a/ts-react-linter-driven-development/skills/refactoring/SKILL.md +++ b/ts-react-linter-driven-development/skills/refactoring/SKILL.md @@ -48,7 +48,7 @@ detector: those rows are review findings the hunters own alone. | A tuple return (`[T, boolean]`, `[Data, Error]`); a `Record`, `string[]` or `Map` of primitives crossing a function boundary or nested in a type (review-only) | `../../rules/R1-primitive-obsession.md` (Name the Container — Q7 nested, Q8 flat across a boundary; the receivers' lookups, filters and loops are the methods of the type that does not exist yet) | | `sonarjs/no-duplicate-string` on an enum-shaped literal; `no-magic-numbers` on a domain value; `sonarjs/max-union-size` | `../../rules/R1-primitive-obsession.md` (Name enum strings, or a named type) | | `sonarjs/no-identical-functions`; duplicated code (review-only) | `../../rules/R1-primitive-obsession.md` (extract shared type/logic); duplicated `switch`/if-chains on the same kind/type discriminator → `../../rules/R11-conditional-dispatch.md` | -| `sonarjs/max-lines` (600) | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | +| `sonarjs/max-lines`; `scripts/ldd-lint.sh file-length` and `package-size` findings | `../../rules/R5-vertical-slice.md` — mechanics in ``, `sed -n '/^## File and package routing/,/^## Preparatory mode/p'` over this skill's `reference.md` | | `react/no-multi-comp` | `../../rules/R5-vertical-slice.md` — one component per file; the page folder decides where the second goes | | `@typescript-eslint/no-explicit-any` / `@typescript-eslint/no-unsafe-*` / `@typescript-eslint/no-non-null-assertion` / `@typescript-eslint/no-unnecessary-condition` at a boundary | `../../rules/R2-self-validating-types.md` (parse at the boundary — a guard, not an assertion) | | `@typescript-eslint/switch-exhaustiveness-check` (when configured; review-only otherwise) / `sonarjs/no-nested-switch` / `sonarjs/max-switch-cases` / `sonarjs/no-small-switch` | `../../rules/R11-conditional-dispatch.md` — handle the case at the single dispatch site and close it with `default: return assertNever(x)`; a second `switch` appearing is the R11 violation itself | diff --git a/ts-react-linter-driven-development/skills/refactoring/reference.md b/ts-react-linter-driven-development/skills/refactoring/reference.md index de13fd2e..447b39ff 100644 --- a/ts-react-linter-driven-development/skills/refactoring/reference.md +++ b/ts-react-linter-driven-development/skills/refactoring/reference.md @@ -72,7 +72,7 @@ decomposition): "Multi-rule procedures", below. ## File and package routing -**`sonarjs/max-lines` (600; or a module over ~450 lines where the repository sets no limit — count):** +**`sonarjs/max-lines`, or a module over 450 code lines where the repository sets no limit** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: | File pattern | Action | |---|---| @@ -82,15 +82,20 @@ decomposition): "Multi-rule procedures", below. | Several components in one file (`react/no-multi-comp`) | One component per file; the page folder decides where the second lands (R4, R5) | -**Package-size zones** — count non-test `.ts`/`.tsx` modules per directory: +**Package-size zones** — non-test `.ts`/`.tsx` modules per directory, `index.ts`, declaration files and stories left out: ``` -find -maxdepth 1 -type f \( -name '*.ts' -o -name '*.tsx' \) -not -name '*.test.*' -not -name '*.spec.*' -not -name '*.stories.*' -not -name '*.d.ts' -not -name 'index.ts' | wc -l +bash /../../scripts/ldd-lint.sh package-size --root . ``` -≤7 green — fine. 8–12 yellow — design review *before the next file lands*. ≥13 red — -**must decompose**. Either zone: run the 3-step design review under "Package -decomposition", printed with this range (it is a *design* review — missing domain types are the +The gap linter counts the non-test, non-generated source files of each directory; +a `.ldd-lint.yaml` at the repository root sets other sizes, and the first line of +its output says which sizes applied. + +Below 8 files green — fine. 8 or more yellow — design review +*before the next file lands*. 13 or more red — **must decompose**. Either +zone: run the 3-step design review under "Package decomposition", printed with this +range (it is a *design* review — missing domain types are the disease, file count the symptom). Invoke @code-designing to validate extracted types. @@ -272,9 +277,9 @@ composition: @code-designing. ### Package decomposition -Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone (≥13 non-test `.ts` -files at one directory level) or yellow zone (8–12) — detection command and zone -table in `` above. +Spans R5 × R4 × R1 × R2. **Trigger**: package-size red zone (13 or more non-test +`.ts` files at one directory level) or yellow zone (8 or more) — the +measuring command and the zones are in `` above. **A package-size violation is a design review, not a mechanical file split.** File count is the symptom; the disease is usually missing domain types or multiple From 705ea84265adbd3d3689b9b42829c3a9e111eb40 Mon Sep 17 00:00:00 2001 From: Dan Mordechay Date: Sun, 18 Oct 2026 18:00:06 +0300 Subject: [PATCH 2/4] ldd-lint: propose the linter setting, guard dash paths, pin more config forms - The threshold line offers the linter setting as a proposal ("a setting to propose, never to add on your own"). An agent that runs the script does not edit a linter config, as the house rules require. - Every path the script gives to find, head or awk starts with "./", and a directory is cut from a path by parameter expansion. A file or directory whose name starts with "-" is measured and never read as an option. A matrix case covers it; dirname failed that case before this change. - The fixture rows pin more config forms in CI: revive's bare-number form (Go plugin), its map form (generic plugin's Go row), revive with the skip flags and sonarjs/max-lines, which count only code lines. - docs/token-budget.md keeps the planned Go analyzer apart from the gap linter's name. Co-Authored-By: Claude Opus 5.5 --- core/README.md | 2 +- core/includes/scripts/ldd-lint-langs.sh | 8 ++--- core/scripts/ldd-lint.sh | 25 +++++++++----- core/scripts/ldd-lint_test.sh | 21 ++++++++++-- docs/token-budget.md | 3 +- .../scripts/ldd-detect_test.sh | 10 +++++- .../scripts/ldd-lint.sh | 33 +++++++++++-------- .../scripts/ldd-lint_test.sh | 31 +++++++++++++++-- .../scripts/ldd-scope_test.sh | 10 +++++- lang/generic/scripts/ldd-fixture.sh | 10 ++++++ lang/go/scripts/ldd-fixture.sh | 10 +++++- lang/python/scripts/ldd-fixture.sh | 6 ++++ lang/ts-react/scripts/ldd-fixture.sh | 8 +++++ .../scripts/ldd-detect_test.sh | 10 ++++++ linter-driven-development/scripts/ldd-lint.sh | 33 +++++++++++-------- .../scripts/ldd-lint_test.sh | 31 +++++++++++++++-- .../scripts/ldd-scope_test.sh | 10 ++++++ .../scripts/ldd-detect_test.sh | 6 ++++ .../scripts/ldd-lint.sh | 33 +++++++++++-------- .../scripts/ldd-lint_test.sh | 27 +++++++++++++-- .../scripts/ldd-scope_test.sh | 6 ++++ .../scripts/ldd-detect_test.sh | 8 +++++ .../scripts/ldd-lint.sh | 33 +++++++++++-------- .../scripts/ldd-lint_test.sh | 29 ++++++++++++++-- .../scripts/ldd-scope_test.sh | 8 +++++ 25 files changed, 330 insertions(+), 81 deletions(-) diff --git a/core/README.md b/core/README.md index 368ce8a3..93855fea 100644 --- a/core/README.md +++ b/core/README.md @@ -186,7 +186,7 @@ differently from the other bindings: own linter config sets a file-length limit and how that linter counts. The table is the one place core names linters, and its `residue-exempt:` marker says so. The matrix reads the binding's `scripts/ldd-fixture.sh` for the source suffix, the - comment forms, a test file's name and a linter config per row. + comment forms, a test file's name and the linter configs per row. - `scripts/check-repo-brain_test.sh` is a core template whose cases read one include, `scripts/repo-brain-fixture.sh`: the fixture rows (marker, code file, second package, sub-project) for every language the rendered gate's adapter serves. The diff --git a/core/includes/scripts/ldd-lint-langs.sh b/core/includes/scripts/ldd-lint-langs.sh index 9ac1a2f3..122ebea4 100644 --- a/core/includes/scripts/ldd-lint-langs.sh +++ b/core/includes/scripts/ldd-lint-langs.sh @@ -15,9 +15,9 @@ # TAB # unreadable TAB # none -# lint_file_length_snippet the setting that makes the repository's -# linter hold the limit, or how to set it when -# no linter rule is known +# lint_file_length_snippet the linter setting that would hold the +# limit, or the .ldd-lint.yaml entry when no +# linter rule is known: text to propose lint_comment_style() { case "$1" in @@ -151,7 +151,7 @@ lint_file_length_snippet() { go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; - *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + *) printf "no linter rule is known for %s; a .ldd-lint.yaml entry file_max_lines: %s would state this limit" "$1" "$2" ;; esac } # ================== end linter table ================== \ No newline at end of file diff --git a/core/scripts/ldd-lint.sh b/core/scripts/ldd-lint.sh index 43407714..f477ba4e 100755 --- a/core/scripts/ldd-lint.sh +++ b/core/scripts/ldd-lint.sh @@ -30,7 +30,8 @@ # 3. the plugin's defaults: {{.FileMaxLines}} code lines per file (blank and # comment-only lines not counted); a warning at {{.PackageWarnFiles}} source files per # directory and the limit at {{.PackageMaxFiles}}; for file-length, the line also -# prints the setting that makes the repository's linter hold the limit +# prints the linter setting that would hold the limit, as a proposal for +# the repository's owners: the script and its caller never edit a config # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -79,7 +80,7 @@ done [[ -d "$ROOT" ]] || die "not a directory: $ROOT" if [[ -n "$FILES_LIST" ]]; then [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" - FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") + [[ "$FILES_LIST" == /* ]] || FILES_LIST="$(pwd)/$FILES_LIST" fi cd "$ROOT" || exit 2 @@ -140,11 +141,14 @@ while IFS= read -r f; do done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" # measured — exit 0 when the file counts for this language: not under -# an excluded directory, not a test, not generated. +# an excluded directory, not a test, not generated. Every path the script +# hands to find, head or awk starts with "./", and a directory is cut from a +# path by parameter expansion, so a name that starts with "-" is never read as +# an option. measured() { printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 lang_is_test "$1" && return 1 - head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + head -20 "./$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 return 0 } @@ -199,7 +203,7 @@ run_file_length() { # — every measured file of that language say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi fi unit="lines" @@ -208,7 +212,7 @@ run_file_length() { # — every measured file of that language (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } @@ -226,18 +230,21 @@ run_package_size() { # — every directory that holds a measured file of th while IFS= read -r d; do n=0 while IFS= read -r f; do - f="${f#./}" + f="${f#./}"; f="${f#./}" [[ "$(lang_of_path "$f")" == "$id" ]] || continue [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue measured "$f" || continue n=$((n + 1)) - done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + done < <(find "./$d" -maxdepth 1 -type f | LC_ALL=C sort) if (( n >= lim )); then finding "$d" "$n source files (limit $lim, $src)" elif (( n >= warn )); then finding "$d" "$n source files (warning at $warn, $src)" fi - done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do + measured "$f" || continue + case "$f" in */*) printf '%s\n' "${f%/*}" ;; *) printf '.\n' ;; esac + done | LC_ALL=C sort -u) } # ---------- run ---------- diff --git a/core/scripts/ldd-lint_test.sh b/core/scripts/ldd-lint_test.sh index 412b401b..88efc602 100644 --- a/core/scripts/ldd-lint_test.sh +++ b/core/scripts/ldd-lint_test.sh @@ -125,12 +125,21 @@ run_lint file-length expect_exit 0 && expect_no_finding && ok finish -begin "the plugin's default names the setting that makes the linter hold it" +begin "the plugin's default names a linter setting to propose, never to add" write_src "src/a$FX_EXT" 1 run_lint file-length -expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish +begin "a linter that skips comments and blank lines is counted its way" +if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish +else + write_src "src/a$FX_EXT" 90 20 20 + run_lint file-length + expect_exit 0 && expect_has "covered by" && expect_no_finding && ok + finish +fi + begin "the repository's linter config holds the limit and is counted its way" fx_write_linter_config 100 write_src "src/a$FX_EXT" 90 20 @@ -213,6 +222,14 @@ run_lint file-length expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok finish +begin "a path that starts with a dash is measured, never read as an option" +fill_dir "-pkg" "$MAX_FILES"; write_src "-src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint package-size; first="$OUT" +run_lint file-length +expect_exit 0 && { [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] || bad "the dash-named directory was not counted"; } \ + && [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] && expect_has "-src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish + begin "two runs print the same output" write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" run_lint package-size; first="$OUT" diff --git a/docs/token-budget.md b/docs/token-budget.md index 18b53d3f..0d70a4bb 100644 --- a/docs/token-budget.md +++ b/docs/token-budget.md @@ -244,7 +244,8 @@ it. ### S6 — the mechanical questions become a program -A Go analyzer, `ldd-lint` *(planned)* under `tools/`, answers the falsifying +A Go analyzer under `tools/` *(planned)*, a different program from the plugin's gap +linter `scripts/ldd-lint.sh`, answers the falsifying questions that are AST facts: package-level mutable state and manufactured root contexts (R8), test placement (R7), function and file length (R3), the mechanical halves of concurrency safety (R10) and mutation discipline (R12). It emits findings diff --git a/go-linter-driven-development/scripts/ldd-detect_test.sh b/go-linter-driven-development/scripts/ldd-detect_test.sh index 8b6d5ab6..06df2052 100644 --- a/go-linter-driven-development/scripts/ldd-detect_test.sh +++ b/go-linter-driven-development/scripts/ldd-detect_test.sh @@ -145,6 +145,9 @@ questions_in_rules() { # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -199,8 +202,13 @@ GO fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } +# The bare-number form; the generic plugin's Go row writes the map form. fx_write_linter_config() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [%s]\n - name: exported\n' "$1" > "$REPO/.golangci.yaml" +} + +fx_write_linter_config_code() { # + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" } fx_write_linter_config_unreadable() { diff --git a/go-linter-driven-development/scripts/ldd-lint.sh b/go-linter-driven-development/scripts/ldd-lint.sh index d0d16c96..8ce0a856 100755 --- a/go-linter-driven-development/scripts/ldd-lint.sh +++ b/go-linter-driven-development/scripts/ldd-lint.sh @@ -30,7 +30,8 @@ # 3. the plugin's defaults: 450 code lines per file (blank and # comment-only lines not counted); a warning at 8 source files per # directory and the limit at 13; for file-length, the line also -# prints the setting that makes the repository's linter hold the limit +# prints the linter setting that would hold the limit, as a proposal for +# the repository's owners: the script and its caller never edit a config # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -79,7 +80,7 @@ done [[ -d "$ROOT" ]] || die "not a directory: $ROOT" if [[ -n "$FILES_LIST" ]]; then [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" - FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") + [[ "$FILES_LIST" == /* ]] || FILES_LIST="$(pwd)/$FILES_LIST" fi cd "$ROOT" || exit 2 @@ -210,9 +211,9 @@ lang_is_test() { # TAB # unreadable TAB # none -# lint_file_length_snippet the setting that makes the repository's -# linter hold the limit, or how to set it when -# no linter rule is known +# lint_file_length_snippet the linter setting that would hold the +# limit, or the .ldd-lint.yaml entry when no +# linter rule is known: text to propose lint_comment_style() { case "$1" in @@ -346,7 +347,7 @@ lint_file_length_snippet() { go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; - *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + *) printf "no linter rule is known for %s; a .ldd-lint.yaml entry file_max_lines: %s would state this limit" "$1" "$2" ;; esac } # ================== end linter table ================== @@ -402,11 +403,14 @@ while IFS= read -r f; do done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" # measured — exit 0 when the file counts for this language: not under -# an excluded directory, not a test, not generated. +# an excluded directory, not a test, not generated. Every path the script +# hands to find, head or awk starts with "./", and a directory is cut from a +# path by parameter expansion, so a name that starts with "-" is never read as +# an option. measured() { printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 lang_is_test "$1" && return 1 - head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + head -20 "./$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 return 0 } @@ -461,7 +465,7 @@ run_file_length() { # — every measured file of that language say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi fi unit="lines" @@ -470,7 +474,7 @@ run_file_length() { # — every measured file of that language (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } @@ -488,18 +492,21 @@ run_package_size() { # — every directory that holds a measured file of th while IFS= read -r d; do n=0 while IFS= read -r f; do - f="${f#./}" + f="${f#./}"; f="${f#./}" [[ "$(lang_of_path "$f")" == "$id" ]] || continue [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue measured "$f" || continue n=$((n + 1)) - done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + done < <(find "./$d" -maxdepth 1 -type f | LC_ALL=C sort) if (( n >= lim )); then finding "$d" "$n source files (limit $lim, $src)" elif (( n >= warn )); then finding "$d" "$n source files (warning at $warn, $src)" fi - done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do + measured "$f" || continue + case "$f" in */*) printf '%s\n' "${f%/*}" ;; *) printf '.\n' ;; esac + done | LC_ALL=C sort -u) } # ---------- run ---------- diff --git a/go-linter-driven-development/scripts/ldd-lint_test.sh b/go-linter-driven-development/scripts/ldd-lint_test.sh index 10713ad4..5f674ed8 100644 --- a/go-linter-driven-development/scripts/ldd-lint_test.sh +++ b/go-linter-driven-development/scripts/ldd-lint_test.sh @@ -109,6 +109,9 @@ fill_dir() { # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -163,8 +166,13 @@ GO fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } +# The bare-number form; the generic plugin's Go row writes the map form. fx_write_linter_config() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [%s]\n - name: exported\n' "$1" > "$REPO/.golangci.yaml" +} + +fx_write_linter_config_code() { # + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" } fx_write_linter_config_unreadable() { @@ -209,12 +217,21 @@ run_lint file-length expect_exit 0 && expect_no_finding && ok finish -begin "the plugin's default names the setting that makes the linter hold it" +begin "the plugin's default names a linter setting to propose, never to add" write_src "src/a$FX_EXT" 1 run_lint file-length -expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish +begin "a linter that skips comments and blank lines is counted its way" +if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish +else + write_src "src/a$FX_EXT" 90 20 20 + run_lint file-length + expect_exit 0 && expect_has "covered by" && expect_no_finding && ok + finish +fi + begin "the repository's linter config holds the limit and is counted its way" fx_write_linter_config 100 write_src "src/a$FX_EXT" 90 20 @@ -297,6 +314,14 @@ run_lint file-length expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok finish +begin "a path that starts with a dash is measured, never read as an option" +fill_dir "-pkg" "$MAX_FILES"; write_src "-src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint package-size; first="$OUT" +run_lint file-length +expect_exit 0 && { [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] || bad "the dash-named directory was not counted"; } \ + && [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] && expect_has "-src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish + begin "two runs print the same output" write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" run_lint package-size; first="$OUT" diff --git a/go-linter-driven-development/scripts/ldd-scope_test.sh b/go-linter-driven-development/scripts/ldd-scope_test.sh index 05fd0474..7426136a 100755 --- a/go-linter-driven-development/scripts/ldd-scope_test.sh +++ b/go-linter-driven-development/scripts/ldd-scope_test.sh @@ -121,6 +121,9 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -175,8 +178,13 @@ GO fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } +# The bare-number form; the generic plugin's Go row writes the map form. fx_write_linter_config() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [%s]\n - name: exported\n' "$1" > "$REPO/.golangci.yaml" +} + +fx_write_linter_config_code() { # + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" } fx_write_linter_config_unreadable() { diff --git a/lang/generic/scripts/ldd-fixture.sh b/lang/generic/scripts/ldd-fixture.sh index a076a206..d33afdc9 100644 --- a/lang/generic/scripts/ldd-fixture.sh +++ b/lang/generic/scripts/ldd-fixture.sh @@ -22,6 +22,9 @@ # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -133,6 +136,13 @@ fx_write_linter_config() { # esac } +fx_write_linter_config_code() { # + case "$ROW" in + go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" ;; + python) return 1 ;; + esac +} + fx_write_linter_config_unreadable() { case "$ROW" in go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; diff --git a/lang/go/scripts/ldd-fixture.sh b/lang/go/scripts/ldd-fixture.sh index 71fb7305..4ff4943f 100644 --- a/lang/go/scripts/ldd-fixture.sh +++ b/lang/go/scripts/ldd-fixture.sh @@ -22,6 +22,9 @@ # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -76,8 +79,13 @@ GO fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } +# The bare-number form; the generic plugin's Go row writes the map form. fx_write_linter_config() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [%s]\n - name: exported\n' "$1" > "$REPO/.golangci.yaml" +} + +fx_write_linter_config_code() { # + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" } fx_write_linter_config_unreadable() { diff --git a/lang/python/scripts/ldd-fixture.sh b/lang/python/scripts/ldd-fixture.sh index 15e28284..5c5ec55a 100644 --- a/lang/python/scripts/ldd-fixture.sh +++ b/lang/python/scripts/ldd-fixture.sh @@ -22,6 +22,9 @@ # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -74,6 +77,9 @@ fx_write_linter_config() { # printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" } +# pylint counts every line; ruff has no file-length rule. +fx_write_linter_config_code() { return 1; } + fx_write_linter_config_unreadable() { printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" } diff --git a/lang/ts-react/scripts/ldd-fixture.sh b/lang/ts-react/scripts/ldd-fixture.sh index 12a95747..055fc6b8 100644 --- a/lang/ts-react/scripts/ldd-fixture.sh +++ b/lang/ts-react/scripts/ldd-fixture.sh @@ -22,6 +22,9 @@ # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -78,6 +81,11 @@ fx_write_linter_config() { # printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" } +# sonarjs/max-lines counts only code lines. +fx_write_linter_config_code() { # + printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: %s }],\n 'max-lines-per-function': ['error', 50],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +} + # Every ESLint config form resolves: a rule named without a limit takes # ESLint's documented default. fx_write_linter_config_unreadable() { return 1; } diff --git a/linter-driven-development/scripts/ldd-detect_test.sh b/linter-driven-development/scripts/ldd-detect_test.sh index cd5b9a29..7f8c7f3e 100644 --- a/linter-driven-development/scripts/ldd-detect_test.sh +++ b/linter-driven-development/scripts/ldd-detect_test.sh @@ -145,6 +145,9 @@ questions_in_rules() { # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -256,6 +259,13 @@ fx_write_linter_config() { # esac } +fx_write_linter_config_code() { # + case "$ROW" in + go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" ;; + python) return 1 ;; + esac +} + fx_write_linter_config_unreadable() { case "$ROW" in go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; diff --git a/linter-driven-development/scripts/ldd-lint.sh b/linter-driven-development/scripts/ldd-lint.sh index ccb47daf..eec28c96 100755 --- a/linter-driven-development/scripts/ldd-lint.sh +++ b/linter-driven-development/scripts/ldd-lint.sh @@ -30,7 +30,8 @@ # 3. the plugin's defaults: 450 code lines per file (blank and # comment-only lines not counted); a warning at 8 source files per # directory and the limit at 13; for file-length, the line also -# prints the setting that makes the repository's linter hold the limit +# prints the linter setting that would hold the limit, as a proposal for +# the repository's owners: the script and its caller never edit a config # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -79,7 +80,7 @@ done [[ -d "$ROOT" ]] || die "not a directory: $ROOT" if [[ -n "$FILES_LIST" ]]; then [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" - FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") + [[ "$FILES_LIST" == /* ]] || FILES_LIST="$(pwd)/$FILES_LIST" fi cd "$ROOT" || exit 2 @@ -326,9 +327,9 @@ lang_is_test() { # TAB # unreadable TAB # none -# lint_file_length_snippet the setting that makes the repository's -# linter hold the limit, or how to set it when -# no linter rule is known +# lint_file_length_snippet the linter setting that would hold the +# limit, or the .ldd-lint.yaml entry when no +# linter rule is known: text to propose lint_comment_style() { case "$1" in @@ -462,7 +463,7 @@ lint_file_length_snippet() { go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; - *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + *) printf "no linter rule is known for %s; a .ldd-lint.yaml entry file_max_lines: %s would state this limit" "$1" "$2" ;; esac } # ================== end linter table ================== @@ -518,11 +519,14 @@ while IFS= read -r f; do done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" # measured — exit 0 when the file counts for this language: not under -# an excluded directory, not a test, not generated. +# an excluded directory, not a test, not generated. Every path the script +# hands to find, head or awk starts with "./", and a directory is cut from a +# path by parameter expansion, so a name that starts with "-" is never read as +# an option. measured() { printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 lang_is_test "$1" && return 1 - head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + head -20 "./$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 return 0 } @@ -577,7 +581,7 @@ run_file_length() { # — every measured file of that language say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi fi unit="lines" @@ -586,7 +590,7 @@ run_file_length() { # — every measured file of that language (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } @@ -604,18 +608,21 @@ run_package_size() { # — every directory that holds a measured file of th while IFS= read -r d; do n=0 while IFS= read -r f; do - f="${f#./}" + f="${f#./}"; f="${f#./}" [[ "$(lang_of_path "$f")" == "$id" ]] || continue [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue measured "$f" || continue n=$((n + 1)) - done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + done < <(find "./$d" -maxdepth 1 -type f | LC_ALL=C sort) if (( n >= lim )); then finding "$d" "$n source files (limit $lim, $src)" elif (( n >= warn )); then finding "$d" "$n source files (warning at $warn, $src)" fi - done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do + measured "$f" || continue + case "$f" in */*) printf '%s\n' "${f%/*}" ;; *) printf '.\n' ;; esac + done | LC_ALL=C sort -u) } # ---------- run ---------- diff --git a/linter-driven-development/scripts/ldd-lint_test.sh b/linter-driven-development/scripts/ldd-lint_test.sh index ddeceae5..3554f8ef 100644 --- a/linter-driven-development/scripts/ldd-lint_test.sh +++ b/linter-driven-development/scripts/ldd-lint_test.sh @@ -109,6 +109,9 @@ fill_dir() { # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -220,6 +223,13 @@ fx_write_linter_config() { # esac } +fx_write_linter_config_code() { # + case "$ROW" in + go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" ;; + python) return 1 ;; + esac +} + fx_write_linter_config_unreadable() { case "$ROW" in go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; @@ -265,12 +275,21 @@ run_lint file-length expect_exit 0 && expect_no_finding && ok finish -begin "the plugin's default names the setting that makes the linter hold it" +begin "the plugin's default names a linter setting to propose, never to add" write_src "src/a$FX_EXT" 1 run_lint file-length -expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish +begin "a linter that skips comments and blank lines is counted its way" +if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish +else + write_src "src/a$FX_EXT" 90 20 20 + run_lint file-length + expect_exit 0 && expect_has "covered by" && expect_no_finding && ok + finish +fi + begin "the repository's linter config holds the limit and is counted its way" fx_write_linter_config 100 write_src "src/a$FX_EXT" 90 20 @@ -353,6 +372,14 @@ run_lint file-length expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok finish +begin "a path that starts with a dash is measured, never read as an option" +fill_dir "-pkg" "$MAX_FILES"; write_src "-src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint package-size; first="$OUT" +run_lint file-length +expect_exit 0 && { [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] || bad "the dash-named directory was not counted"; } \ + && [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] && expect_has "-src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish + begin "two runs print the same output" write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" run_lint package-size; first="$OUT" diff --git a/linter-driven-development/scripts/ldd-scope_test.sh b/linter-driven-development/scripts/ldd-scope_test.sh index 1543e338..3588efae 100755 --- a/linter-driven-development/scripts/ldd-scope_test.sh +++ b/linter-driven-development/scripts/ldd-scope_test.sh @@ -121,6 +121,9 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -232,6 +235,13 @@ fx_write_linter_config() { # esac } +fx_write_linter_config_code() { # + case "$ROW" in + go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" ;; + python) return 1 ;; + esac +} + fx_write_linter_config_unreadable() { case "$ROW" in go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; diff --git a/python-linter-driven-development/scripts/ldd-detect_test.sh b/python-linter-driven-development/scripts/ldd-detect_test.sh index f4ab9071..3c8ef4e0 100644 --- a/python-linter-driven-development/scripts/ldd-detect_test.sh +++ b/python-linter-driven-development/scripts/ldd-detect_test.sh @@ -145,6 +145,9 @@ questions_in_rules() { # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -197,6 +200,9 @@ fx_write_linter_config() { # printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" } +# pylint counts every line; ruff has no file-length rule. +fx_write_linter_config_code() { return 1; } + fx_write_linter_config_unreadable() { printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" } diff --git a/python-linter-driven-development/scripts/ldd-lint.sh b/python-linter-driven-development/scripts/ldd-lint.sh index 84956d45..c4932f7a 100755 --- a/python-linter-driven-development/scripts/ldd-lint.sh +++ b/python-linter-driven-development/scripts/ldd-lint.sh @@ -30,7 +30,8 @@ # 3. the plugin's defaults: 450 code lines per file (blank and # comment-only lines not counted); a warning at 8 source files per # directory and the limit at 13; for file-length, the line also -# prints the setting that makes the repository's linter hold the limit +# prints the linter setting that would hold the limit, as a proposal for +# the repository's owners: the script and its caller never edit a config # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -79,7 +80,7 @@ done [[ -d "$ROOT" ]] || die "not a directory: $ROOT" if [[ -n "$FILES_LIST" ]]; then [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" - FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") + [[ "$FILES_LIST" == /* ]] || FILES_LIST="$(pwd)/$FILES_LIST" fi cd "$ROOT" || exit 2 @@ -213,9 +214,9 @@ lang_is_test() { # TAB # unreadable TAB # none -# lint_file_length_snippet the setting that makes the repository's -# linter hold the limit, or how to set it when -# no linter rule is known +# lint_file_length_snippet the linter setting that would hold the +# limit, or the .ldd-lint.yaml entry when no +# linter rule is known: text to propose lint_comment_style() { case "$1" in @@ -349,7 +350,7 @@ lint_file_length_snippet() { go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; - *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + *) printf "no linter rule is known for %s; a .ldd-lint.yaml entry file_max_lines: %s would state this limit" "$1" "$2" ;; esac } # ================== end linter table ================== @@ -405,11 +406,14 @@ while IFS= read -r f; do done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" # measured — exit 0 when the file counts for this language: not under -# an excluded directory, not a test, not generated. +# an excluded directory, not a test, not generated. Every path the script +# hands to find, head or awk starts with "./", and a directory is cut from a +# path by parameter expansion, so a name that starts with "-" is never read as +# an option. measured() { printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 lang_is_test "$1" && return 1 - head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + head -20 "./$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 return 0 } @@ -464,7 +468,7 @@ run_file_length() { # — every measured file of that language say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi fi unit="lines" @@ -473,7 +477,7 @@ run_file_length() { # — every measured file of that language (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } @@ -491,18 +495,21 @@ run_package_size() { # — every directory that holds a measured file of th while IFS= read -r d; do n=0 while IFS= read -r f; do - f="${f#./}" + f="${f#./}"; f="${f#./}" [[ "$(lang_of_path "$f")" == "$id" ]] || continue [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue measured "$f" || continue n=$((n + 1)) - done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + done < <(find "./$d" -maxdepth 1 -type f | LC_ALL=C sort) if (( n >= lim )); then finding "$d" "$n source files (limit $lim, $src)" elif (( n >= warn )); then finding "$d" "$n source files (warning at $warn, $src)" fi - done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do + measured "$f" || continue + case "$f" in */*) printf '%s\n' "${f%/*}" ;; *) printf '.\n' ;; esac + done | LC_ALL=C sort -u) } # ---------- run ---------- diff --git a/python-linter-driven-development/scripts/ldd-lint_test.sh b/python-linter-driven-development/scripts/ldd-lint_test.sh index 7890e850..814eea45 100644 --- a/python-linter-driven-development/scripts/ldd-lint_test.sh +++ b/python-linter-driven-development/scripts/ldd-lint_test.sh @@ -109,6 +109,9 @@ fill_dir() { # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -161,6 +164,9 @@ fx_write_linter_config() { # printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" } +# pylint counts every line; ruff has no file-length rule. +fx_write_linter_config_code() { return 1; } + fx_write_linter_config_unreadable() { printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" } @@ -203,12 +209,21 @@ run_lint file-length expect_exit 0 && expect_no_finding && ok finish -begin "the plugin's default names the setting that makes the linter hold it" +begin "the plugin's default names a linter setting to propose, never to add" write_src "src/a$FX_EXT" 1 run_lint file-length -expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish +begin "a linter that skips comments and blank lines is counted its way" +if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish +else + write_src "src/a$FX_EXT" 90 20 20 + run_lint file-length + expect_exit 0 && expect_has "covered by" && expect_no_finding && ok + finish +fi + begin "the repository's linter config holds the limit and is counted its way" fx_write_linter_config 100 write_src "src/a$FX_EXT" 90 20 @@ -291,6 +306,14 @@ run_lint file-length expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok finish +begin "a path that starts with a dash is measured, never read as an option" +fill_dir "-pkg" "$MAX_FILES"; write_src "-src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint package-size; first="$OUT" +run_lint file-length +expect_exit 0 && { [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] || bad "the dash-named directory was not counted"; } \ + && [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] && expect_has "-src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish + begin "two runs print the same output" write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" run_lint package-size; first="$OUT" diff --git a/python-linter-driven-development/scripts/ldd-scope_test.sh b/python-linter-driven-development/scripts/ldd-scope_test.sh index c489d907..7f332f46 100755 --- a/python-linter-driven-development/scripts/ldd-scope_test.sh +++ b/python-linter-driven-development/scripts/ldd-scope_test.sh @@ -121,6 +121,9 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -173,6 +176,9 @@ fx_write_linter_config() { # printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" } +# pylint counts every line; ruff has no file-length rule. +fx_write_linter_config_code() { return 1; } + fx_write_linter_config_unreadable() { printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" } diff --git a/ts-react-linter-driven-development/scripts/ldd-detect_test.sh b/ts-react-linter-driven-development/scripts/ldd-detect_test.sh index 375f923b..28f00eca 100644 --- a/ts-react-linter-driven-development/scripts/ldd-detect_test.sh +++ b/ts-react-linter-driven-development/scripts/ldd-detect_test.sh @@ -145,6 +145,9 @@ questions_in_rules() { # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -201,6 +204,11 @@ fx_write_linter_config() { # printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" } +# sonarjs/max-lines counts only code lines. +fx_write_linter_config_code() { # + printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: %s }],\n 'max-lines-per-function': ['error', 50],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +} + # Every ESLint config form resolves: a rule named without a limit takes # ESLint's documented default. fx_write_linter_config_unreadable() { return 1; } diff --git a/ts-react-linter-driven-development/scripts/ldd-lint.sh b/ts-react-linter-driven-development/scripts/ldd-lint.sh index a26ea48b..a54e968d 100755 --- a/ts-react-linter-driven-development/scripts/ldd-lint.sh +++ b/ts-react-linter-driven-development/scripts/ldd-lint.sh @@ -30,7 +30,8 @@ # 3. the plugin's defaults: 450 code lines per file (blank and # comment-only lines not counted); a warning at 8 source files per # directory and the limit at 13; for file-length, the line also -# prints the setting that makes the repository's linter hold the limit +# prints the linter setting that would hold the limit, as a proposal for +# the repository's owners: the script and its caller never edit a config # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -79,7 +80,7 @@ done [[ -d "$ROOT" ]] || die "not a directory: $ROOT" if [[ -n "$FILES_LIST" ]]; then [[ -f "$FILES_LIST" ]] || die "no such file list: $FILES_LIST" - FILES_LIST=$(cd "$(dirname "$FILES_LIST")" && pwd)/$(basename "$FILES_LIST") + [[ "$FILES_LIST" == /* ]] || FILES_LIST="$(pwd)/$FILES_LIST" fi cd "$ROOT" || exit 2 @@ -223,9 +224,9 @@ lang_is_test() { # TAB # unreadable TAB # none -# lint_file_length_snippet the setting that makes the repository's -# linter hold the limit, or how to set it when -# no linter rule is known +# lint_file_length_snippet the linter setting that would hold the +# limit, or the .ldd-lint.yaml entry when no +# linter rule is known: text to propose lint_comment_style() { case "$1" in @@ -359,7 +360,7 @@ lint_file_length_snippet() { go) printf "revive in .golangci.yaml: - name: file-length-limit, arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]" "$2" ;; typescript|javascript) printf "the eslint config: 'sonarjs/max-lines': ['error', { maximum: %s }]" "$2" ;; python) printf "pyproject.toml: [tool.pylint.format] max-module-lines = %s (pylint counts every line, comments and blank lines too)" "$2" ;; - *) printf "no linter rule is known for %s; set file_max_lines: %s in .ldd-lint.yaml to keep this limit" "$1" "$2" ;; + *) printf "no linter rule is known for %s; a .ldd-lint.yaml entry file_max_lines: %s would state this limit" "$1" "$2" ;; esac } # ================== end linter table ================== @@ -415,11 +416,14 @@ while IFS= read -r f; do done < <(list_candidates) | LC_ALL=C sort -u > "$SCOPE" # measured — exit 0 when the file counts for this language: not under -# an excluded directory, not a test, not generated. +# an excluded directory, not a test, not generated. Every path the script +# hands to find, head or awk starts with "./", and a directory is cut from a +# path by parameter expansion, so a name that starts with "-" is never read as +# an option. measured() { printf '%s\n' "$1" | grep -qE -- "$LANG_EXCLUDE_RE" && return 1 lang_is_test "$1" && return 1 - head -20 "$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 + head -20 "./$1" 2>/dev/null | grep -qE -- "$LANG_GENERATED_RE" && return 1 return 0 } @@ -474,7 +478,7 @@ run_file_length() { # — every measured file of that language say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; add $(lint_file_length_snippet "$id" "$max")" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi fi unit="lines" @@ -483,7 +487,7 @@ run_file_length() { # — every measured file of that language (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "$f") + n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } @@ -501,18 +505,21 @@ run_package_size() { # — every directory that holds a measured file of th while IFS= read -r d; do n=0 while IFS= read -r f; do - f="${f#./}" + f="${f#./}"; f="${f#./}" [[ "$(lang_of_path "$f")" == "$id" ]] || continue [[ -n "$skip" ]] && printf '%s\n' "$f" | grep -qE -- "$skip" && continue measured "$f" || continue n=$((n + 1)) - done < <(find "$d" -maxdepth 1 -type f | LC_ALL=C sort) + done < <(find "./$d" -maxdepth 1 -type f | LC_ALL=C sort) if (( n >= lim )); then finding "$d" "$n source files (limit $lim, $src)" elif (( n >= warn )); then finding "$d" "$n source files (warning at $warn, $src)" fi - done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do measured "$f" && dirname "$f"; done | LC_ALL=C sort -u) + done < <(awk -F'\t' -v id="$id" '$1 == id { print $2 }' "$SCOPE" | while IFS= read -r f; do + measured "$f" || continue + case "$f" in */*) printf '%s\n' "${f%/*}" ;; *) printf '.\n' ;; esac + done | LC_ALL=C sort -u) } # ---------- run ---------- diff --git a/ts-react-linter-driven-development/scripts/ldd-lint_test.sh b/ts-react-linter-driven-development/scripts/ldd-lint_test.sh index 36b35bac..55b0182e 100644 --- a/ts-react-linter-driven-development/scripts/ldd-lint_test.sh +++ b/ts-react-linter-driven-development/scripts/ldd-lint_test.sh @@ -109,6 +109,9 @@ fill_dir() { # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -165,6 +168,11 @@ fx_write_linter_config() { # printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" } +# sonarjs/max-lines counts only code lines. +fx_write_linter_config_code() { # + printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: %s }],\n 'max-lines-per-function': ['error', 50],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +} + # Every ESLint config form resolves: a rule named without a limit takes # ESLint's documented default. fx_write_linter_config_unreadable() { return 1; } @@ -207,12 +215,21 @@ run_lint file-length expect_exit 0 && expect_no_finding && ok finish -begin "the plugin's default names the setting that makes the linter hold it" +begin "the plugin's default names a linter setting to propose, never to add" write_src "src/a$FX_EXT" 1 run_lint file-length -expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "; add " && ok +expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish +begin "a linter that skips comments and blank lines is counted its way" +if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish +else + write_src "src/a$FX_EXT" 90 20 20 + run_lint file-length + expect_exit 0 && expect_has "covered by" && expect_no_finding && ok + finish +fi + begin "the repository's linter config holds the limit and is counted its way" fx_write_linter_config 100 write_src "src/a$FX_EXT" 90 20 @@ -295,6 +312,14 @@ run_lint file-length expect_exit 0 && expect_has "src/kept$FX_EXT:" && expect_not "build/out$FX_EXT:" && ok finish +begin "a path that starts with a dash is measured, never read as an option" +fill_dir "-pkg" "$MAX_FILES"; write_src "-src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint package-size; first="$OUT" +run_lint file-length +expect_exit 0 && { [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] || bad "the dash-named directory was not counted"; } \ + && [[ "$first" == *"-pkg: $MAX_FILES source files"* ]] && expect_has "-src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish + begin "two runs print the same output" write_src "src/over$FX_EXT" $((MAX_LINES + 1)); fill_dir pkg "$MAX_FILES" run_lint package-size; first="$OUT" diff --git a/ts-react-linter-driven-development/scripts/ldd-scope_test.sh b/ts-react-linter-driven-development/scripts/ldd-scope_test.sh index 262304ce..6a7e6bae 100755 --- a/ts-react-linter-driven-development/scripts/ldd-scope_test.sh +++ b/ts-react-linter-driven-development/scripts/ldd-scope_test.sh @@ -121,6 +121,9 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # fx_write_linter_config the repository linter config that holds # the file-length limit at , counting # every line +# fx_write_linter_config_code the config of a linter that counts only +# code lines, holding the limit at ; +# exit 1 when the language has none # fx_write_linter_config_unreadable a config that names the rule with a limit # the linter table cannot read; exit 1 when # every config form is readable @@ -177,6 +180,11 @@ fx_write_linter_config() { # printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" } +# sonarjs/max-lines counts only code lines. +fx_write_linter_config_code() { # + printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: %s }],\n 'max-lines-per-function': ['error', 50],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +} + # Every ESLint config form resolves: a rule named without a limit takes # ESLint's documented default. fx_write_linter_config_unreadable() { return 1; } From de188536cafd8d8e99d7cc32de67accf304052be Mon Sep 17 00:00:00 2001 From: Dan Mordechay Date: Sun, 18 Oct 2026 18:36:19 +0300 Subject: [PATCH 3/4] ldd-lint: a named file-length rule hands file length to the linter The gap linter read each linter's limit and copied each linter's way of counting. That code was a second copy of the linters' rules to maintain and test. The linter is the authority for every threshold it owns, so the script now asks one question: does a linter config at the repository root name a file-length rule (revive file-length-limit, sonarjs/max-lines, ESLint max-lines, pylint max-module-lines) on a line that is not a comment? - Yes: the threshold line names the rule, and the script measures no file length. The rule's limit, its counting and whether it is on are the repository's decision. - No: the script counts code lines against .ldd-lint.yaml or the plugin default, as before. The config readers, the counting modes and their fixture builders are removed (about 120 lines). The matrix pins the new behavior: a named rule, a rule of a similar name (argument-limit, max-args, max-lines-per-function) and a commented-out rule, for every fixture row. Co-Authored-By: Claude Opus 5.5 --- core/README.md | 11 +- core/includes/scripts/ldd-lint-langs.sh | 143 +++--------- .../refactoring/file-and-package-routing.md | 2 +- core/scripts/ldd-lint.sh | 69 +++--- core/scripts/ldd-lint_test.sh | 38 ++-- go-linter-driven-development/CHANGELOG.md | 20 +- .../scripts/ldd-detect_test.sh | 31 ++- .../scripts/ldd-lint.sh | 212 +++++------------- .../scripts/ldd-lint_test.sh | 69 +++--- .../scripts/ldd-scope_test.sh | 31 ++- .../skills/refactoring/reference.md | 2 +- lang/generic/passthrough/CHANGELOG.md | 20 +- lang/generic/scripts/ldd-fixture.sh | 39 ++-- lang/go/passthrough/CHANGELOG.md | 20 +- lang/go/scripts/ldd-fixture.sh | 31 ++- .../refactoring/file-and-package-routing.md | 2 +- lang/python/passthrough/CHANGELOG.md | 20 +- lang/python/scripts/ldd-fixture.sh | 29 ++- .../refactoring/file-and-package-routing.md | 2 +- lang/ts-react/passthrough/CHANGELOG.md | 20 +- lang/ts-react/scripts/ldd-fixture.sh | 33 ++- .../refactoring/file-and-package-routing.md | 2 +- linter-driven-development/CHANGELOG.md | 20 +- .../scripts/ldd-detect_test.sh | 39 ++-- linter-driven-development/scripts/ldd-lint.sh | 212 +++++------------- .../scripts/ldd-lint_test.sh | 77 +++---- .../scripts/ldd-scope_test.sh | 39 ++-- .../skills/refactoring/reference.md | 2 +- python-linter-driven-development/CHANGELOG.md | 20 +- .../scripts/ldd-detect_test.sh | 29 ++- .../scripts/ldd-lint.sh | 212 +++++------------- .../scripts/ldd-lint_test.sh | 67 +++--- .../scripts/ldd-scope_test.sh | 29 ++- .../skills/refactoring/reference.md | 2 +- .../CHANGELOG.md | 20 +- .../scripts/ldd-detect_test.sh | 33 ++- .../scripts/ldd-lint.sh | 212 +++++------------- .../scripts/ldd-lint_test.sh | 71 +++--- .../scripts/ldd-scope_test.sh | 33 ++- .../skills/refactoring/reference.md | 2 +- 40 files changed, 708 insertions(+), 1257 deletions(-) diff --git a/core/README.md b/core/README.md index 93855fea..07f0e7a4 100644 --- a/core/README.md +++ b/core/README.md @@ -182,11 +182,14 @@ differently from the other bindings: source files per directory, where the repository's linter does not hold them. It reads the language table and the language block, and a third include with a core default, `scripts/ldd-lint-langs.sh`, the linter table: per language id, the - comment syntax, the files a directory count leaves out, and where the repository's - own linter config sets a file-length limit and how that linter counts. The table is - the one place core names linters, and its `residue-exempt:` marker says so. The + comment syntax, the files a directory count leaves out, and whether the + repository's own linter config names a file-length rule. A named rule belongs to + the linter, so the script reads no limit out of a config and copies no linter's way + of counting. The table is the one place core names linters, and its + `residue-exempt:` marker says so. The matrix reads the binding's `scripts/ldd-fixture.sh` for the source suffix, the - comment forms, a test file's name and the linter configs per row. + comment forms, a test file's name and two linter configs per row: one that names the + file-length rule and one that names a rule of a similar name. - `scripts/check-repo-brain_test.sh` is a core template whose cases read one include, `scripts/repo-brain-fixture.sh`: the fixture rows (marker, code file, second package, sub-project) for every language the rendered gate's adapter serves. The diff --git a/core/includes/scripts/ldd-lint-langs.sh b/core/includes/scripts/ldd-lint-langs.sh index 122ebea4..70a2d4fb 100644 --- a/core/includes/scripts/ldd-lint-langs.sh +++ b/core/includes/scripts/ldd-lint-langs.sh @@ -1,8 +1,8 @@ # ==================== linter table ==================== # What the gap linter needs to know per language id: how a comment is written, -# which source files a directory count leaves out, and where the repository's -# own linter sets a file-length limit. The same table is rendered into every -# plugin, so two plugins measure one file the same way. +# which source files a directory count leaves out, and whether the +# repository's own linter holds the file-length limit. The same table is +# rendered into every plugin, so two plugins measure one file the same way. # residue-exempt: the table names every language's linter and config file on # purpose, the same in every rendering. # @@ -10,11 +10,12 @@ # lint_package_skip_re ERE over a relative path: source files a # directory count leaves out (an index module, # a package marker, generated stubs); "" for none -# lint_file_length_config one line, from the repository root: -# covered TAB TAB TAB -# TAB -# unreadable TAB -# none +# lint_file_length_rule " in " when a linter config +# at the repository root names a file-length +# rule on a line that is not a comment, else +# nothing. A named rule belongs to the linter: +# its limit, its counting and whether it is on +# are the repository's decision. # lint_file_length_snippet the linter setting that would hold the # limit, or the .ldd-lint.yaml entry when no # linter rule is known: text to propose @@ -36,114 +37,36 @@ lint_package_skip_re() { return 0 } -# _lint_window — the key's line and the lines after it, up to -# the next rule entry or 15 lines, joined by spaces; one window per match. The -# patterns reach awk as strings, so they use bracket expressions, never -# backslash escapes, which awk -v would rewrite. -_lint_window() { - awk -v key="$1" -v q="'" ' - BEGIN { - entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" - qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" - } - function flush() { if (w != "") print w; w = ""; n = 0 } - { - if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() - if ($0 ~ key) { flush(); n = 1; w = $0; next } - if (n > 0) { w = w " " $0; if (++n > 15) flush() } - } - END { flush() } - ' "$2" -} - -# _lint_number — the first match's digits -_lint_number() { - printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' -} - -# _lint_off — exit 0 when the rule in the window is switched off -_lint_off() { - printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" -} - -_lint_config_go() { - local f w max sc sb unread="" - for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do +# _lint_names ... — print the first file that matches the pattern +# on a line that is not a comment +_lint_names() { + local re="$1" f; shift + for f in "$@"; do [[ -f "$f" ]] || continue - grep -q 'file-length-limit' "$f" 2>/dev/null || continue - case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac - while IFS= read -r w; do - [[ "$w" =~ disabled:[[:space:]]*true ]] && continue - max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") - if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi - (( max > 0 )) || continue - sc=0; sb=0 - [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") + grep -vE '^[[:space:]]*(#|//|;)' -- "$f" 2>/dev/null | grep -qE -- "$re" && { printf '%s' "$f"; return 0; } done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' + return 1 } -_lint_config_eslint() { - local f w max sc sb - for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ - .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do - [[ -f "$f" ]] || continue - grep -q 'max-lines' "$f" 2>/dev/null || continue - # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") - printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" - return 0 - done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") - # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") - sc=0; sb=0 - [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") - done - printf 'none\n' -} - -_lint_config_python() { - local f line max unread="" - for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do - [[ -f "$f" ]] || continue - line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) - [[ -n "$line" ]] || continue - max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") - if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi - (( max > 0 )) || continue - # pylint counts every line of the module. - printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" - return 0 - done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' -} - -lint_file_length_config() { +lint_file_length_rule() { + local f case "$1" in - go) _lint_config_go ;; - typescript|javascript) _lint_config_eslint ;; - python) _lint_config_python ;; - *) printf 'none\n' ;; + go) + f=$(_lint_names 'file-length-limit' .golangci.yaml .golangci.yml .golangci.toml .golangci.json) \ + && printf 'revive file-length-limit in %s' "$f" ;; + typescript|javascript) + local configs=(eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml) + if f=$(_lint_names 'sonarjs/max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'sonarjs/max-lines in %s' "$f" + elif f=$(_lint_names '(^|[^-/a-zA-Z])max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'max-lines in %s' "$f" + fi ;; + python) + f=$(_lint_names '^[[:space:]]*max-module-lines[[:space:]]*=' pyproject.toml .pylintrc pylintrc setup.cfg tox.ini) \ + && printf 'pylint max-module-lines in %s' "$f" ;; esac + return 0 } lint_file_length_snippet() { diff --git a/core/includes/skills/refactoring/file-and-package-routing.md b/core/includes/skills/refactoring/file-and-package-routing.md index 081e6705..c4aba52a 100644 --- a/core/includes/skills/refactoring/file-and-package-routing.md +++ b/core/includes/skills/refactoring/file-and-package-routing.md @@ -1,4 +1,4 @@ -**A file over the file-length limit** — the repository linter's limit where it sets one, else {{.FileMaxLines}} code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: +**A file over the file-length limit** — the repository linter reports it where the linter has a file-length rule; elsewhere a file over {{.FileMaxLines}} code lines is one, and `bash /../../scripts/ldd-lint.sh file-length --root .` lists them, or names the linter rule that holds the limit: | File pattern | Action | |---|---| diff --git a/core/scripts/ldd-lint.sh b/core/scripts/ldd-lint.sh index f477ba4e..a3d1b805 100755 --- a/core/scripts/ldd-lint.sh +++ b/core/scripts/ldd-lint.sh @@ -3,8 +3,8 @@ # # Measures the design triggers no linter holds, or that the repository's linter # does not hold yet: lines per source file and source files per directory. It -# never edits a file. Where the repository's linter holds a limit, that limit -# is the one applied, counted the way that linter counts. +# never edits a file. Where the repository's linter has a file-length rule, the +# linter owns file length and this script leaves it alone. # # Usage: bash scripts/ldd-lint.sh [options] # file-length source files over the file-length limit @@ -20,18 +20,17 @@ # know ({{.Lang}} block below) # --test-re test-file pattern over relative paths # -# Thresholds, per language, first that applies — the "threshold:" line says -# which one a run used: -# 1. the repository's linter config holds the rule: its limit, counted the -# way that linter counts (file-length only; no linter counts files per -# directory); a config the table cannot read is named, then skipped -# 2. .ldd-lint.yaml at the repository root: file_max_lines, -# package_warn_files, package_max_files, each a positive integer -# 3. the plugin's defaults: {{.FileMaxLines}} code lines per file (blank and -# comment-only lines not counted); a warning at {{.PackageWarnFiles}} source files per -# directory and the limit at {{.PackageMaxFiles}}; for file-length, the line also +# Thresholds, per language — the "threshold:" line says which one a run used: +# file-length: when a linter config at the repository root names a +# file-length rule, the linter holds it: the line names the rule and the +# script measures nothing more. Else the limit is file_max_lines in +# .ldd-lint.yaml, else the plugin's default, {{.FileMaxLines}} code lines (blank and +# comment-only lines are not counted); with the default, the line also # prints the linter setting that would hold the limit, as a proposal for # the repository's owners: the script and its caller never edit a config +# package-size: no linter counts files per directory. package_warn_files and +# package_max_files in .ldd-lint.yaml, else the plugin's defaults: a +# warning at {{.PackageWarnFiles}} source files per directory and the limit at {{.PackageMaxFiles}} # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -153,8 +152,8 @@ measured() { } # ---------- file-length ---------- -# COUNT_AWK prints a file's line count under the style's comment syntax: -# skipc drops comment-only lines (block comments included), skipb blank lines. +# COUNT_AWK prints a file's code lines under the style's comment syntax: blank +# lines and comment-only lines, block comments included, are not counted. COUNT_AWK=' function comment_only(t, rest, i) { if (inblock) { @@ -175,45 +174,31 @@ function comment_only(t, rest, i) { } { t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) - if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } - c = comment_only(t) - if (c && skipc) next + if (t == "" || comment_only(t)) next n++ } END { print n + 0 } ' run_file_length() { # — every measured file of that language - local id="$1" cfg status max sc sb label src unit f n o - cfg=$(lint_file_length_config "$id") - IFS=$'\t' read -r status max sc sb label <<< "$cfg" - if [[ "$status" == "unreadable" ]]; then - # an unreadable row carries the rule and file in its second field - say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" - status=none + local id="$1" rule max src f n o + rule=$(lint_file_length_rule "$id") + if [[ -n "$rule" ]]; then + say "threshold: $id file length — held by $rule; the repository's linter reports long files, so this script measures none" + return 0 fi - if [[ "$status" == "covered" ]]; then - src="$label" - say "threshold: $id file length — covered by $label (limit $max)" + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else - sc=1; sb=1 - o=$(override file_max_lines) || exit 2 - if [[ -n "$o" ]]; then - max="$o"; src="$OVERRIDE_FILE" - say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" - else - max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" - fi + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi - unit="lines" - (( sc && sb )) && unit="code lines" - (( sc && ! sb )) && unit="lines without comments" - (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") - (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + n=$(awk -v style="$(lint_comment_style "$id")" "$COUNT_AWK" "./$f") + (( n > max )) && finding "$f" "$n code lines (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } diff --git a/core/scripts/ldd-lint_test.sh b/core/scripts/ldd-lint_test.sh index 88efc602..a60b0553 100644 --- a/core/scripts/ldd-lint_test.sh +++ b/core/scripts/ldd-lint_test.sh @@ -131,31 +131,27 @@ run_lint file-length expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish -begin "a linter that skips comments and blank lines is counted its way" -if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish -else - write_src "src/a$FX_EXT" 90 20 20 - run_lint file-length - expect_exit 0 && expect_has "covered by" && expect_no_finding && ok - finish -fi +begin "a file-length rule in the linter config hands file length to the linter" +fx_write_linter_config +write_src "src/over$FX_EXT" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_has "held by " && expect_no_finding && ok +finish -begin "the repository's linter config holds the limit and is counted its way" -fx_write_linter_config 100 -write_src "src/a$FX_EXT" 90 20 +begin "a similar rule of another name leaves file length to the gap linter" +fx_write_linter_config_other +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) run_lint file-length -expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok finish -begin "a linter config whose limit cannot be read is named, and the next source applies" -if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish -else - write_src "src/over$FX_EXT" $((MAX_LINES + 1)) - run_lint file-length - expect_exit 0 && expect_has "its limit cannot be read here" \ - && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok - finish -fi +begin "a file-length rule that is commented out leaves file length to the gap linter" +fx_write_linter_config +sed "s|^|$FX_LINTER_COMMENT |" "$REPO/$FX_LINTER_CONFIG" > "$REPO/cfg.tmp" && mv "$REPO/cfg.tmp" "$REPO/$FX_LINTER_CONFIG" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish begin ".ldd-lint.yaml sets the limit when the linter holds none" printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" diff --git a/go-linter-driven-development/CHANGELOG.md b/go-linter-driven-development/CHANGELOG.md index bcfe2328..5f4269d4 100644 --- a/go-linter-driven-development/CHANGELOG.md +++ b/go-linter-driven-development/CHANGELOG.md @@ -9,20 +9,20 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio - **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` measures lines per source file (`file-length`) and source files per directory - (`package-size`). Where the repository's linter config holds the file-length - limit, the script uses that limit and counts lines the way that linter counts. - Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and - otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 - source files per directory and the limit at 13. The first line of the output - names the limit that applied; with the defaults it also prints the setting that - makes the repository's linter hold the limit. `--tsv` writes findings as rows. - The script never edits a file. + (`package-size`). Where the repository's linter config names a file-length rule, + the linter holds file length and the script measures none. Otherwise a + `.ldd-lint.yaml` at the repository root sets the limits, and otherwise the + plugin's defaults apply: 450 code lines per file, a warning at 8 source files per + directory and the limit at 13. The first line of the output names the source of + the limit; with the defaults it also prints a linter setting to propose to the + repository's owners. `--tsv` writes findings as rows. The script never edits a + file. ### Changed - **One set of thresholds.** The skills and routing tables name the same limits the - gap linter applies: the repository linter's limit where it sets one, else - 450 code lines per file and 8 and 13 files per directory. The package-size + gap linter applies: the repository linter's rule where it has one, else 450 + code lines per file and 8 and 13 files per directory. The package-size procedure runs the gap linter in place of a hand-written `find` command. ### Removed diff --git a/go-linter-driven-development/scripts/ldd-detect_test.sh b/go-linter-driven-development/scripts/ldd-detect_test.sh index 06df2052..7758f66f 100644 --- a/go-linter-driven-development/scripts/ldd-detect_test.sh +++ b/go-linter-driven-development/scripts/ldd-detect_test.sh @@ -142,19 +142,19 @@ questions_in_rules() { # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="go" FX_NATIVE="go" use_row() { + FX_LINTER_CONFIG='.golangci.yaml' + FX_LINTER_COMMENT='#' FX_EXT='.go' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -202,17 +202,12 @@ GO fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } -# The bare-number form; the generic plugin's Go row writes the map form. -fx_write_linter_config() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [%s]\n - name: exported\n' "$1" > "$REPO/.golangci.yaml" +fx_write_linter_config() { + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: 400}]\n' > "$REPO/.golangci.yaml" } -fx_write_linter_config_code() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" -} - -fx_write_linter_config_unreadable() { - printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" +fx_write_linter_config_other() { + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: argument-limit\n arguments: [4]\n' > "$REPO/.golangci.yaml" } # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- diff --git a/go-linter-driven-development/scripts/ldd-lint.sh b/go-linter-driven-development/scripts/ldd-lint.sh index 8ce0a856..55029bde 100755 --- a/go-linter-driven-development/scripts/ldd-lint.sh +++ b/go-linter-driven-development/scripts/ldd-lint.sh @@ -3,8 +3,8 @@ # # Measures the design triggers no linter holds, or that the repository's linter # does not hold yet: lines per source file and source files per directory. It -# never edits a file. Where the repository's linter holds a limit, that limit -# is the one applied, counted the way that linter counts. +# never edits a file. Where the repository's linter has a file-length rule, the +# linter owns file length and this script leaves it alone. # # Usage: bash scripts/ldd-lint.sh [options] # file-length source files over the file-length limit @@ -20,18 +20,17 @@ # know (Go block below) # --test-re test-file pattern over relative paths # -# Thresholds, per language, first that applies — the "threshold:" line says -# which one a run used: -# 1. the repository's linter config holds the rule: its limit, counted the -# way that linter counts (file-length only; no linter counts files per -# directory); a config the table cannot read is named, then skipped -# 2. .ldd-lint.yaml at the repository root: file_max_lines, -# package_warn_files, package_max_files, each a positive integer -# 3. the plugin's defaults: 450 code lines per file (blank and -# comment-only lines not counted); a warning at 8 source files per -# directory and the limit at 13; for file-length, the line also +# Thresholds, per language — the "threshold:" line says which one a run used: +# file-length: when a linter config at the repository root names a +# file-length rule, the linter holds it: the line names the rule and the +# script measures nothing more. Else the limit is file_max_lines in +# .ldd-lint.yaml, else the plugin's default, 450 code lines (blank and +# comment-only lines are not counted); with the default, the line also # prints the linter setting that would hold the limit, as a proposal for # the repository's owners: the script and its caller never edit a config +# package-size: no linter counts files per directory. package_warn_files and +# package_max_files in .ldd-lint.yaml, else the plugin's defaults: a +# warning at 8 source files per directory and the limit at 13 # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -196,9 +195,9 @@ lang_is_test() { # ==================== linter table ==================== # What the gap linter needs to know per language id: how a comment is written, -# which source files a directory count leaves out, and where the repository's -# own linter sets a file-length limit. The same table is rendered into every -# plugin, so two plugins measure one file the same way. +# which source files a directory count leaves out, and whether the +# repository's own linter holds the file-length limit. The same table is +# rendered into every plugin, so two plugins measure one file the same way. # residue-exempt: the table names every language's linter and config file on # purpose, the same in every rendering. # @@ -206,11 +205,12 @@ lang_is_test() { # lint_package_skip_re ERE over a relative path: source files a # directory count leaves out (an index module, # a package marker, generated stubs); "" for none -# lint_file_length_config one line, from the repository root: -# covered TAB TAB TAB -# TAB -# unreadable TAB -# none +# lint_file_length_rule " in " when a linter config +# at the repository root names a file-length +# rule on a line that is not a comment, else +# nothing. A named rule belongs to the linter: +# its limit, its counting and whether it is on +# are the repository's decision. # lint_file_length_snippet the linter setting that would hold the # limit, or the .ldd-lint.yaml entry when no # linter rule is known: text to propose @@ -232,114 +232,36 @@ lint_package_skip_re() { return 0 } -# _lint_window — the key's line and the lines after it, up to -# the next rule entry or 15 lines, joined by spaces; one window per match. The -# patterns reach awk as strings, so they use bracket expressions, never -# backslash escapes, which awk -v would rewrite. -_lint_window() { - awk -v key="$1" -v q="'" ' - BEGIN { - entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" - qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" - } - function flush() { if (w != "") print w; w = ""; n = 0 } - { - if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() - if ($0 ~ key) { flush(); n = 1; w = $0; next } - if (n > 0) { w = w " " $0; if (++n > 15) flush() } - } - END { flush() } - ' "$2" -} - -# _lint_number — the first match's digits -_lint_number() { - printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' -} - -# _lint_off — exit 0 when the rule in the window is switched off -_lint_off() { - printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" -} - -_lint_config_go() { - local f w max sc sb unread="" - for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do - [[ -f "$f" ]] || continue - grep -q 'file-length-limit' "$f" 2>/dev/null || continue - case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac - while IFS= read -r w; do - [[ "$w" =~ disabled:[[:space:]]*true ]] && continue - max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") - if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi - (( max > 0 )) || continue - sc=0; sb=0 - [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") - done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' -} - -_lint_config_eslint() { - local f w max sc sb - for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ - .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do +# _lint_names ... — print the first file that matches the pattern +# on a line that is not a comment +_lint_names() { + local re="$1" f; shift + for f in "$@"; do [[ -f "$f" ]] || continue - grep -q 'max-lines' "$f" 2>/dev/null || continue - # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") - printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" - return 0 - done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") - # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") - sc=0; sb=0 - [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") + grep -vE '^[[:space:]]*(#|//|;)' -- "$f" 2>/dev/null | grep -qE -- "$re" && { printf '%s' "$f"; return 0; } done - printf 'none\n' + return 1 } -_lint_config_python() { - local f line max unread="" - for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do - [[ -f "$f" ]] || continue - line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) - [[ -n "$line" ]] || continue - max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") - if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi - (( max > 0 )) || continue - # pylint counts every line of the module. - printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" - return 0 - done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' -} - -lint_file_length_config() { +lint_file_length_rule() { + local f case "$1" in - go) _lint_config_go ;; - typescript|javascript) _lint_config_eslint ;; - python) _lint_config_python ;; - *) printf 'none\n' ;; + go) + f=$(_lint_names 'file-length-limit' .golangci.yaml .golangci.yml .golangci.toml .golangci.json) \ + && printf 'revive file-length-limit in %s' "$f" ;; + typescript|javascript) + local configs=(eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml) + if f=$(_lint_names 'sonarjs/max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'sonarjs/max-lines in %s' "$f" + elif f=$(_lint_names '(^|[^-/a-zA-Z])max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'max-lines in %s' "$f" + fi ;; + python) + f=$(_lint_names '^[[:space:]]*max-module-lines[[:space:]]*=' pyproject.toml .pylintrc pylintrc setup.cfg tox.ini) \ + && printf 'pylint max-module-lines in %s' "$f" ;; esac + return 0 } lint_file_length_snippet() { @@ -415,8 +337,8 @@ measured() { } # ---------- file-length ---------- -# COUNT_AWK prints a file's line count under the style's comment syntax: -# skipc drops comment-only lines (block comments included), skipb blank lines. +# COUNT_AWK prints a file's code lines under the style's comment syntax: blank +# lines and comment-only lines, block comments included, are not counted. COUNT_AWK=' function comment_only(t, rest, i) { if (inblock) { @@ -437,45 +359,31 @@ function comment_only(t, rest, i) { } { t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) - if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } - c = comment_only(t) - if (c && skipc) next + if (t == "" || comment_only(t)) next n++ } END { print n + 0 } ' run_file_length() { # — every measured file of that language - local id="$1" cfg status max sc sb label src unit f n o - cfg=$(lint_file_length_config "$id") - IFS=$'\t' read -r status max sc sb label <<< "$cfg" - if [[ "$status" == "unreadable" ]]; then - # an unreadable row carries the rule and file in its second field - say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" - status=none + local id="$1" rule max src f n o + rule=$(lint_file_length_rule "$id") + if [[ -n "$rule" ]]; then + say "threshold: $id file length — held by $rule; the repository's linter reports long files, so this script measures none" + return 0 fi - if [[ "$status" == "covered" ]]; then - src="$label" - say "threshold: $id file length — covered by $label (limit $max)" + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else - sc=1; sb=1 - o=$(override file_max_lines) || exit 2 - if [[ -n "$o" ]]; then - max="$o"; src="$OVERRIDE_FILE" - say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" - else - max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" - fi + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi - unit="lines" - (( sc && sb )) && unit="code lines" - (( sc && ! sb )) && unit="lines without comments" - (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") - (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + n=$(awk -v style="$(lint_comment_style "$id")" "$COUNT_AWK" "./$f") + (( n > max )) && finding "$f" "$n code lines (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } diff --git a/go-linter-driven-development/scripts/ldd-lint_test.sh b/go-linter-driven-development/scripts/ldd-lint_test.sh index 5f674ed8..3201b7e9 100644 --- a/go-linter-driven-development/scripts/ldd-lint_test.sh +++ b/go-linter-driven-development/scripts/ldd-lint_test.sh @@ -106,19 +106,19 @@ fill_dir() { # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="go" FX_NATIVE="go" use_row() { + FX_LINTER_CONFIG='.golangci.yaml' + FX_LINTER_COMMENT='#' FX_EXT='.go' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -166,17 +166,12 @@ GO fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } -# The bare-number form; the generic plugin's Go row writes the map form. -fx_write_linter_config() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [%s]\n - name: exported\n' "$1" > "$REPO/.golangci.yaml" +fx_write_linter_config() { + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: 400}]\n' > "$REPO/.golangci.yaml" } -fx_write_linter_config_code() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" -} - -fx_write_linter_config_unreadable() { - printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" +fx_write_linter_config_other() { + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: argument-limit\n arguments: [4]\n' > "$REPO/.golangci.yaml" } # ========================= cases ========================= @@ -223,31 +218,27 @@ run_lint file-length expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish -begin "a linter that skips comments and blank lines is counted its way" -if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish -else - write_src "src/a$FX_EXT" 90 20 20 - run_lint file-length - expect_exit 0 && expect_has "covered by" && expect_no_finding && ok - finish -fi +begin "a file-length rule in the linter config hands file length to the linter" +fx_write_linter_config +write_src "src/over$FX_EXT" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_has "held by " && expect_no_finding && ok +finish -begin "the repository's linter config holds the limit and is counted its way" -fx_write_linter_config 100 -write_src "src/a$FX_EXT" 90 20 +begin "a similar rule of another name leaves file length to the gap linter" +fx_write_linter_config_other +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) run_lint file-length -expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok finish -begin "a linter config whose limit cannot be read is named, and the next source applies" -if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish -else - write_src "src/over$FX_EXT" $((MAX_LINES + 1)) - run_lint file-length - expect_exit 0 && expect_has "its limit cannot be read here" \ - && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok - finish -fi +begin "a file-length rule that is commented out leaves file length to the gap linter" +fx_write_linter_config +sed "s|^|$FX_LINTER_COMMENT |" "$REPO/$FX_LINTER_CONFIG" > "$REPO/cfg.tmp" && mv "$REPO/cfg.tmp" "$REPO/$FX_LINTER_CONFIG" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish begin ".ldd-lint.yaml sets the limit when the linter holds none" printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" diff --git a/go-linter-driven-development/scripts/ldd-scope_test.sh b/go-linter-driven-development/scripts/ldd-scope_test.sh index 7426136a..302851d3 100755 --- a/go-linter-driven-development/scripts/ldd-scope_test.sh +++ b/go-linter-driven-development/scripts/ldd-scope_test.sh @@ -118,19 +118,19 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="go" FX_NATIVE="go" use_row() { + FX_LINTER_CONFIG='.golangci.yaml' + FX_LINTER_COMMENT='#' FX_EXT='.go' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -178,17 +178,12 @@ GO fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } -# The bare-number form; the generic plugin's Go row writes the map form. -fx_write_linter_config() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [%s]\n - name: exported\n' "$1" > "$REPO/.golangci.yaml" +fx_write_linter_config() { + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: 400}]\n' > "$REPO/.golangci.yaml" } -fx_write_linter_config_code() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" -} - -fx_write_linter_config_unreadable() { - printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" +fx_write_linter_config_other() { + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: argument-limit\n arguments: [4]\n' > "$REPO/.golangci.yaml" } # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- diff --git a/go-linter-driven-development/skills/refactoring/reference.md b/go-linter-driven-development/skills/refactoring/reference.md index 963a8f8d..1e07c47a 100644 --- a/go-linter-driven-development/skills/refactoring/reference.md +++ b/go-linter-driven-development/skills/refactoring/reference.md @@ -66,7 +66,7 @@ decomposition): "Multi-rule procedures", below. ## File and package routing -**revive `file-length-limit`, or a file over 450 code lines where the repository sets no limit** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: +**revive `file-length-limit`, or a file over 450 code lines where the repository's config has no such rule** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them, or names the linter rule that holds the limit: | File pattern | Action | |---|---| diff --git a/lang/generic/passthrough/CHANGELOG.md b/lang/generic/passthrough/CHANGELOG.md index 4cf64e87..384e96a3 100644 --- a/lang/generic/passthrough/CHANGELOG.md +++ b/lang/generic/passthrough/CHANGELOG.md @@ -9,20 +9,20 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio - **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` measures lines per source file (`file-length`) and source files per directory - (`package-size`). Where the repository's linter config holds the file-length - limit, the script uses that limit and counts lines the way that linter counts. - Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and - otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 - source files per directory and the limit at 13. The first line of the output - names the limit that applied; with the defaults it also prints the setting that - makes the repository's linter hold the limit. `--tsv` writes findings as rows. - The script never edits a file. + (`package-size`). Where the repository's linter config names a file-length rule, + the linter holds file length and the script measures none. Otherwise a + `.ldd-lint.yaml` at the repository root sets the limits, and otherwise the + plugin's defaults apply: 450 code lines per file, a warning at 8 source files per + directory and the limit at 13. The first line of the output names the source of + the limit; with the defaults it also prints a linter setting to propose to the + repository's owners. `--tsv` writes findings as rows. The script never edits a + file. ### Changed - **One set of thresholds.** The skills and routing tables name the same limits the - gap linter applies: the repository linter's limit where it sets one, else - 450 code lines per file and 8 and 13 files per directory. The package-size + gap linter applies: the repository linter's rule where it has one, else 450 + code lines per file and 8 and 13 files per directory. The package-size procedure runs the gap linter in place of a hand-written `find` command. ### Fixed diff --git a/lang/generic/scripts/ldd-fixture.sh b/lang/generic/scripts/ldd-fixture.sh index d33afdc9..5cf2630f 100644 --- a/lang/generic/scripts/ldd-fixture.sh +++ b/lang/generic/scripts/ldd-fixture.sh @@ -19,21 +19,21 @@ # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="go python" FX_NATIVE="go python d typescript" use_row() { case "$1" in go) + FX_LINTER_CONFIG='.golangci.yml' + FX_LINTER_COMMENT='#' FX_EXT='.go' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -44,6 +44,8 @@ use_row() { FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' ;; python) + FX_LINTER_CONFIG='.pylintrc' + FX_LINTER_COMMENT=';' FX_EXT='.py' FX_LINE_COMMENT='# note' FX_BLOCK_OPEN='' @@ -129,23 +131,16 @@ fx_test_file() { esac } -fx_write_linter_config() { # +fx_write_linter_config() { case "$ROW" in - go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" ;; - python) printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" ;; + go) printf 'linters-settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: 400\n' > "$REPO/.golangci.yml" ;; + python) printf '[MAIN]\nmax-module-lines=400\n' > "$REPO/.pylintrc" ;; esac } -fx_write_linter_config_code() { # +fx_write_linter_config_other() { case "$ROW" in - go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" ;; - python) return 1 ;; - esac -} - -fx_write_linter_config_unreadable() { - case "$ROW" in - go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; - python) printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" ;; + go) printf 'linters-settings:\n revive:\n rules:\n - name: argument-limit\n arguments: [4]\n' > "$REPO/.golangci.yml" ;; + python) printf '[MAIN]\nmax-args=4\n' > "$REPO/.pylintrc" ;; esac } diff --git a/lang/go/passthrough/CHANGELOG.md b/lang/go/passthrough/CHANGELOG.md index bcfe2328..5f4269d4 100644 --- a/lang/go/passthrough/CHANGELOG.md +++ b/lang/go/passthrough/CHANGELOG.md @@ -9,20 +9,20 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio - **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` measures lines per source file (`file-length`) and source files per directory - (`package-size`). Where the repository's linter config holds the file-length - limit, the script uses that limit and counts lines the way that linter counts. - Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and - otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 - source files per directory and the limit at 13. The first line of the output - names the limit that applied; with the defaults it also prints the setting that - makes the repository's linter hold the limit. `--tsv` writes findings as rows. - The script never edits a file. + (`package-size`). Where the repository's linter config names a file-length rule, + the linter holds file length and the script measures none. Otherwise a + `.ldd-lint.yaml` at the repository root sets the limits, and otherwise the + plugin's defaults apply: 450 code lines per file, a warning at 8 source files per + directory and the limit at 13. The first line of the output names the source of + the limit; with the defaults it also prints a linter setting to propose to the + repository's owners. `--tsv` writes findings as rows. The script never edits a + file. ### Changed - **One set of thresholds.** The skills and routing tables name the same limits the - gap linter applies: the repository linter's limit where it sets one, else - 450 code lines per file and 8 and 13 files per directory. The package-size + gap linter applies: the repository linter's rule where it has one, else 450 + code lines per file and 8 and 13 files per directory. The package-size procedure runs the gap linter in place of a hand-written `find` command. ### Removed diff --git a/lang/go/scripts/ldd-fixture.sh b/lang/go/scripts/ldd-fixture.sh index 4ff4943f..142614eb 100644 --- a/lang/go/scripts/ldd-fixture.sh +++ b/lang/go/scripts/ldd-fixture.sh @@ -19,19 +19,19 @@ # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="go" FX_NATIVE="go" use_row() { + FX_LINTER_CONFIG='.golangci.yaml' + FX_LINTER_COMMENT='#' FX_EXT='.go' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -79,15 +79,10 @@ GO fx_test_file() { printf '%s/m%s_test.go' "$1" "$2"; } -# The bare-number form; the generic plugin's Go row writes the map form. -fx_write_linter_config() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [%s]\n - name: exported\n' "$1" > "$REPO/.golangci.yaml" +fx_write_linter_config() { + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: 400}]\n' > "$REPO/.golangci.yaml" } -fx_write_linter_config_code() { # - printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" -} - -fx_write_linter_config_unreadable() { - printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" +fx_write_linter_config_other() { + printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: argument-limit\n arguments: [4]\n' > "$REPO/.golangci.yaml" } diff --git a/lang/go/skills/refactoring/file-and-package-routing.md b/lang/go/skills/refactoring/file-and-package-routing.md index ac2a2fc9..bdaad44a 100644 --- a/lang/go/skills/refactoring/file-and-package-routing.md +++ b/lang/go/skills/refactoring/file-and-package-routing.md @@ -1,4 +1,4 @@ -**revive `file-length-limit`, or a file over {{.FileMaxLines}} code lines where the repository sets no limit** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: +**revive `file-length-limit`, or a file over {{.FileMaxLines}} code lines where the repository's config has no such rule** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them, or names the linter rule that holds the limit: | File pattern | Action | |---|---| diff --git a/lang/python/passthrough/CHANGELOG.md b/lang/python/passthrough/CHANGELOG.md index 884c2d4a..e2965821 100644 --- a/lang/python/passthrough/CHANGELOG.md +++ b/lang/python/passthrough/CHANGELOG.md @@ -9,20 +9,20 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio - **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` measures lines per source file (`file-length`) and source files per directory - (`package-size`). Where the repository's linter config holds the file-length - limit, the script uses that limit and counts lines the way that linter counts. - Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and - otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 - source files per directory and the limit at 13. The first line of the output - names the limit that applied; with the defaults it also prints the setting that - makes the repository's linter hold the limit. `--tsv` writes findings as rows. - The script never edits a file. + (`package-size`). Where the repository's linter config names a file-length rule, + the linter holds file length and the script measures none. Otherwise a + `.ldd-lint.yaml` at the repository root sets the limits, and otherwise the + plugin's defaults apply: 450 code lines per file, a warning at 8 source files per + directory and the limit at 13. The first line of the output names the source of + the limit; with the defaults it also prints a linter setting to propose to the + repository's owners. `--tsv` writes findings as rows. The script never edits a + file. ### Changed - **One set of thresholds.** The skills and routing tables name the same limits the - gap linter applies: the repository linter's limit where it sets one, else - 450 code lines per file and 8 and 13 files per directory. The package-size + gap linter applies: the repository linter's rule where it has one, else 450 + code lines per file and 8 and 13 files per directory. The package-size procedure runs the gap linter in place of a hand-written `find` command. ### Fixed diff --git a/lang/python/scripts/ldd-fixture.sh b/lang/python/scripts/ldd-fixture.sh index 5c5ec55a..b3338f2e 100644 --- a/lang/python/scripts/ldd-fixture.sh +++ b/lang/python/scripts/ldd-fixture.sh @@ -19,19 +19,19 @@ # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="python" FX_NATIVE="python" use_row() { + FX_LINTER_CONFIG='pyproject.toml' + FX_LINTER_COMMENT='#' FX_EXT='.py' FX_LINE_COMMENT='# note' FX_BLOCK_OPEN='' @@ -73,13 +73,10 @@ PY fx_test_file() { printf '%s/test_m%s.py' "$1" "$2"; } -fx_write_linter_config() { # - printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" +fx_write_linter_config() { + printf '[tool.pylint.format]\nmax-module-lines = 400\n' > "$REPO/pyproject.toml" } -# pylint counts every line; ruff has no file-length rule. -fx_write_linter_config_code() { return 1; } - -fx_write_linter_config_unreadable() { - printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" +fx_write_linter_config_other() { + printf '[tool.pylint.design]\nmax-args = 4\n' > "$REPO/pyproject.toml" } diff --git a/lang/python/skills/refactoring/file-and-package-routing.md b/lang/python/skills/refactoring/file-and-package-routing.md index 10b430a7..13790fca 100644 --- a/lang/python/skills/refactoring/file-and-package-routing.md +++ b/lang/python/skills/refactoring/file-and-package-routing.md @@ -1,4 +1,4 @@ -**A module over the file-length limit** — pylint's `max-module-lines` where the repository sets it (ruff has no file-length rule), else {{.FileMaxLines}} code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: +**A module over the file-length limit** — pylint's `max-module-lines` where the repository sets it (ruff has no file-length rule), else {{.FileMaxLines}} code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them, or names the linter rule that holds the limit: | File pattern | Action | |---|---| diff --git a/lang/ts-react/passthrough/CHANGELOG.md b/lang/ts-react/passthrough/CHANGELOG.md index cad904eb..69e73fd0 100644 --- a/lang/ts-react/passthrough/CHANGELOG.md +++ b/lang/ts-react/passthrough/CHANGELOG.md @@ -9,20 +9,20 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio - **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` measures lines per source file (`file-length`) and source files per directory - (`package-size`). Where the repository's linter config holds the file-length - limit, the script uses that limit and counts lines the way that linter counts. - Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and - otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 - source files per directory and the limit at 13. The first line of the output - names the limit that applied; with the defaults it also prints the setting that - makes the repository's linter hold the limit. `--tsv` writes findings as rows. - The script never edits a file. + (`package-size`). Where the repository's linter config names a file-length rule, + the linter holds file length and the script measures none. Otherwise a + `.ldd-lint.yaml` at the repository root sets the limits, and otherwise the + plugin's defaults apply: 450 code lines per file, a warning at 8 source files per + directory and the limit at 13. The first line of the output names the source of + the limit; with the defaults it also prints a linter setting to propose to the + repository's owners. `--tsv` writes findings as rows. The script never edits a + file. ### Changed - **One set of thresholds.** The skills and routing tables name the same limits the - gap linter applies: the repository linter's limit where it sets one, else - 450 code lines per file and 8 and 13 files per directory. The package-size + gap linter applies: the repository linter's rule where it has one, else 450 + code lines per file and 8 and 13 files per directory. The package-size procedure runs the gap linter in place of a hand-written `find` command. ### Fixed diff --git a/lang/ts-react/scripts/ldd-fixture.sh b/lang/ts-react/scripts/ldd-fixture.sh index 055fc6b8..00ae60a8 100644 --- a/lang/ts-react/scripts/ldd-fixture.sh +++ b/lang/ts-react/scripts/ldd-fixture.sh @@ -19,15 +19,13 @@ # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="typescript" FX_NATIVE="typescript" @@ -35,6 +33,8 @@ FX_NATIVE="typescript" # names for the files it creates from it, and `fresh.ts*` is no file name. # Every file the row writes is a `.ts` file, so both globs select the same set. use_row() { + FX_LINTER_CONFIG='eslint.config.mjs' + FX_LINTER_COMMENT='//' FX_EXT='.ts' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -76,16 +76,11 @@ TS fx_test_file() { printf '%s/m%s.test.ts' "$1" "$2"; } -# ESLint's own max-lines counts every line unless told to skip comments. -fx_write_linter_config() { # - printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +fx_write_linter_config() { + printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: 400 }],\n },\n}]\n" > "$REPO/eslint.config.mjs" } -# sonarjs/max-lines counts only code lines. -fx_write_linter_config_code() { # - printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: %s }],\n 'max-lines-per-function': ['error', 50],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +# max-lines-per-function limits a function, not a file. +fx_write_linter_config_other() { + printf "export default [{\n rules: {\n 'max-lines-per-function': ['error', 50],\n 'sonarjs/max-lines-per-function': ['error', { maximum: 50 }],\n },\n}]\n" > "$REPO/eslint.config.mjs" } - -# Every ESLint config form resolves: a rule named without a limit takes -# ESLint's documented default. -fx_write_linter_config_unreadable() { return 1; } diff --git a/lang/ts-react/skills/refactoring/file-and-package-routing.md b/lang/ts-react/skills/refactoring/file-and-package-routing.md index 350ef1ff..3b32f433 100644 --- a/lang/ts-react/skills/refactoring/file-and-package-routing.md +++ b/lang/ts-react/skills/refactoring/file-and-package-routing.md @@ -1,4 +1,4 @@ -**`sonarjs/max-lines`, or a module over {{.FileMaxLines}} code lines where the repository sets no limit** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: +**`sonarjs/max-lines`, or a module over {{.FileMaxLines}} code lines where the repository's config has no such rule** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them, or names the linter rule that holds the limit: | File pattern | Action | |---|---| diff --git a/linter-driven-development/CHANGELOG.md b/linter-driven-development/CHANGELOG.md index 4cf64e87..384e96a3 100644 --- a/linter-driven-development/CHANGELOG.md +++ b/linter-driven-development/CHANGELOG.md @@ -9,20 +9,20 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio - **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` measures lines per source file (`file-length`) and source files per directory - (`package-size`). Where the repository's linter config holds the file-length - limit, the script uses that limit and counts lines the way that linter counts. - Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and - otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 - source files per directory and the limit at 13. The first line of the output - names the limit that applied; with the defaults it also prints the setting that - makes the repository's linter hold the limit. `--tsv` writes findings as rows. - The script never edits a file. + (`package-size`). Where the repository's linter config names a file-length rule, + the linter holds file length and the script measures none. Otherwise a + `.ldd-lint.yaml` at the repository root sets the limits, and otherwise the + plugin's defaults apply: 450 code lines per file, a warning at 8 source files per + directory and the limit at 13. The first line of the output names the source of + the limit; with the defaults it also prints a linter setting to propose to the + repository's owners. `--tsv` writes findings as rows. The script never edits a + file. ### Changed - **One set of thresholds.** The skills and routing tables name the same limits the - gap linter applies: the repository linter's limit where it sets one, else - 450 code lines per file and 8 and 13 files per directory. The package-size + gap linter applies: the repository linter's rule where it has one, else 450 + code lines per file and 8 and 13 files per directory. The package-size procedure runs the gap linter in place of a hand-written `find` command. ### Fixed diff --git a/linter-driven-development/scripts/ldd-detect_test.sh b/linter-driven-development/scripts/ldd-detect_test.sh index 7f8c7f3e..15d54081 100644 --- a/linter-driven-development/scripts/ldd-detect_test.sh +++ b/linter-driven-development/scripts/ldd-detect_test.sh @@ -142,21 +142,21 @@ questions_in_rules() { # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="go python" FX_NATIVE="go python d typescript" use_row() { case "$1" in go) + FX_LINTER_CONFIG='.golangci.yml' + FX_LINTER_COMMENT='#' FX_EXT='.go' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -167,6 +167,8 @@ use_row() { FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' ;; python) + FX_LINTER_CONFIG='.pylintrc' + FX_LINTER_COMMENT=';' FX_EXT='.py' FX_LINE_COMMENT='# note' FX_BLOCK_OPEN='' @@ -252,24 +254,17 @@ fx_test_file() { esac } -fx_write_linter_config() { # +fx_write_linter_config() { case "$ROW" in - go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" ;; - python) printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" ;; + go) printf 'linters-settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: 400\n' > "$REPO/.golangci.yml" ;; + python) printf '[MAIN]\nmax-module-lines=400\n' > "$REPO/.pylintrc" ;; esac } -fx_write_linter_config_code() { # +fx_write_linter_config_other() { case "$ROW" in - go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" ;; - python) return 1 ;; - esac -} - -fx_write_linter_config_unreadable() { - case "$ROW" in - go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; - python) printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" ;; + go) printf 'linters-settings:\n revive:\n rules:\n - name: argument-limit\n arguments: [4]\n' > "$REPO/.golangci.yml" ;; + python) printf '[MAIN]\nmax-args=4\n' > "$REPO/.pylintrc" ;; esac } diff --git a/linter-driven-development/scripts/ldd-lint.sh b/linter-driven-development/scripts/ldd-lint.sh index eec28c96..72a2a227 100755 --- a/linter-driven-development/scripts/ldd-lint.sh +++ b/linter-driven-development/scripts/ldd-lint.sh @@ -3,8 +3,8 @@ # # Measures the design triggers no linter holds, or that the repository's linter # does not hold yet: lines per source file and source files per directory. It -# never edits a file. Where the repository's linter holds a limit, that limit -# is the one applied, counted the way that linter counts. +# never edits a file. Where the repository's linter has a file-length rule, the +# linter owns file length and this script leaves it alone. # # Usage: bash scripts/ldd-lint.sh [options] # file-length source files over the file-length limit @@ -20,18 +20,17 @@ # know (any-language block below) # --test-re test-file pattern over relative paths # -# Thresholds, per language, first that applies — the "threshold:" line says -# which one a run used: -# 1. the repository's linter config holds the rule: its limit, counted the -# way that linter counts (file-length only; no linter counts files per -# directory); a config the table cannot read is named, then skipped -# 2. .ldd-lint.yaml at the repository root: file_max_lines, -# package_warn_files, package_max_files, each a positive integer -# 3. the plugin's defaults: 450 code lines per file (blank and -# comment-only lines not counted); a warning at 8 source files per -# directory and the limit at 13; for file-length, the line also +# Thresholds, per language — the "threshold:" line says which one a run used: +# file-length: when a linter config at the repository root names a +# file-length rule, the linter holds it: the line names the rule and the +# script measures nothing more. Else the limit is file_max_lines in +# .ldd-lint.yaml, else the plugin's default, 450 code lines (blank and +# comment-only lines are not counted); with the default, the line also # prints the linter setting that would hold the limit, as a proposal for # the repository's owners: the script and its caller never edit a config +# package-size: no linter counts files per directory. package_warn_files and +# package_max_files in .ldd-lint.yaml, else the plugin's defaults: a +# warning at 8 source files per directory and the limit at 13 # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -312,9 +311,9 @@ lang_is_test() { # ==================== linter table ==================== # What the gap linter needs to know per language id: how a comment is written, -# which source files a directory count leaves out, and where the repository's -# own linter sets a file-length limit. The same table is rendered into every -# plugin, so two plugins measure one file the same way. +# which source files a directory count leaves out, and whether the +# repository's own linter holds the file-length limit. The same table is +# rendered into every plugin, so two plugins measure one file the same way. # residue-exempt: the table names every language's linter and config file on # purpose, the same in every rendering. # @@ -322,11 +321,12 @@ lang_is_test() { # lint_package_skip_re ERE over a relative path: source files a # directory count leaves out (an index module, # a package marker, generated stubs); "" for none -# lint_file_length_config one line, from the repository root: -# covered TAB TAB TAB -# TAB -# unreadable TAB -# none +# lint_file_length_rule " in " when a linter config +# at the repository root names a file-length +# rule on a line that is not a comment, else +# nothing. A named rule belongs to the linter: +# its limit, its counting and whether it is on +# are the repository's decision. # lint_file_length_snippet the linter setting that would hold the # limit, or the .ldd-lint.yaml entry when no # linter rule is known: text to propose @@ -348,114 +348,36 @@ lint_package_skip_re() { return 0 } -# _lint_window — the key's line and the lines after it, up to -# the next rule entry or 15 lines, joined by spaces; one window per match. The -# patterns reach awk as strings, so they use bracket expressions, never -# backslash escapes, which awk -v would rewrite. -_lint_window() { - awk -v key="$1" -v q="'" ' - BEGIN { - entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" - qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" - } - function flush() { if (w != "") print w; w = ""; n = 0 } - { - if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() - if ($0 ~ key) { flush(); n = 1; w = $0; next } - if (n > 0) { w = w " " $0; if (++n > 15) flush() } - } - END { flush() } - ' "$2" -} - -# _lint_number — the first match's digits -_lint_number() { - printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' -} - -# _lint_off — exit 0 when the rule in the window is switched off -_lint_off() { - printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" -} - -_lint_config_go() { - local f w max sc sb unread="" - for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do +# _lint_names ... — print the first file that matches the pattern +# on a line that is not a comment +_lint_names() { + local re="$1" f; shift + for f in "$@"; do [[ -f "$f" ]] || continue - grep -q 'file-length-limit' "$f" 2>/dev/null || continue - case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac - while IFS= read -r w; do - [[ "$w" =~ disabled:[[:space:]]*true ]] && continue - max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") - if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi - (( max > 0 )) || continue - sc=0; sb=0 - [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") + grep -vE '^[[:space:]]*(#|//|;)' -- "$f" 2>/dev/null | grep -qE -- "$re" && { printf '%s' "$f"; return 0; } done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' + return 1 } -_lint_config_eslint() { - local f w max sc sb - for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ - .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do - [[ -f "$f" ]] || continue - grep -q 'max-lines' "$f" 2>/dev/null || continue - # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") - printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" - return 0 - done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") - # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") - sc=0; sb=0 - [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") - done - printf 'none\n' -} - -_lint_config_python() { - local f line max unread="" - for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do - [[ -f "$f" ]] || continue - line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) - [[ -n "$line" ]] || continue - max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") - if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi - (( max > 0 )) || continue - # pylint counts every line of the module. - printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" - return 0 - done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' -} - -lint_file_length_config() { +lint_file_length_rule() { + local f case "$1" in - go) _lint_config_go ;; - typescript|javascript) _lint_config_eslint ;; - python) _lint_config_python ;; - *) printf 'none\n' ;; + go) + f=$(_lint_names 'file-length-limit' .golangci.yaml .golangci.yml .golangci.toml .golangci.json) \ + && printf 'revive file-length-limit in %s' "$f" ;; + typescript|javascript) + local configs=(eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml) + if f=$(_lint_names 'sonarjs/max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'sonarjs/max-lines in %s' "$f" + elif f=$(_lint_names '(^|[^-/a-zA-Z])max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'max-lines in %s' "$f" + fi ;; + python) + f=$(_lint_names '^[[:space:]]*max-module-lines[[:space:]]*=' pyproject.toml .pylintrc pylintrc setup.cfg tox.ini) \ + && printf 'pylint max-module-lines in %s' "$f" ;; esac + return 0 } lint_file_length_snippet() { @@ -531,8 +453,8 @@ measured() { } # ---------- file-length ---------- -# COUNT_AWK prints a file's line count under the style's comment syntax: -# skipc drops comment-only lines (block comments included), skipb blank lines. +# COUNT_AWK prints a file's code lines under the style's comment syntax: blank +# lines and comment-only lines, block comments included, are not counted. COUNT_AWK=' function comment_only(t, rest, i) { if (inblock) { @@ -553,45 +475,31 @@ function comment_only(t, rest, i) { } { t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) - if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } - c = comment_only(t) - if (c && skipc) next + if (t == "" || comment_only(t)) next n++ } END { print n + 0 } ' run_file_length() { # — every measured file of that language - local id="$1" cfg status max sc sb label src unit f n o - cfg=$(lint_file_length_config "$id") - IFS=$'\t' read -r status max sc sb label <<< "$cfg" - if [[ "$status" == "unreadable" ]]; then - # an unreadable row carries the rule and file in its second field - say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" - status=none + local id="$1" rule max src f n o + rule=$(lint_file_length_rule "$id") + if [[ -n "$rule" ]]; then + say "threshold: $id file length — held by $rule; the repository's linter reports long files, so this script measures none" + return 0 fi - if [[ "$status" == "covered" ]]; then - src="$label" - say "threshold: $id file length — covered by $label (limit $max)" + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else - sc=1; sb=1 - o=$(override file_max_lines) || exit 2 - if [[ -n "$o" ]]; then - max="$o"; src="$OVERRIDE_FILE" - say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" - else - max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" - fi + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi - unit="lines" - (( sc && sb )) && unit="code lines" - (( sc && ! sb )) && unit="lines without comments" - (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") - (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + n=$(awk -v style="$(lint_comment_style "$id")" "$COUNT_AWK" "./$f") + (( n > max )) && finding "$f" "$n code lines (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } diff --git a/linter-driven-development/scripts/ldd-lint_test.sh b/linter-driven-development/scripts/ldd-lint_test.sh index 3554f8ef..ad237c52 100644 --- a/linter-driven-development/scripts/ldd-lint_test.sh +++ b/linter-driven-development/scripts/ldd-lint_test.sh @@ -106,21 +106,21 @@ fill_dir() { # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="go python" FX_NATIVE="go python d typescript" use_row() { case "$1" in go) + FX_LINTER_CONFIG='.golangci.yml' + FX_LINTER_COMMENT='#' FX_EXT='.go' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -131,6 +131,8 @@ use_row() { FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' ;; python) + FX_LINTER_CONFIG='.pylintrc' + FX_LINTER_COMMENT=';' FX_EXT='.py' FX_LINE_COMMENT='# note' FX_BLOCK_OPEN='' @@ -216,24 +218,17 @@ fx_test_file() { esac } -fx_write_linter_config() { # +fx_write_linter_config() { case "$ROW" in - go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" ;; - python) printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" ;; + go) printf 'linters-settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: 400\n' > "$REPO/.golangci.yml" ;; + python) printf '[MAIN]\nmax-module-lines=400\n' > "$REPO/.pylintrc" ;; esac } -fx_write_linter_config_code() { # +fx_write_linter_config_other() { case "$ROW" in - go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" ;; - python) return 1 ;; - esac -} - -fx_write_linter_config_unreadable() { - case "$ROW" in - go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; - python) printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" ;; + go) printf 'linters-settings:\n revive:\n rules:\n - name: argument-limit\n arguments: [4]\n' > "$REPO/.golangci.yml" ;; + python) printf '[MAIN]\nmax-args=4\n' > "$REPO/.pylintrc" ;; esac } @@ -281,31 +276,27 @@ run_lint file-length expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish -begin "a linter that skips comments and blank lines is counted its way" -if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish -else - write_src "src/a$FX_EXT" 90 20 20 - run_lint file-length - expect_exit 0 && expect_has "covered by" && expect_no_finding && ok - finish -fi +begin "a file-length rule in the linter config hands file length to the linter" +fx_write_linter_config +write_src "src/over$FX_EXT" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_has "held by " && expect_no_finding && ok +finish -begin "the repository's linter config holds the limit and is counted its way" -fx_write_linter_config 100 -write_src "src/a$FX_EXT" 90 20 +begin "a similar rule of another name leaves file length to the gap linter" +fx_write_linter_config_other +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) run_lint file-length -expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok finish -begin "a linter config whose limit cannot be read is named, and the next source applies" -if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish -else - write_src "src/over$FX_EXT" $((MAX_LINES + 1)) - run_lint file-length - expect_exit 0 && expect_has "its limit cannot be read here" \ - && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok - finish -fi +begin "a file-length rule that is commented out leaves file length to the gap linter" +fx_write_linter_config +sed "s|^|$FX_LINTER_COMMENT |" "$REPO/$FX_LINTER_CONFIG" > "$REPO/cfg.tmp" && mv "$REPO/cfg.tmp" "$REPO/$FX_LINTER_CONFIG" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish begin ".ldd-lint.yaml sets the limit when the linter holds none" printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" diff --git a/linter-driven-development/scripts/ldd-scope_test.sh b/linter-driven-development/scripts/ldd-scope_test.sh index 3588efae..9af4d86b 100755 --- a/linter-driven-development/scripts/ldd-scope_test.sh +++ b/linter-driven-development/scripts/ldd-scope_test.sh @@ -118,21 +118,21 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="go python" FX_NATIVE="go python d typescript" use_row() { case "$1" in go) + FX_LINTER_CONFIG='.golangci.yml' + FX_LINTER_COMMENT='#' FX_EXT='.go' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -143,6 +143,8 @@ use_row() { FX_SUPPRESS_LINE='var cache = map[string]string{} //nolint:gochecknoglobals // TODO' ;; python) + FX_LINTER_CONFIG='.pylintrc' + FX_LINTER_COMMENT=';' FX_EXT='.py' FX_LINE_COMMENT='# note' FX_BLOCK_OPEN='' @@ -228,24 +230,17 @@ fx_test_file() { esac } -fx_write_linter_config() { # +fx_write_linter_config() { case "$ROW" in - go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: %s\n' "$1" > "$REPO/.golangci.yaml" ;; - python) printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" ;; + go) printf 'linters-settings:\n revive:\n rules:\n - name: file-length-limit\n arguments:\n - max: 400\n' > "$REPO/.golangci.yml" ;; + python) printf '[MAIN]\nmax-module-lines=400\n' > "$REPO/.pylintrc" ;; esac } -fx_write_linter_config_code() { # +fx_write_linter_config_other() { case "$ROW" in - go) printf 'version: "2"\nlinters:\n settings:\n revive:\n rules:\n - name: file-length-limit\n arguments: [{max: %s, skip-comments: true, skip-blank-lines: true}]\n' "$1" > "$REPO/.golangci.yaml" ;; - python) return 1 ;; - esac -} - -fx_write_linter_config_unreadable() { - case "$ROW" in - go) printf '[[linters.settings.revive.rules]]\nname = "file-length-limit"\narguments = [{max = 100}]\n' > "$REPO/.golangci.toml" ;; - python) printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" ;; + go) printf 'linters-settings:\n revive:\n rules:\n - name: argument-limit\n arguments: [4]\n' > "$REPO/.golangci.yml" ;; + python) printf '[MAIN]\nmax-args=4\n' > "$REPO/.pylintrc" ;; esac } diff --git a/linter-driven-development/skills/refactoring/reference.md b/linter-driven-development/skills/refactoring/reference.md index 55de1a54..d726a80e 100644 --- a/linter-driven-development/skills/refactoring/reference.md +++ b/linter-driven-development/skills/refactoring/reference.md @@ -65,7 +65,7 @@ decomposition): "Multi-rule procedures", below. ## File and package routing -**A file over the file-length limit** — the repository linter's limit where it sets one, else 450 code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: +**A file over the file-length limit** — the repository linter reports it where the linter has a file-length rule; elsewhere a file over 450 code lines is one, and `bash /../../scripts/ldd-lint.sh file-length --root .` lists them, or names the linter rule that holds the limit: | File pattern | Action | |---|---| diff --git a/python-linter-driven-development/CHANGELOG.md b/python-linter-driven-development/CHANGELOG.md index 884c2d4a..e2965821 100644 --- a/python-linter-driven-development/CHANGELOG.md +++ b/python-linter-driven-development/CHANGELOG.md @@ -9,20 +9,20 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio - **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` measures lines per source file (`file-length`) and source files per directory - (`package-size`). Where the repository's linter config holds the file-length - limit, the script uses that limit and counts lines the way that linter counts. - Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and - otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 - source files per directory and the limit at 13. The first line of the output - names the limit that applied; with the defaults it also prints the setting that - makes the repository's linter hold the limit. `--tsv` writes findings as rows. - The script never edits a file. + (`package-size`). Where the repository's linter config names a file-length rule, + the linter holds file length and the script measures none. Otherwise a + `.ldd-lint.yaml` at the repository root sets the limits, and otherwise the + plugin's defaults apply: 450 code lines per file, a warning at 8 source files per + directory and the limit at 13. The first line of the output names the source of + the limit; with the defaults it also prints a linter setting to propose to the + repository's owners. `--tsv` writes findings as rows. The script never edits a + file. ### Changed - **One set of thresholds.** The skills and routing tables name the same limits the - gap linter applies: the repository linter's limit where it sets one, else - 450 code lines per file and 8 and 13 files per directory. The package-size + gap linter applies: the repository linter's rule where it has one, else 450 + code lines per file and 8 and 13 files per directory. The package-size procedure runs the gap linter in place of a hand-written `find` command. ### Fixed diff --git a/python-linter-driven-development/scripts/ldd-detect_test.sh b/python-linter-driven-development/scripts/ldd-detect_test.sh index 3c8ef4e0..e322df7d 100644 --- a/python-linter-driven-development/scripts/ldd-detect_test.sh +++ b/python-linter-driven-development/scripts/ldd-detect_test.sh @@ -142,19 +142,19 @@ questions_in_rules() { # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="python" FX_NATIVE="python" use_row() { + FX_LINTER_CONFIG='pyproject.toml' + FX_LINTER_COMMENT='#' FX_EXT='.py' FX_LINE_COMMENT='# note' FX_BLOCK_OPEN='' @@ -196,15 +196,12 @@ PY fx_test_file() { printf '%s/test_m%s.py' "$1" "$2"; } -fx_write_linter_config() { # - printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" +fx_write_linter_config() { + printf '[tool.pylint.format]\nmax-module-lines = 400\n' > "$REPO/pyproject.toml" } -# pylint counts every line; ruff has no file-length rule. -fx_write_linter_config_code() { return 1; } - -fx_write_linter_config_unreadable() { - printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" +fx_write_linter_config_other() { + printf '[tool.pylint.design]\nmax-args = 4\n' > "$REPO/pyproject.toml" } # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- diff --git a/python-linter-driven-development/scripts/ldd-lint.sh b/python-linter-driven-development/scripts/ldd-lint.sh index c4932f7a..dc3a4419 100755 --- a/python-linter-driven-development/scripts/ldd-lint.sh +++ b/python-linter-driven-development/scripts/ldd-lint.sh @@ -3,8 +3,8 @@ # # Measures the design triggers no linter holds, or that the repository's linter # does not hold yet: lines per source file and source files per directory. It -# never edits a file. Where the repository's linter holds a limit, that limit -# is the one applied, counted the way that linter counts. +# never edits a file. Where the repository's linter has a file-length rule, the +# linter owns file length and this script leaves it alone. # # Usage: bash scripts/ldd-lint.sh [options] # file-length source files over the file-length limit @@ -20,18 +20,17 @@ # know (Python block below) # --test-re test-file pattern over relative paths # -# Thresholds, per language, first that applies — the "threshold:" line says -# which one a run used: -# 1. the repository's linter config holds the rule: its limit, counted the -# way that linter counts (file-length only; no linter counts files per -# directory); a config the table cannot read is named, then skipped -# 2. .ldd-lint.yaml at the repository root: file_max_lines, -# package_warn_files, package_max_files, each a positive integer -# 3. the plugin's defaults: 450 code lines per file (blank and -# comment-only lines not counted); a warning at 8 source files per -# directory and the limit at 13; for file-length, the line also +# Thresholds, per language — the "threshold:" line says which one a run used: +# file-length: when a linter config at the repository root names a +# file-length rule, the linter holds it: the line names the rule and the +# script measures nothing more. Else the limit is file_max_lines in +# .ldd-lint.yaml, else the plugin's default, 450 code lines (blank and +# comment-only lines are not counted); with the default, the line also # prints the linter setting that would hold the limit, as a proposal for # the repository's owners: the script and its caller never edit a config +# package-size: no linter counts files per directory. package_warn_files and +# package_max_files in .ldd-lint.yaml, else the plugin's defaults: a +# warning at 8 source files per directory and the limit at 13 # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -199,9 +198,9 @@ lang_is_test() { # ==================== linter table ==================== # What the gap linter needs to know per language id: how a comment is written, -# which source files a directory count leaves out, and where the repository's -# own linter sets a file-length limit. The same table is rendered into every -# plugin, so two plugins measure one file the same way. +# which source files a directory count leaves out, and whether the +# repository's own linter holds the file-length limit. The same table is +# rendered into every plugin, so two plugins measure one file the same way. # residue-exempt: the table names every language's linter and config file on # purpose, the same in every rendering. # @@ -209,11 +208,12 @@ lang_is_test() { # lint_package_skip_re ERE over a relative path: source files a # directory count leaves out (an index module, # a package marker, generated stubs); "" for none -# lint_file_length_config one line, from the repository root: -# covered TAB TAB TAB -# TAB -# unreadable TAB -# none +# lint_file_length_rule " in " when a linter config +# at the repository root names a file-length +# rule on a line that is not a comment, else +# nothing. A named rule belongs to the linter: +# its limit, its counting and whether it is on +# are the repository's decision. # lint_file_length_snippet the linter setting that would hold the # limit, or the .ldd-lint.yaml entry when no # linter rule is known: text to propose @@ -235,114 +235,36 @@ lint_package_skip_re() { return 0 } -# _lint_window — the key's line and the lines after it, up to -# the next rule entry or 15 lines, joined by spaces; one window per match. The -# patterns reach awk as strings, so they use bracket expressions, never -# backslash escapes, which awk -v would rewrite. -_lint_window() { - awk -v key="$1" -v q="'" ' - BEGIN { - entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" - qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" - } - function flush() { if (w != "") print w; w = ""; n = 0 } - { - if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() - if ($0 ~ key) { flush(); n = 1; w = $0; next } - if (n > 0) { w = w " " $0; if (++n > 15) flush() } - } - END { flush() } - ' "$2" -} - -# _lint_number — the first match's digits -_lint_number() { - printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' -} - -# _lint_off — exit 0 when the rule in the window is switched off -_lint_off() { - printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" -} - -_lint_config_go() { - local f w max sc sb unread="" - for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do - [[ -f "$f" ]] || continue - grep -q 'file-length-limit' "$f" 2>/dev/null || continue - case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac - while IFS= read -r w; do - [[ "$w" =~ disabled:[[:space:]]*true ]] && continue - max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") - if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi - (( max > 0 )) || continue - sc=0; sb=0 - [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") - done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' -} - -_lint_config_eslint() { - local f w max sc sb - for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ - .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do +# _lint_names ... — print the first file that matches the pattern +# on a line that is not a comment +_lint_names() { + local re="$1" f; shift + for f in "$@"; do [[ -f "$f" ]] || continue - grep -q 'max-lines' "$f" 2>/dev/null || continue - # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") - printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" - return 0 - done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") - # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") - sc=0; sb=0 - [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") + grep -vE '^[[:space:]]*(#|//|;)' -- "$f" 2>/dev/null | grep -qE -- "$re" && { printf '%s' "$f"; return 0; } done - printf 'none\n' -} - -_lint_config_python() { - local f line max unread="" - for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do - [[ -f "$f" ]] || continue - line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) - [[ -n "$line" ]] || continue - max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") - if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi - (( max > 0 )) || continue - # pylint counts every line of the module. - printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" - return 0 - done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' + return 1 } -lint_file_length_config() { +lint_file_length_rule() { + local f case "$1" in - go) _lint_config_go ;; - typescript|javascript) _lint_config_eslint ;; - python) _lint_config_python ;; - *) printf 'none\n' ;; + go) + f=$(_lint_names 'file-length-limit' .golangci.yaml .golangci.yml .golangci.toml .golangci.json) \ + && printf 'revive file-length-limit in %s' "$f" ;; + typescript|javascript) + local configs=(eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml) + if f=$(_lint_names 'sonarjs/max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'sonarjs/max-lines in %s' "$f" + elif f=$(_lint_names '(^|[^-/a-zA-Z])max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'max-lines in %s' "$f" + fi ;; + python) + f=$(_lint_names '^[[:space:]]*max-module-lines[[:space:]]*=' pyproject.toml .pylintrc pylintrc setup.cfg tox.ini) \ + && printf 'pylint max-module-lines in %s' "$f" ;; esac + return 0 } lint_file_length_snippet() { @@ -418,8 +340,8 @@ measured() { } # ---------- file-length ---------- -# COUNT_AWK prints a file's line count under the style's comment syntax: -# skipc drops comment-only lines (block comments included), skipb blank lines. +# COUNT_AWK prints a file's code lines under the style's comment syntax: blank +# lines and comment-only lines, block comments included, are not counted. COUNT_AWK=' function comment_only(t, rest, i) { if (inblock) { @@ -440,45 +362,31 @@ function comment_only(t, rest, i) { } { t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) - if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } - c = comment_only(t) - if (c && skipc) next + if (t == "" || comment_only(t)) next n++ } END { print n + 0 } ' run_file_length() { # — every measured file of that language - local id="$1" cfg status max sc sb label src unit f n o - cfg=$(lint_file_length_config "$id") - IFS=$'\t' read -r status max sc sb label <<< "$cfg" - if [[ "$status" == "unreadable" ]]; then - # an unreadable row carries the rule and file in its second field - say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" - status=none + local id="$1" rule max src f n o + rule=$(lint_file_length_rule "$id") + if [[ -n "$rule" ]]; then + say "threshold: $id file length — held by $rule; the repository's linter reports long files, so this script measures none" + return 0 fi - if [[ "$status" == "covered" ]]; then - src="$label" - say "threshold: $id file length — covered by $label (limit $max)" + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else - sc=1; sb=1 - o=$(override file_max_lines) || exit 2 - if [[ -n "$o" ]]; then - max="$o"; src="$OVERRIDE_FILE" - say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" - else - max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" - fi + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi - unit="lines" - (( sc && sb )) && unit="code lines" - (( sc && ! sb )) && unit="lines without comments" - (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") - (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + n=$(awk -v style="$(lint_comment_style "$id")" "$COUNT_AWK" "./$f") + (( n > max )) && finding "$f" "$n code lines (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } diff --git a/python-linter-driven-development/scripts/ldd-lint_test.sh b/python-linter-driven-development/scripts/ldd-lint_test.sh index 814eea45..874b2d4a 100644 --- a/python-linter-driven-development/scripts/ldd-lint_test.sh +++ b/python-linter-driven-development/scripts/ldd-lint_test.sh @@ -106,19 +106,19 @@ fill_dir() { # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="python" FX_NATIVE="python" use_row() { + FX_LINTER_CONFIG='pyproject.toml' + FX_LINTER_COMMENT='#' FX_EXT='.py' FX_LINE_COMMENT='# note' FX_BLOCK_OPEN='' @@ -160,15 +160,12 @@ PY fx_test_file() { printf '%s/test_m%s.py' "$1" "$2"; } -fx_write_linter_config() { # - printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" +fx_write_linter_config() { + printf '[tool.pylint.format]\nmax-module-lines = 400\n' > "$REPO/pyproject.toml" } -# pylint counts every line; ruff has no file-length rule. -fx_write_linter_config_code() { return 1; } - -fx_write_linter_config_unreadable() { - printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" +fx_write_linter_config_other() { + printf '[tool.pylint.design]\nmax-args = 4\n' > "$REPO/pyproject.toml" } # ========================= cases ========================= @@ -215,31 +212,27 @@ run_lint file-length expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish -begin "a linter that skips comments and blank lines is counted its way" -if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish -else - write_src "src/a$FX_EXT" 90 20 20 - run_lint file-length - expect_exit 0 && expect_has "covered by" && expect_no_finding && ok - finish -fi +begin "a file-length rule in the linter config hands file length to the linter" +fx_write_linter_config +write_src "src/over$FX_EXT" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_has "held by " && expect_no_finding && ok +finish -begin "the repository's linter config holds the limit and is counted its way" -fx_write_linter_config 100 -write_src "src/a$FX_EXT" 90 20 +begin "a similar rule of another name leaves file length to the gap linter" +fx_write_linter_config_other +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) run_lint file-length -expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok finish -begin "a linter config whose limit cannot be read is named, and the next source applies" -if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish -else - write_src "src/over$FX_EXT" $((MAX_LINES + 1)) - run_lint file-length - expect_exit 0 && expect_has "its limit cannot be read here" \ - && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok - finish -fi +begin "a file-length rule that is commented out leaves file length to the gap linter" +fx_write_linter_config +sed "s|^|$FX_LINTER_COMMENT |" "$REPO/$FX_LINTER_CONFIG" > "$REPO/cfg.tmp" && mv "$REPO/cfg.tmp" "$REPO/$FX_LINTER_CONFIG" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish begin ".ldd-lint.yaml sets the limit when the linter holds none" printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" diff --git a/python-linter-driven-development/scripts/ldd-scope_test.sh b/python-linter-driven-development/scripts/ldd-scope_test.sh index 7f332f46..b0dfd39d 100755 --- a/python-linter-driven-development/scripts/ldd-scope_test.sh +++ b/python-linter-driven-development/scripts/ldd-scope_test.sh @@ -118,19 +118,19 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="python" FX_NATIVE="python" use_row() { + FX_LINTER_CONFIG='pyproject.toml' + FX_LINTER_COMMENT='#' FX_EXT='.py' FX_LINE_COMMENT='# note' FX_BLOCK_OPEN='' @@ -172,15 +172,12 @@ PY fx_test_file() { printf '%s/test_m%s.py' "$1" "$2"; } -fx_write_linter_config() { # - printf '[tool.pylint.format]\nmax-module-lines = %s\n' "$1" > "$REPO/pyproject.toml" +fx_write_linter_config() { + printf '[tool.pylint.format]\nmax-module-lines = 400\n' > "$REPO/pyproject.toml" } -# pylint counts every line; ruff has no file-length rule. -fx_write_linter_config_code() { return 1; } - -fx_write_linter_config_unreadable() { - printf '[tool.pylint.format]\nmax-module-lines = "${MODULE_LIMIT}"\n' > "$REPO/pyproject.toml" +fx_write_linter_config_other() { + printf '[tool.pylint.design]\nmax-args = 4\n' > "$REPO/pyproject.toml" } # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- diff --git a/python-linter-driven-development/skills/refactoring/reference.md b/python-linter-driven-development/skills/refactoring/reference.md index 925e3a99..d5f6e4be 100644 --- a/python-linter-driven-development/skills/refactoring/reference.md +++ b/python-linter-driven-development/skills/refactoring/reference.md @@ -70,7 +70,7 @@ decomposition): "Multi-rule procedures", below. ## File and package routing -**A module over the file-length limit** — pylint's `max-module-lines` where the repository sets it (ruff has no file-length rule), else 450 code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: +**A module over the file-length limit** — pylint's `max-module-lines` where the repository sets it (ruff has no file-length rule), else 450 code lines; `bash /../../scripts/ldd-lint.sh file-length --root .` lists them, or names the linter rule that holds the limit: | File pattern | Action | |---|---| diff --git a/ts-react-linter-driven-development/CHANGELOG.md b/ts-react-linter-driven-development/CHANGELOG.md index cad904eb..69e73fd0 100644 --- a/ts-react-linter-driven-development/CHANGELOG.md +++ b/ts-react-linter-driven-development/CHANGELOG.md @@ -9,20 +9,20 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versio - **A gap linter for the design triggers no linter holds.** `scripts/ldd-lint.sh` measures lines per source file (`file-length`) and source files per directory - (`package-size`). Where the repository's linter config holds the file-length - limit, the script uses that limit and counts lines the way that linter counts. - Otherwise a `.ldd-lint.yaml` at the repository root sets the limits, and - otherwise the plugin's defaults apply: 450 code lines per file, a warning at 8 - source files per directory and the limit at 13. The first line of the output - names the limit that applied; with the defaults it also prints the setting that - makes the repository's linter hold the limit. `--tsv` writes findings as rows. - The script never edits a file. + (`package-size`). Where the repository's linter config names a file-length rule, + the linter holds file length and the script measures none. Otherwise a + `.ldd-lint.yaml` at the repository root sets the limits, and otherwise the + plugin's defaults apply: 450 code lines per file, a warning at 8 source files per + directory and the limit at 13. The first line of the output names the source of + the limit; with the defaults it also prints a linter setting to propose to the + repository's owners. `--tsv` writes findings as rows. The script never edits a + file. ### Changed - **One set of thresholds.** The skills and routing tables name the same limits the - gap linter applies: the repository linter's limit where it sets one, else - 450 code lines per file and 8 and 13 files per directory. The package-size + gap linter applies: the repository linter's rule where it has one, else 450 + code lines per file and 8 and 13 files per directory. The package-size procedure runs the gap linter in place of a hand-written `find` command. ### Fixed diff --git a/ts-react-linter-driven-development/scripts/ldd-detect_test.sh b/ts-react-linter-driven-development/scripts/ldd-detect_test.sh index 28f00eca..0f748273 100644 --- a/ts-react-linter-driven-development/scripts/ldd-detect_test.sh +++ b/ts-react-linter-driven-development/scripts/ldd-detect_test.sh @@ -142,15 +142,13 @@ questions_in_rules() { # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="typescript" FX_NATIVE="typescript" @@ -158,6 +156,8 @@ FX_NATIVE="typescript" # names for the files it creates from it, and `fresh.ts*` is no file name. # Every file the row writes is a `.ts` file, so both globs select the same set. use_row() { + FX_LINTER_CONFIG='eslint.config.mjs' + FX_LINTER_COMMENT='//' FX_EXT='.ts' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -199,20 +199,15 @@ TS fx_test_file() { printf '%s/m%s.test.ts' "$1" "$2"; } -# ESLint's own max-lines counts every line unless told to skip comments. -fx_write_linter_config() { # - printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +fx_write_linter_config() { + printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: 400 }],\n },\n}]\n" > "$REPO/eslint.config.mjs" } -# sonarjs/max-lines counts only code lines. -fx_write_linter_config_code() { # - printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: %s }],\n 'max-lines-per-function': ['error', 50],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +# max-lines-per-function limits a function, not a file. +fx_write_linter_config_other() { + printf "export default [{\n rules: {\n 'max-lines-per-function': ['error', 50],\n 'sonarjs/max-lines-per-function': ['error', { maximum: 50 }],\n },\n}]\n" > "$REPO/eslint.config.mjs" } -# Every ESLint config form resolves: a rule named without a limit takes -# ESLint's documented default. -fx_write_linter_config_unreadable() { return 1; } - # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/ts-react-linter-driven-development/scripts/ldd-lint.sh b/ts-react-linter-driven-development/scripts/ldd-lint.sh index a54e968d..7eeec311 100755 --- a/ts-react-linter-driven-development/scripts/ldd-lint.sh +++ b/ts-react-linter-driven-development/scripts/ldd-lint.sh @@ -3,8 +3,8 @@ # # Measures the design triggers no linter holds, or that the repository's linter # does not hold yet: lines per source file and source files per directory. It -# never edits a file. Where the repository's linter holds a limit, that limit -# is the one applied, counted the way that linter counts. +# never edits a file. Where the repository's linter has a file-length rule, the +# linter owns file length and this script leaves it alone. # # Usage: bash scripts/ldd-lint.sh [options] # file-length source files over the file-length limit @@ -20,18 +20,17 @@ # know (TypeScript + React block below) # --test-re test-file pattern over relative paths # -# Thresholds, per language, first that applies — the "threshold:" line says -# which one a run used: -# 1. the repository's linter config holds the rule: its limit, counted the -# way that linter counts (file-length only; no linter counts files per -# directory); a config the table cannot read is named, then skipped -# 2. .ldd-lint.yaml at the repository root: file_max_lines, -# package_warn_files, package_max_files, each a positive integer -# 3. the plugin's defaults: 450 code lines per file (blank and -# comment-only lines not counted); a warning at 8 source files per -# directory and the limit at 13; for file-length, the line also +# Thresholds, per language — the "threshold:" line says which one a run used: +# file-length: when a linter config at the repository root names a +# file-length rule, the linter holds it: the line names the rule and the +# script measures nothing more. Else the limit is file_max_lines in +# .ldd-lint.yaml, else the plugin's default, 450 code lines (blank and +# comment-only lines are not counted); with the default, the line also # prints the linter setting that would hold the limit, as a proposal for # the repository's owners: the script and its caller never edit a config +# package-size: no linter counts files per directory. package_warn_files and +# package_max_files in .ldd-lint.yaml, else the plugin's defaults: a +# warning at 8 source files per directory and the limit at 13 # # What is measured: the language's source files, minus test files, generated # files and the directories the language block excludes. A directory count @@ -209,9 +208,9 @@ lang_is_test() { # ==================== linter table ==================== # What the gap linter needs to know per language id: how a comment is written, -# which source files a directory count leaves out, and where the repository's -# own linter sets a file-length limit. The same table is rendered into every -# plugin, so two plugins measure one file the same way. +# which source files a directory count leaves out, and whether the +# repository's own linter holds the file-length limit. The same table is +# rendered into every plugin, so two plugins measure one file the same way. # residue-exempt: the table names every language's linter and config file on # purpose, the same in every rendering. # @@ -219,11 +218,12 @@ lang_is_test() { # lint_package_skip_re ERE over a relative path: source files a # directory count leaves out (an index module, # a package marker, generated stubs); "" for none -# lint_file_length_config one line, from the repository root: -# covered TAB TAB TAB -# TAB -# unreadable TAB -# none +# lint_file_length_rule " in " when a linter config +# at the repository root names a file-length +# rule on a line that is not a comment, else +# nothing. A named rule belongs to the linter: +# its limit, its counting and whether it is on +# are the repository's decision. # lint_file_length_snippet the linter setting that would hold the # limit, or the .ldd-lint.yaml entry when no # linter rule is known: text to propose @@ -245,114 +245,36 @@ lint_package_skip_re() { return 0 } -# _lint_window — the key's line and the lines after it, up to -# the next rule entry or 15 lines, joined by spaces; one window per match. The -# patterns reach awk as strings, so they use bracket expressions, never -# backslash escapes, which awk -v would rewrite. -_lint_window() { - awk -v key="$1" -v q="'" ' - BEGIN { - entry = "^[[:space:]]*-[[:space:]]*[{]?[[:space:]]*name:" - qkey = "^[[:space:]]*[\"" q "][^\"" q "]+[\"" q "][[:space:]]*:" - } - function flush() { if (w != "") print w; w = ""; n = 0 } - { - if (n > 0 && ($0 ~ entry || $0 ~ qkey) && $0 !~ key) flush() - if ($0 ~ key) { flush(); n = 1; w = $0; next } - if (n > 0) { w = w " " $0; if (++n > 15) flush() } - } - END { flush() } - ' "$2" -} - -# _lint_number — the first match's digits -_lint_number() { - printf '%s\n' "$2" | awk -v re="$1" 'match($0, re) { v = substr($0, RSTART, RLENGTH); sub(/.*[^0-9]/, "", v); print v; exit }' -} - -# _lint_off — exit 0 when the rule in the window is switched off -_lint_off() { - printf '%s\n' "$1" | grep -qE "max-lines[\"']?[[:space:]]*:[[:space:]]*[[]?[[:space:]]*[\"']?(off|0)([^0-9]|$)" -} - -_lint_config_go() { - local f w max sc sb unread="" - for f in .golangci.yaml .golangci.yml .golangci.toml .golangci.json; do - [[ -f "$f" ]] || continue - grep -q 'file-length-limit' "$f" 2>/dev/null || continue - case "$f" in *.toml|*.json) unread="revive file-length-limit in $f"; continue ;; esac - while IFS= read -r w; do - [[ "$w" =~ disabled:[[:space:]]*true ]] && continue - max=$(_lint_number 'max:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*[[][[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number 'arguments:[[:space:]]*-[[:space:]]*[0-9]+' "$w") - if [[ -z "$max" ]]; then unread="revive file-length-limit in $f"; continue; fi - (( max > 0 )) || continue - sc=0; sb=0 - [[ "$w" =~ skip-comments:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skip-blank-lines:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\trevive file-length-limit in %s\n' "$max" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "name:[[:space:]]*[\"']?file-length-limit" "$f") - done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' -} - -_lint_config_eslint() { - local f w max sc sb - for f in eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts \ - .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml; do +# _lint_names ... — print the first file that matches the pattern +# on a line that is not a comment +_lint_names() { + local re="$1" f; shift + for f in "$@"; do [[ -f "$f" ]] || continue - grep -q 'max-lines' "$f" 2>/dev/null || continue - # sonarjs/max-lines counts lines of code: blank and comment-only lines are skipped. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number 'maximum[[:space:]]*:[[:space:]]*[0-9]+' "$w") - printf 'covered\t%s\t1\t1\tsonarjs/max-lines in %s\n' "${max:-1000}" "$f" - return 0 - done < <(_lint_window "sonarjs/max-lines[\"']?[[:space:]]*:" "$f") - # ESLint's own max-lines counts every line unless skipComments or skipBlankLines say otherwise. - while IFS= read -r w; do - w="${w%%]*}" - _lint_off "$w" && continue - max=$(_lint_number '[^a-zA-Z]max[[:space:]]*:[[:space:]]*[0-9]+' "$w") - [[ -z "$max" ]] && max=$(_lint_number "(error|warn|[12])[\"']?[[:space:]]*,[[:space:]]*[0-9]+" "$w") - sc=0; sb=0 - [[ "$w" =~ skipComments[[:space:]]*:[[:space:]]*true ]] && sc=1 - [[ "$w" =~ skipBlankLines[[:space:]]*:[[:space:]]*true ]] && sb=1 - printf 'covered\t%s\t%s\t%s\tmax-lines in %s\n' "${max:-300}" "$sc" "$sb" "$f" - return 0 - done < <(_lint_window "(^|[^/A-Za-z-])[\"']?max-lines[\"']?[[:space:]]*:" "$f") + grep -vE '^[[:space:]]*(#|//|;)' -- "$f" 2>/dev/null | grep -qE -- "$re" && { printf '%s' "$f"; return 0; } done - printf 'none\n' -} - -_lint_config_python() { - local f line max unread="" - for f in pyproject.toml .pylintrc pylintrc setup.cfg tox.ini; do - [[ -f "$f" ]] || continue - line=$(grep -E '^[[:space:]]*max-module-lines[[:space:]]*=' "$f" 2>/dev/null | head -1) - [[ -n "$line" ]] || continue - max=$(_lint_number "=[[:space:]]*[\"']?[0-9]+" "$line") - if [[ -z "$max" ]]; then unread="pylint max-module-lines in $f"; continue; fi - (( max > 0 )) || continue - # pylint counts every line of the module. - printf 'covered\t%s\t0\t0\tpylint max-module-lines in %s\n' "$max" "$f" - return 0 - done - [[ -n "$unread" ]] && { printf 'unreadable\t%s\n' "$unread"; return 0; } - printf 'none\n' + return 1 } -lint_file_length_config() { +lint_file_length_rule() { + local f case "$1" in - go) _lint_config_go ;; - typescript|javascript) _lint_config_eslint ;; - python) _lint_config_python ;; - *) printf 'none\n' ;; + go) + f=$(_lint_names 'file-length-limit' .golangci.yaml .golangci.yml .golangci.toml .golangci.json) \ + && printf 'revive file-length-limit in %s' "$f" ;; + typescript|javascript) + local configs=(eslint.config.js eslint.config.mjs eslint.config.cjs eslint.config.ts eslint.config.mts eslint.config.cts + .eslintrc .eslintrc.js .eslintrc.cjs .eslintrc.json .eslintrc.yml .eslintrc.yaml) + if f=$(_lint_names 'sonarjs/max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'sonarjs/max-lines in %s' "$f" + elif f=$(_lint_names '(^|[^-/a-zA-Z])max-lines([^-a-zA-Z]|$)' "${configs[@]}"); then + printf 'max-lines in %s' "$f" + fi ;; + python) + f=$(_lint_names '^[[:space:]]*max-module-lines[[:space:]]*=' pyproject.toml .pylintrc pylintrc setup.cfg tox.ini) \ + && printf 'pylint max-module-lines in %s' "$f" ;; esac + return 0 } lint_file_length_snippet() { @@ -428,8 +350,8 @@ measured() { } # ---------- file-length ---------- -# COUNT_AWK prints a file's line count under the style's comment syntax: -# skipc drops comment-only lines (block comments included), skipb blank lines. +# COUNT_AWK prints a file's code lines under the style's comment syntax: blank +# lines and comment-only lines, block comments included, are not counted. COUNT_AWK=' function comment_only(t, rest, i) { if (inblock) { @@ -450,45 +372,31 @@ function comment_only(t, rest, i) { } { t = $0; sub(/^[[:space:]]+/, "", t); sub(/[[:space:]]+$/, "", t) - if (t == "") { if (inblock) { if (!skipc) n++ } else if (!skipb) n++; next } - c = comment_only(t) - if (c && skipc) next + if (t == "" || comment_only(t)) next n++ } END { print n + 0 } ' run_file_length() { # — every measured file of that language - local id="$1" cfg status max sc sb label src unit f n o - cfg=$(lint_file_length_config "$id") - IFS=$'\t' read -r status max sc sb label <<< "$cfg" - if [[ "$status" == "unreadable" ]]; then - # an unreadable row carries the rule and file in its second field - say "threshold: $id file length — $max is set, but its limit cannot be read here; the next source applies" - status=none + local id="$1" rule max src f n o + rule=$(lint_file_length_rule "$id") + if [[ -n "$rule" ]]; then + say "threshold: $id file length — held by $rule; the repository's linter reports long files, so this script measures none" + return 0 fi - if [[ "$status" == "covered" ]]; then - src="$label" - say "threshold: $id file length — covered by $label (limit $max)" + o=$(override file_max_lines) || exit 2 + if [[ -n "$o" ]]; then + max="$o"; src="$OVERRIDE_FILE" + say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" else - sc=1; sb=1 - o=$(override file_max_lines) || exit 2 - if [[ -n "$o" ]]; then - max="$o"; src="$OVERRIDE_FILE" - say "threshold: $id file length — set by $OVERRIDE_FILE ($max code lines)" - else - max="$DEFAULT_FILE_MAX_LINES"; src="profile default" - say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" - fi + max="$DEFAULT_FILE_MAX_LINES"; src="profile default" + say "threshold: $id file length — profile default ($max code lines); the repository's linter does not hold it; a setting to propose, never to add on your own: $(lint_file_length_snippet "$id" "$max")" fi - unit="lines" - (( sc && sb )) && unit="code lines" - (( sc && ! sb )) && unit="lines without comments" - (( ! sc && sb )) && unit="non-blank lines" while IFS=$'\t' read -r _ f; do measured "$f" || continue - n=$(awk -v style="$(lint_comment_style "$id")" -v skipc="$sc" -v skipb="$sb" "$COUNT_AWK" "./$f") - (( n > max )) && finding "$f" "$n $unit (limit $max, $src)" + n=$(awk -v style="$(lint_comment_style "$id")" "$COUNT_AWK" "./$f") + (( n > max )) && finding "$f" "$n code lines (limit $max, $src)" done < <(awk -F'\t' -v id="$id" '$1 == id' "$SCOPE") } diff --git a/ts-react-linter-driven-development/scripts/ldd-lint_test.sh b/ts-react-linter-driven-development/scripts/ldd-lint_test.sh index 55b0182e..15ec9848 100644 --- a/ts-react-linter-driven-development/scripts/ldd-lint_test.sh +++ b/ts-react-linter-driven-development/scripts/ldd-lint_test.sh @@ -106,15 +106,13 @@ fill_dir() { # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="typescript" FX_NATIVE="typescript" @@ -122,6 +120,8 @@ FX_NATIVE="typescript" # names for the files it creates from it, and `fresh.ts*` is no file name. # Every file the row writes is a `.ts` file, so both globs select the same set. use_row() { + FX_LINTER_CONFIG='eslint.config.mjs' + FX_LINTER_COMMENT='//' FX_EXT='.ts' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -163,20 +163,15 @@ TS fx_test_file() { printf '%s/m%s.test.ts' "$1" "$2"; } -# ESLint's own max-lines counts every line unless told to skip comments. -fx_write_linter_config() { # - printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +fx_write_linter_config() { + printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: 400 }],\n },\n}]\n" > "$REPO/eslint.config.mjs" } -# sonarjs/max-lines counts only code lines. -fx_write_linter_config_code() { # - printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: %s }],\n 'max-lines-per-function': ['error', 50],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +# max-lines-per-function limits a function, not a file. +fx_write_linter_config_other() { + printf "export default [{\n rules: {\n 'max-lines-per-function': ['error', 50],\n 'sonarjs/max-lines-per-function': ['error', { maximum: 50 }],\n },\n}]\n" > "$REPO/eslint.config.mjs" } -# Every ESLint config form resolves: a rule named without a limit takes -# ESLint's documented default. -fx_write_linter_config_unreadable() { return 1; } - # ========================= cases ========================= run_row_cases() { # runs every case against the current row @@ -221,31 +216,27 @@ run_lint file-length expect_exit 0 && expect_has "profile default ($MAX_LINES code lines)" && expect_has "a setting to propose, never to add on your own: " && ok finish -begin "a linter that skips comments and blank lines is counted its way" -if ! fx_write_linter_config_code 100; then skip "no linter of this language counts only code lines"; finish -else - write_src "src/a$FX_EXT" 90 20 20 - run_lint file-length - expect_exit 0 && expect_has "covered by" && expect_no_finding && ok - finish -fi +begin "a file-length rule in the linter config hands file length to the linter" +fx_write_linter_config +write_src "src/over$FX_EXT" $((MAX_LINES + 50)) +run_lint file-length +expect_exit 0 && expect_has "held by " && expect_no_finding && ok +finish -begin "the repository's linter config holds the limit and is counted its way" -fx_write_linter_config 100 -write_src "src/a$FX_EXT" 90 20 +begin "a similar rule of another name leaves file length to the gap linter" +fx_write_linter_config_other +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) run_lint file-length -expect_exit 0 && expect_has "covered by" && expect_has "src/a$FX_EXT: 110 lines (limit 100, " && ok +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok finish -begin "a linter config whose limit cannot be read is named, and the next source applies" -if ! fx_write_linter_config_unreadable; then skip "every config form of this language is readable"; finish -else - write_src "src/over$FX_EXT" $((MAX_LINES + 1)) - run_lint file-length - expect_exit 0 && expect_has "its limit cannot be read here" \ - && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines (limit $MAX_LINES, profile default)" && ok - finish -fi +begin "a file-length rule that is commented out leaves file length to the gap linter" +fx_write_linter_config +sed "s|^|$FX_LINTER_COMMENT |" "$REPO/$FX_LINTER_CONFIG" > "$REPO/cfg.tmp" && mv "$REPO/cfg.tmp" "$REPO/$FX_LINTER_CONFIG" +write_src "src/over$FX_EXT" $((MAX_LINES + 1)) +run_lint file-length +expect_exit 0 && expect_not "held by " && expect_has "src/over$FX_EXT: $((MAX_LINES + 1)) code lines" && ok +finish begin ".ldd-lint.yaml sets the limit when the linter holds none" printf 'file_max_lines: 100 # this repository\n' > "$REPO/.ldd-lint.yaml" diff --git a/ts-react-linter-driven-development/scripts/ldd-scope_test.sh b/ts-react-linter-driven-development/scripts/ldd-scope_test.sh index 6a7e6bae..4a899e4b 100755 --- a/ts-react-linter-driven-development/scripts/ldd-scope_test.sh +++ b/ts-react-linter-driven-development/scripts/ldd-scope_test.sh @@ -118,15 +118,13 @@ mk_repo() { fx_write_marker "$REPO" fixture; fx_write_code; mk_docs; } # FX_BLOCK_CLOSE language has none, and the line that closes it # FX_SKIP_FILE a source file name a directory count leaves out # fx_test_file a test file's path in -# fx_write_linter_config the repository linter config that holds -# the file-length limit at , counting -# every line -# fx_write_linter_config_code the config of a linter that counts only -# code lines, holding the limit at ; -# exit 1 when the language has none -# fx_write_linter_config_unreadable a config that names the rule with a limit -# the linter table cannot read; exit 1 when -# every config form is readable +# FX_LINTER_CONFIG the path, under the repository root, of the file that +# fx_write_linter_config writes, and FX_LINTER_COMMENT the +# marker that turns one of its lines into a comment +# fx_write_linter_config the repository linter config that names +# the language's file-length rule +# fx_write_linter_config_other a linter config that names a rule of a +# similar name and no file-length rule FX_ROWS="typescript" FX_NATIVE="typescript" @@ -134,6 +132,8 @@ FX_NATIVE="typescript" # names for the files it creates from it, and `fresh.ts*` is no file name. # Every file the row writes is a `.ts` file, so both globs select the same set. use_row() { + FX_LINTER_CONFIG='eslint.config.mjs' + FX_LINTER_COMMENT='//' FX_EXT='.ts' FX_LINE_COMMENT='// note' FX_BLOCK_OPEN='/*' @@ -175,20 +175,15 @@ TS fx_test_file() { printf '%s/m%s.test.ts' "$1" "$2"; } -# ESLint's own max-lines counts every line unless told to skip comments. -fx_write_linter_config() { # - printf "export default [{\n rules: {\n 'max-lines': ['error', { max: %s }],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +fx_write_linter_config() { + printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: 400 }],\n },\n}]\n" > "$REPO/eslint.config.mjs" } -# sonarjs/max-lines counts only code lines. -fx_write_linter_config_code() { # - printf "export default [{\n rules: {\n 'sonarjs/max-lines': ['error', { maximum: %s }],\n 'max-lines-per-function': ['error', 50],\n },\n}]\n" "$1" > "$REPO/eslint.config.mjs" +# max-lines-per-function limits a function, not a file. +fx_write_linter_config_other() { + printf "export default [{\n rules: {\n 'max-lines-per-function': ['error', 50],\n 'sonarjs/max-lines-per-function': ['error', { maximum: 50 }],\n },\n}]\n" > "$REPO/eslint.config.mjs" } -# Every ESLint config form resolves: a rule named without a limit takes -# ESLint's documented default. -fx_write_linter_config_unreadable() { return 1; } - # ---------- the mixed repository: D, Python, Go and TypeScript in one tree ---------- # residue-exempt: the fixture is the same four-language tree for every plugin. # The same fixture for every plugin: a repository with setup.cfg and go.mod at diff --git a/ts-react-linter-driven-development/skills/refactoring/reference.md b/ts-react-linter-driven-development/skills/refactoring/reference.md index 447b39ff..266ce8a9 100644 --- a/ts-react-linter-driven-development/skills/refactoring/reference.md +++ b/ts-react-linter-driven-development/skills/refactoring/reference.md @@ -72,7 +72,7 @@ decomposition): "Multi-rule procedures", below. ## File and package routing -**`sonarjs/max-lines`, or a module over 450 code lines where the repository sets no limit** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them and names the limit it applied: +**`sonarjs/max-lines`, or a module over 450 code lines where the repository's config has no such rule** — `bash /../../scripts/ldd-lint.sh file-length --root .` lists them, or names the linter rule that holds the limit: | File pattern | Action | |---|---| From ec20a7eceac01cf63c545620c7394de6291ee8d7 Mon Sep 17 00:00:00 2001 From: Dan Mordechay Date: Sun, 18 Oct 2026 19:32:54 +0300 Subject: [PATCH 4/4] ldd-lint: strip comments with string literals kept, count from the git inventory Fix five review findings: - A rule name inside a block or trailing comment of a linter config no longer counts as a named rule. A new LINT_STRIP_AWK removes comments one character at a time and keeps string literals. The config check and the line count both use it. - A block comment that opens after code on a line now ends the code there. The lines inside the block are not counted. - --root, --files, --glob or --test-re as the last argument is a usage error (exit 2). Before, the loop never ended. - The directory count reads the repository inventory (git ls-files when the root is a repository). A git-ignored sibling no longer counts, and a --files run counts each directory it names in full. - *.stories.ts and the other story files are left out of the TypeScript directory count again. Each fix has a matrix case that fails on the previous script. Co-Authored-By: Claude Opus 5.5 --- core/includes/scripts/ldd-lint-langs.sh | 67 ++++++- core/scripts/ldd-lint.sh | 114 +++++------ core/scripts/ldd-lint_test.sh | 60 +++++- .../scripts/ldd-detect_test.sh | 10 +- .../scripts/ldd-lint.sh | 181 ++++++++++++------ .../scripts/ldd-lint_test.sh | 70 ++++++- .../scripts/ldd-scope_test.sh | 10 +- lang/generic/scripts/ldd-fixture.sh | 15 +- lang/go/scripts/ldd-fixture.sh | 10 +- lang/python/scripts/ldd-fixture.sh | 10 +- lang/ts-react/scripts/ldd-fixture.sh | 10 +- .../scripts/ldd-detect_test.sh | 15 +- linter-driven-development/scripts/ldd-lint.sh | 181 ++++++++++++------ .../scripts/ldd-lint_test.sh | 75 +++++++- .../scripts/ldd-scope_test.sh | 15 +- .../scripts/ldd-detect_test.sh | 10 +- .../scripts/ldd-lint.sh | 181 ++++++++++++------ .../scripts/ldd-lint_test.sh | 70 ++++++- .../scripts/ldd-scope_test.sh | 10 +- .../scripts/ldd-detect_test.sh | 10 +- .../scripts/ldd-lint.sh | 181 ++++++++++++------ .../scripts/ldd-lint_test.sh | 70 ++++++- .../scripts/ldd-scope_test.sh | 10 +- 23 files changed, 1039 insertions(+), 346 deletions(-) diff --git a/core/includes/scripts/ldd-lint-langs.sh b/core/includes/scripts/ldd-lint-langs.sh index 70a2d4fb..4a7509b8 100644 --- a/core/includes/scripts/ldd-lint-langs.sh +++ b/core/includes/scripts/ldd-lint-langs.sh @@ -7,12 +7,17 @@ # purpose, the same in every rendering. # # lint_comment_style c (// and /* */), d (also /+ +/) or hash (#) +# lint_strip_comments