Skip to content

chore(process): measure the process, then fix what the measurement found - #366

Merged
difficcd merged 1 commit into
mainfrom
process/cycle-1
Oct 14, 2026
Merged

difficcd merged 1 commit into
mainfrom
process/cycle-1

Conversation

@difficcd

Copy link
Copy Markdown
Owner

A first Measure → Analyze → Change cycle on the development process itself. It starts from a written review of how this project is built — incremental development run on XP practices, checked practice by practice, ending in a risk plan and five steps. The review ordered those steps with the measurement baseline first: it is the cheapest, and every other step is judged by it. So that is where this began, and the changes below were chosen from what the numbers showed, not from the list.

The full record is docs/process/2026-10-14.md.

Measure

scripts/process-metrics.mjs reads the DevOps measures straight out of the repository's own records — merged PRs, issues, Actions runs, git history. Nothing to install; rerunnable at the end of each cycle, with --json saved beside the record for diffing.

measure all history last 30 days
deployment frequency 32.8 / week 39.7 / week
automated test pass ratio 96.9% 99.7%
defect escape rate 43.2% 46.5%
fixes carrying a test 43.6% 60.4%

Three corrections were needed before the numbers could be trusted — 55 PR titles written as prose and counted as nothing, App.jsx/App.tsx split across two names, translation tables ranking as "hotspots" — and the escape rate moved from 48.5% to 43.2% because of them. The record lists each.

Analyze

  • CI says yes; defects ship anyway. 97–99.7% pass while nearly one fix lands per two changes.
  • They land in one file. App: 63 fix-touches. The next: 11.
  • Test-after, measured. 56% of all fixes landed without touching a test.
  • CI ran four of its ten gates. ci.yml claimed to run npm run check and instead listed four steps by hand. Every gate added locally since — helper index, unreachable, unused imports, stroke writes, native dialogs, i18n — bound only on a machine where someone ran it. CONTRIBUTING said "npm run check is what CI runs"; it was not.
  • The map was 29% missing. ARCHITECTURE.md named 108 of 152 files under src/; export/ had no section.

Change

change expected to move
C1 scripts/fix-needs-test.mjs — a branch with a fix: commit must change test/, or carry a No-Test: trailer with a reason fixes carrying a test: 60% → ~100%
C2 scripts/architecture-index.mjs — every src/ file must be named in the map; the 44 added stays 152/152
C3 CI runs npm run check as one step — one list cannot drift from itself pass ratio may dip at first; that is six more gates counting
C4 needs-verification label on #338 and #4, counted by the metrics read 0 because nothing recorded the wait; now 2, should reach 0

The expected directions are written into the record now, so the next measurement is checked against a prediction rather than fitted to it afterwards.

Verification

  • npm run check green locally: 1312 tests; both new gates run in it.
  • The fix-needs-test gate checked end to end on a throwaway branch, not only through its unit tests: a fix: with no test fails; a No-Test: reason passes; an empty No-Test: fails; a fix with its test in a separate commit on the same branch passes.
  • This PR is itself the first CI run of npm run check with full history, so the check job here is what proves C1 and C3 work in Actions.

Also corrected because they were the same drift: README's guard table with its last row stranded outside it, "six static guards" (there were seven; now nine), CONTRIBUTING's squash-merge where the repository rebase-merges, and "read it before editing App.jsx".

🤖 Generated with Claude Code

A first Measure -> Analyze -> Change cycle, started from a written review of how this project is
developed (incremental development on XP practices). The review's own order was baseline first,
since it is the cheapest step and the one everything after is judged by, so that is where this
began - and the changes come from what the numbers showed.

Measure. scripts/process-metrics.mjs reads the measures straight out of GitHub and git: no tool
to install, rerunnable each cycle. Baseline, all history / last 30 days:

    deployment frequency     32.8 / 39.7 merges a week
    CI pass ratio            96.9% / 99.7%
    defect escape rate       43.2% / 46.5%
    fixes carrying a test    43.6% / 60.4%

Analyze. CI says yes to nearly everything while almost one fix lands per two changes. Fixes
concentrate in App (63 fix-touches; the next file has 11). Over half of all fixes landed
without a test. And two things found on the way:

- CI ran four of the ten gates. ci.yml listed its steps by hand, and every gate added to
  `npm run check` since - helper index, unreachable, unused imports, stroke writes, native
  dialogs, i18n - bound only on a machine where someone ran it.
- ARCHITECTURE.md did not name 44 of the 152 files under src/ (29%); export/ had no section.

Change.
- CI runs `npm run check` as one step, so the two lists cannot drift apart again.
- scripts/fix-needs-test.mjs: a branch with a fix: commit must change something under test/,
  or say why in a No-Test: trailer. The targeted test-first the review asked for.
- scripts/architecture-index.mjs: every file under src/ must be named in ARCHITECTURE.md. The
  44 were added.
- A needs-verification label, so deferred device checks are counted (it read 0 while #338
  waited, because nothing recorded the wait).

README, README.ko and CONTRIBUTING corrected where they had drifted too: a guard table with its
last row stranded outside it, "six guards" when there were seven, squash-merge where the
repository rebase-merges, and "npm run check is what CI runs", which was not true until now.

The record, with what each change is expected to move at the next measurement, is in
docs/process/2026-10-14.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
easy-mv-maker Ready Ready Preview Oct 14, 2026 8:41am UTC

@difficcd
difficcd merged commit f66bdd0 into main Oct 14, 2026
6 checks passed
@difficcd
difficcd deleted the process/cycle-1 branch October 14, 2026 09:12

This branch was successfully deployed

1 active deployment
Preview — d7d37c0f Deployed Oct 14, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant