fix(SatisfactionCapture): numbered list items must not parse as a 1/10 rating - #1670
Open
asdf8675309 wants to merge 2 commits into
Open
Conversation
…0 rating
parseExplicitRating() treats a prompt beginning with a digit as an explicit
satisfaction rating. Its reject guard only bailed when the character after
the digit matched [/.\dA-Za-z], so a ")" or "]" passed straight through and
an ordinary numbered list parsed as rating 1:
"1) install the CLI, 2) run the setup script"
-> { rating: 1, comment: ") install the CLI, 2) run the setup script" }
That is not cosmetic. A rating <= 3 triggers captureFailure(), which copies
the entire session transcript to disk as a permanent failure record, and
writes a 1/10 row to the satisfaction ledger that feeds goal tracking. A
numbered question list therefore manufactures a failure that never happened.
Extends the existing guard's character class with ")" and "]". "." was
already covered, so "1. first thing" was never affected. Also exports the
predicate so it is unit-testable in isolation; no signature change.
Verified against every rating form the function currently accepts: 34 inputs
run through the pre-change and post-change implementations, 6 changed, all
of them numbered-list shapes, 0 changes to any legitimate rating form.
Adds SatisfactionCapture.test.ts (18 cases). The repo has no test runner, so
it is self-contained: `bun run SatisfactionCapture.test.ts` prints PASS/FAIL
per case and exits non-zero on failure.
… digit
The ")"/"]" guard in the previous commit closes the list-marker shape, but not
the more common one. "-" is an accepted rating separator, so:
"2 - add the header, but also back the file up somewhere outside the install"
is shape-identical to "8 - nice" and still parses as rating 2. Sibling detection
does not help: two of the three observed cases have no second list marker.
Length separates them. Real rating comments are a few words ("nice", "solid
work"); these tails ran 138, 145 and 258 characters. Caps the comment at 80
chars for the BARE-DIGIT path only — "N/10" and word forms are unambiguous
rating syntax and stay uncapped, so "8/10 <long comment>" is unaffected.
Evidence from one real install: across 607 ledger rows, every row the explicit
path ever produced was a false positive — five for five, zero genuine ratings.
Three of the five are this shape; the other two were the ")" shape. The path is
supposed to be the high-signal one, which is why the failure went unnoticed.
Test file grows to 23 cases: the three long-tail shapes, plus two asserting the
unambiguous forms are NOT capped. Re-ran the 34-input pre/post comparison — 6
changed, all list shapes, 0 legitimate rating forms affected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
parseExplicitRating()inLifeOS/install/hooks/SatisfactionCapture.hook.tsreads a user prompt for an explicit 1–10 satisfaction rating. It reads ordinary prose that begins with a number as a rating. Three shapes, each hidden behind the previous one:1) what does this flag do?2 - config.md -- add the do-not-edit header, and also back up the file first1 thing that's worth noting…·3 action items to take from this…The first is a straightforward gap: the reject guard bailed only when the character after the digit matched
[/.\dA-Za-z], so)and]passed.SENTENCE_STARTERSis a word list (items?|things?|steps?|…) with no defense against a list delimiter.The other two cannot be fixed syntactically.
-is an accepted rating separator, so2 - <prose>is shape-identical to8 - nice. And1 thing that's worth noting…is just English. Only comment length separates them.This is not cosmetic. A rating < 5 writes a learning note; ≤ 3 also calls
captureFailure(), which copies the entire session transcript to disk as a permanent failure record; every rating writes a row to the satisfaction ledger that feeds goal tracking. One misparse contaminates three surfaces at once — and because the kebab-case bundle name is LLM-generated from the phantom rating, a plain question acquires a fabricated failure description that reads as authoritative to everything downstream.Change
One file, one concern, two commits.
/^[/.\dA-Za-z]/→/^[/.)\]\dA-Za-z]/.MAX_BARE_DIGIT_COMMENT = 80, capping the trailing comment on the bare-digit path only. TheN/10and word forms are unambiguous rating syntax and are deliberately left uncapped, so8/10 <long comment>still works.parseExplicitRatingso it is unit-testable in isolation. No signature change..was already in the class, so1. first thingwas never affected.Why it's safe
The failure mode is a false negative: an ambiguous input returns
nulland one rating goes unrecorded. That is strictly better than the alternative, where a false positive writes a permanent transcript copy, a learning note, and a phantom ledger row. The code comments state this so a future reader doesn't relax it back toward permissiveness.Existing installs cannot break on upgrade. The change only ever converts a rating into
null, and only for a digit followed by)/], or a bare digit trailed by more than 80 characters of prose. No currently-accepted rating form produces either shape.Verification
No test runner exists in this repo, so the case table is the proof — and
SatisfactionCapture.test.tsships alongside so it becomes runnable the momentbun testis wired up.I ran 34 inputs through the pre-change and post-change implementations side by side, extracting both versions of the function into isolated modules and diffing the outputs rather than reasoning about the regex. 6 changed, all numbered-list shapes. 0 legitimate rating forms changed.
1) install the CLI, 2) run the setup script{rating:1, comment:") install…"}null✅1) what does this flag do?{rating:1, comment:") what does…"}null✅2] bracket form{rating:2, comment:"] bracket form"}null✅1)·10){rating:1}·{rating:10}null✅2 - config.md -- add the do-not-edit header, and also back up the file first{rating:2, comment:"config.md --…"}null✅1 thing that's worth noting, the weekly quota reset Friday evening…{rating:1, comment:"thing that's…"}null✅1. first thing\n2. second thingnullnull(already correct)8/10 really solid work here, especially the part where you caught the edge case{rating:8, comment:"really solid…"}eight really solid work here, especially the part where you caught the edge case{rating:8, comment:"really solid…"}8·10·7·1·0·11ten·nine·seven out of tenEight great work·10/10, thank you·9 / 10·8 out of 10 nice8 - nice·8: nice·4 - could be better·9 solid work·5 meh2/10 items·2 items·3 files changed·8.nullnullbun run SatisfactionCapture.test.ts→ 23/23 passed, exit 0. Tested on macOS (Darwin 25.5.0, arm64), Bun 1.3.14.Field evidence. On one real install, across 607 ledger rows, the explicit-rating path produced 5 rows in its entire history and every one was a false positive — zero genuine explicit ratings, ever. Three shapes accounted for them. The reason this went unnoticed for months is that the path is supposed to be the high-signal one, so its output was never suspected; the phantoms simply made the feature look alive. Ten failure bundles and five learning notes were written from those five misparses.
Found but NOT changed
1 ) item— whitespace before the delimiter — still parses when the tail is under 80 characters. The guard inspects only the character immediately after the digit; skipping whitespace would also swallow8 . nice, a legitimate-ish rating shape. Left alone and documented in the code.N/10,N out of 10) andSENTENCE_STARTERSare untouched. Neither interacts with this bug, and both were correct for every case in the table.bun testcannot resolveyamlin a fresh checkout of this directory (SatisfactionCapture.hook.ts→lib/identity.ts→yaml), becausebun testuses stricter resolution thanbun run's auto-install. Pre-existing and unrelated to this change — reproducible with a bareimport { parse } from 'yaml'. I did not touchpackage.json; the test targetsbun runfor exactly this reason.Scope
Nothing outside
parseExplicitRatingis touched. No dependencies added, no config changed, no other hook affected. Full diff: +22/−2 in the hook plus one new 113-line test file (135 insertions, 2 deletions total).