From a85cd0a6f89ddf2756dba58457e8540b94860f45 Mon Sep 17 00:00:00 2001
From: Dylan Piercey <pierceydylan@gmail.com>
Date: Mon, 28 Sep 2026 10:05:06 -0700
Subject: [PATCH] docs(agent-feedback): file text body and non-null assertion
 gaps

File the parser gaps left after 5.18.0. Text-only bodies are lexed as
JavaScript, so prose and CSS bodies need a mode that recognizes only
placeholders and the closing tag (`<title>It's</title>` fails today), and
`<script>` bodies need regex literals lexed so `/["']/g` does not open a
string. A TypeScript `!` after whitespace or after `}` is still read as a
prefix operator.

Also update the escaped-quote item with the `<script>` regex cases it
covers (`/\"/g` runs to the end of the file, and `/^https?:\/\//` silently
comments out the rest of its line), and a fix direction that keeps
`\</style>` closing its tag.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---
 .../items/2026-07-30-escaped-quote-in-parsed-text.md |  4 ++--
 ...-a-text-body-mode-that-does-not-lex-javascript.md | 12 ++++++++++++
 ...6-treat-a-spaced-non-null-assertion-as-postfix.md | 12 ++++++++++++
 ...09-28-lex-regular-expressions-in-script-bodies.md | 12 ++++++++++++
 4 files changed, 38 insertions(+), 2 deletions(-)
 create mode 100644 agent-feedback/items/2026-09-25-add-a-text-body-mode-that-does-not-lex-javascript.md
 create mode 100644 agent-feedback/items/2026-09-26-treat-a-spaced-non-null-assertion-as-postfix.md
 create mode 100644 agent-feedback/items/2026-09-28-lex-regular-expressions-in-script-bodies.md

diff --git a/agent-feedback/items/2026-07-30-escaped-quote-in-parsed-text.md b/agent-feedback/items/2026-07-30-escaped-quote-in-parsed-text.md
index f9ed051..7b3a9e6 100644
--- a/agent-feedback/items/2026-07-30-escaped-quote-in-parsed-text.md
+++ b/agent-feedback/items/2026-07-30-escaped-quote-in-parsed-text.md
@@ -7,6 +7,6 @@ site: src/states/PARSED_TEXT_CONTENT.ts › PARSED_TEXT_CONTENT
 
 # Treat a backslash-escaped quote as text in parsed-text bodies and parsed strings
 
-Inside a `TagType.text` body the only backslash handling is `STATE.checkForPlaceholder`, which declines unless the run of backslashes leads to `${`, so a `\` falls through to the eager text run and the `"`/`'` after it still enters `STATE.PARSED_STRING`. `src/states/PARSED_STRING.ts` › `PARSED_STRING` has the mirror hole, where the quote of `\"` matches `str.quoteCharCode` and closes the string. An odd number of escaped quotes therefore runs to EOF and emits `INVALID_TEMPLATE_STRING` "EOF reached while parsing string expression"; marko maps `<style>`/`<script>` to `TagType.text`, so it surfaces as a code frame on the line after the tag. Escaped quotes are legal in CSS selectors and `content:` strings, so `<style>.a\" { color: red }</style>` and `content: 'it\'s'` are both unparseable today. The defect is quote-specific rather than backslash-general: `\2014` parses fine, and `src/__tests__/fixtures/parsed-text-style-tag` passes only because its escaped quotes come in pairs. Fix symmetrically in the two files: when `checkForPlaceholder` declines a `CODE.BACK_SLASH` and the next char is a quote (the active `quoteCharCode` in `PARSED_STRING`), consume both chars as text instead of letting the quote change state. Lock it in with a new `src/__tests__/fixtures/` dir plus `pnpm test:update`.
+A `TagType.text` body (`<style>`, `<script>`) is lexed as JavaScript strings, templates and comments, and a backslash counts only when `STATE.checkForPlaceholder` finds an escaped `${`. Otherwise it falls through as text and the character after it still switches state; `src/states/PARSED_STRING.ts` › `PARSED_STRING` has the mirror hole, where the quote of `\"` closes the string. So `<style>.a\" {}` and `content: 'it\'s'` run to the end of the file with "EOF reached while parsing string expression", as does a `<script>` regex with an escaped quote (`s.replace(/\"/g, "&quot;")`). A `<script>` regex with `\//` (`/^https?:\/\//`) opens a line comment instead, which silently drops every placeholder after it on that line. Fix both states: inside a parsed string a backslash escapes the next character; in the body it escapes a quote, backtick, slash or backslash, but not `<` or a newline, so `\</style>` still closes the tag and delimited blocks still end at the line. `src/scanner.c` › `scan_frame_content` in marko-js/tree-sitter mirrors both paths and needs the same change. The unescaped quote in `/["']/` is a separate gap: see "Lex regular expression literals in `<script>` bodies".
 
-Check: `node --input-type=module -e 'import{createParser,TagType}from"./src/index.ts";const p=createParser({onError:e=>console.log("ERR",e.message),onText:r=>console.log("text",JSON.stringify(p.read(r))),onOpenTagName:()=>TagType.text});p.parse("<style>.a\\\" { color: red }</style>")'` prints `ERR EOF reached while parsing string expression` today and should print a single `text` range.
+Check: `node --input-type=module -e 'import{createParser,TagType}from"./src/index.ts";const p=createParser({onError:e=>console.log("ERR",e.message),onPlaceholder:r=>console.log("placeholder",p.read(r.value)),onOpenTagName:()=>TagType.text});p.parse(process.argv[1])' '<script>if (/^https?:\/\//.test(u)) go(${x})</script>'` prints nothing (no `placeholder x`); with `'<style>.a\" {}</style>'` it prints `ERR EOF reached while parsing string expression`.
diff --git a/agent-feedback/items/2026-09-25-add-a-text-body-mode-that-does-not-lex-javascript.md b/agent-feedback/items/2026-09-25-add-a-text-body-mode-that-does-not-lex-javascript.md
new file mode 100644
index 0000000..b661e84
--- /dev/null
+++ b/agent-feedback/items/2026-09-25-add-a-text-body-mode-that-does-not-lex-javascript.md
@@ -0,0 +1,12 @@
+---
+type: bug
+impact: high
+effort: med
+site: src/states/PARSED_TEXT_CONTENT.ts › PARSED_TEXT_CONTENT
+---
+
+# Add a text body mode that does not lex JavaScript, for prose and CSS bodies
+
+`TagType.text` is the only text-only body mode, and `PARSED_TEXT_CONTENT.parse` lexes it as JavaScript: a quote enters `PARSED_STRING`, a backtick `TEMPLATE_STRING`, and `//` or `/*` enter `JS_COMMENT_LINE`/`JS_COMMENT_BLOCK`. That fits `<script>`, but the README's `onOpenTagName` example also returns `TagType.text` for `textarea` and `html-comment`, and marko does the same for `<title>`, `<textarea>` (compiler `marko-html.json` `parse-options.text`) and `<html-comment>`. So an apostrophe in prose (`<title>It's</title>`, `<textarea>Don't</textarea>`) fails with "EOF reached while parsing string expression", `<title>a /* b</title>` swallows the closing tag, and a `${x}` inside backticks stays literal text. `<style>` hits the `//` case: in `url(http://x) ${c}` the `//` opens a line comment, so the placeholder reaches the CSS as a literal `${c}`. Add an opt-in TagType where only placeholders and the matching closing tag are special (HTML's own RAWTEXT/RCDATA rule), keeping existing `TagType.text` events unchanged, and have marko's `packages/compiler/src/babel-plugin/parser.js › onOpenTagName` select it for every `parseOptions.text` tag except `<script>`/`<html-script>`. That mode also settles the `<style>` cases of "Treat a backslash-escaped quote as text in parsed-text bodies and parsed strings", whose `PARSED_STRING` fix `<script>` still needs.
+
+Check: `node --input-type=module -e 'import{createParser,TagType}from"./src/index.ts";const p=createParser({onError:e=>console.log("ERR",e.message),onText:r=>console.log("text",JSON.stringify(p.read(r))),onPlaceholder:r=>console.log("placeholder",JSON.stringify(p.read(r.value))),onOpenTagName:()=>TagType.text});p.parse(process.argv[1])' "<title>It's</title>"` prints `ERR EOF reached while parsing string expression`; the same command with `'<style>.a { background: url(http://x) ${c} }</style>'` prints a single `text` range containing `${c}` and no `placeholder`. In marko, `pnpm run compile -- -o html -d` on a template containing `<title>It's</title>` fails with the same error.
diff --git a/agent-feedback/items/2026-09-26-treat-a-spaced-non-null-assertion-as-postfix.md b/agent-feedback/items/2026-09-26-treat-a-spaced-non-null-assertion-as-postfix.md
new file mode 100644
index 0000000..0e89504
--- /dev/null
+++ b/agent-feedback/items/2026-09-26-treat-a-spaced-non-null-assertion-as-postfix.md
@@ -0,0 +1,12 @@
+---
+type: bug
+impact: low
+effort: low
+site: src/states/EXPRESSION.ts › lookBehindForOperator
+---
+
+# Treat a spaced or `}`-trailing TypeScript `!` as postfix when it ends an unenclosed value
+
+`lookBehindForOperator` reads a trailing `!` (or run of `!`) as postfix only when the character directly before it ends an operand: a word that is not a keyword operator, `)`, `]`, a closing quote or a backtick. A non-null assertion after whitespace (`x !`, which TypeScript accepts) or after an object literal (`{a:1}!`) is still read as a prefix operator waiting for an operand, so an unenclosed attribute, tag-variable or statement value ending in one swallows what follows (`<div a=x ! b/>` gives the one value `x ! b`). Both shapes are rare (prettier prints `x!`), so decide whether they are worth the extra look-behind; if so, walk back over whitespace before testing for an operand. A `}` needs care, since in a statement it can also close a block that a prefix `!` follows. Add a fixture per shape.
+
+Check: `node --input-type=module -e 'import{createParser}from"./src/index.ts";const p=createParser({onAttrValue:r=>console.log("value",JSON.stringify(p.read(r.value)))});p.parse("<div a=x ! b/>")'` prints `value "x ! b"`.
diff --git a/agent-feedback/items/2026-09-28-lex-regular-expressions-in-script-bodies.md b/agent-feedback/items/2026-09-28-lex-regular-expressions-in-script-bodies.md
new file mode 100644
index 0000000..0ffa4a3
--- /dev/null
+++ b/agent-feedback/items/2026-09-28-lex-regular-expressions-in-script-bodies.md
@@ -0,0 +1,12 @@
+---
+type: bug
+impact: med
+effort: med
+site: src/states/PARSED_TEXT_CONTENT.ts › PARSED_TEXT_CONTENT
+---
+
+# Lex regular expression literals in `<script>` bodies
+
+`PARSED_TEXT_CONTENT` lexes a `TagType.text` body as JavaScript strings, templates and comments, but checks a `/` only for `//` and `/*`: it never enters `STATE.REGULAR_EXPRESSION` the way `EXPRESSION` does. A quote inside a regex literal therefore opens a string, so `<script>s.replace(/["']/g, "")</script>` runs to the end of the file with "EOF reached while parsing string expression". (An escaped quote or slash has its own item, "Treat a backslash-escaped quote as text in parsed-text bodies and parsed strings".) Regex lexing cannot simply be added, because the same body mode serves `<style>`, where `url(/favicon.ico)` would read as an unterminated regex. Once `<style>` and the other prose bodies move to a mode that does not lex JavaScript (see "Add a text body mode that does not lex JavaScript"), enter `REGULAR_EXPRESSION` here when the previous significant character cannot end an operand, as `EXPRESSION` does with `canFollowDivision`, and add a fixture.
+
+Check: `node --input-type=module -e 'import{createParser,TagType}from"./src/index.ts";const p=createParser({onError:e=>console.log("ERR",e.message),onOpenTagName:()=>TagType.text});p.parse("<script>s.replace(/[\"\x27]/g, \"\")</script>")'` prints `ERR EOF reached while parsing string expression`.