Skip to content

Draft: add safe class-based syntax highlighting - #38

Draft
maphew wants to merge 12 commits into
mainfrom
feature/mdo-4pm-2-syntax-highlighting
Draft

Draft: add safe class-based syntax highlighting#38
maphew wants to merge 12 commits into
mainfrom
feature/mdo-4pm-2-syntax-highlighting

Conversation

@maphew

@maphew maphew commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Checkpoint summary

Add regression coverage that exercises SVG set and animate URL filtering and fails against the vulnerable dependency release.

Agent-Signature: codex-gpt-5.2-high on behalf of Matt Wilkie
Agent-Signature: codex-unknown-model- on behalf of Matt Wilkie
Checkpoint is blocked on syntect's transitive RUSTSEC-2025-0141 audit warning.

Agent-Signature: codex-unknown-model- on behalf of Matt Wilkie
@maphew

maphew commented Jul 22, 2026

Copy link
Copy Markdown
Owner Author

Blocker research update: syntect's parsing feature itself forces dump-create and dump-load, so feature selection cannot remove bincode while retaining the default syntax set. syntect 5.3.0 and current master still use bincode 1.3.3. Upstream PR trishume/syntect#694 replaces it with serde-wincode; that PR is open, non-draft, mergeable, and all five upstream checks are green, but it is not merged/released. The conservative path is to wait for an upstream release; using the unmerged fork/revision or an audit exemption requires explicit owner approval.

codex-gpt-5.2-high on behalf of Matt Wilkie

maphew and others added 2 commits July 23, 2026 23:09
Address PR #37 review findings:

- Strip YAML front matter before parsing instead of enabling
  ENABLE_YAML_STYLE_METADATA_BLOCKS: pulldown-cmark 0.13 recognizes
  metadata blocks anywhere at indent 0, silently swallowing
  mid-document text framed by `---` lines. The hand-rolled
  yaml_front_matter splitter is now the single source of truth for
  both hiding the block and title derivation.
- Make the splitter conservative and GitHub-like: front matter only at
  the very start of the document, closing `---`/`...` on its own line
  (trailing spaces ok, tabs not), block non-empty with a non-blank
  first line, so `---\n\ntitle: fake\n---` can no longer spoof the
  page title.
- Keep GFM alerts styled with JavaScript disabled: add plain :root
  light-palette defaults and a prefers-color-scheme dark media query
  alongside the existing data-theme theme-toggle overrides.
- Add regression tests for mid-document `---` blocks and
  blank-first-line pseudo front matter; extend the alert CSS test.
- cfg-gate unix-only Duration/Instant test imports so
  `cargo clippy --all-targets -- -D warnings` also passes on Windows.
- Record GFM alerts and YAML front matter in the changelog.

Agent-Signature: claude-code-fable-5 on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GvsZX2KKgN6BrCCrNrJ5BW
Address review findings on the class-based syntax highlighting draft:

- Infinite-loop DoS: distinguish destroyed placeholders (count 0, e.g.
  swallowed as rawtext by a user-authored unclosed <style>/<script>)
  from collisions (count >1). Destroyed fences fall back to plain code
  blocks instead of retrying forever, and collision retries are capped
  at 16 before highlighting is disabled entirely. Both render paths now
  share one spliced_markdown_body loop with a termination guarantee.
- Theme CSS scoping: scope every comma-separated selector in generated
  theme rules, not just the first, so dark palette colors can no longer
  leak into light mode; test now asserts every selector is scoped.
- Size cap: fenced blocks over 200 KB skip syntect (fancy-regex worst
  cases are superlinear) and render as plain escaped code.
- Language aliases: map common fence tokens (sh/shell/zsh, yml,
  csharp/cs, c++, jsonc, ts/tsx/typescript) to bundled syntaxes; the
  bundled set has no TypeScript grammar so those map to JavaScript.
- cargo-audit: ignore RUSTSEC-2025-0141 (bincode 1.x unmaintained,
  transitive via syntect dump-load; no feature-flag remedy) via
  .cargo/audit.toml, verified locally with cargo audit --deny warnings.
- Document why the global class/id attribute allowance is safe.
- Gate Duration/Instant imports in tests/cli.rs behind cfg(unix) so
  clippy -D warnings also passes on Windows hosts.

Agent-Signature: claude-code-fable-5 on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GvsZX2KKgN6BrCCrNrJ5BW
@maphew

maphew commented Jul 24, 2026

Copy link
Copy Markdown
Owner Author

Pushed baf5d12 addressing the review punch list:

1. Infinite-loop DoS (high). The render/sanitize retry loops in safe_markdown_body and convert_with_diagnostics treated every placeholder mismatch as a collision and retried forever. RenderedMarkdown::finish now distinguishes destroyed placeholders (count 0 — e.g. swallowed as rawtext when a user-authored unclosed <style>/<script> precedes a highlighted fence; no discriminator can ever survive that) from collisions (count >1). Both call sites now share a single spliced_markdown_body loop: destroyed fences are re-rendered as plain escaped code blocks via a strictly growing plain_fences set, and collision retries are capped at MAX_PLACEHOLDER_RETRIES = 16 before highlighting is disabled entirely — so termination is guaranteed. Regression tests cover unclosed <style> and <script> before a fence, plus a mixed case asserting only the affected fence is downgraded.

2. Theme CSS scoping (medium). scope_theme_css only prefixed the first selector of syntect's comma-joined selector lists, leaving every subsequent selector unscoped (dark colors leaking into light mode). Each comma-separated selector is now individually prefixed with the theme scope, and generated_theme_css_is_scoped_to_both_manual_palettes now asserts every .mdo-syntect- selector in every rule carries a scope.

3. Size cap (medium). Fenced blocks larger than MAX_HIGHLIGHT_BYTES (200 KB) skip syntect — whose fancy-regex grammars have superlinear worst cases — and render as plain escaped code. Covered by a new unit test.

4. Audit exemption. CI fails on RUSTSEC-2025-0141: bincode 1.3.3 is unmaintained and pulled in transitively via syntect's dump-load feature; syntect offers no feature-flag remedy today. Added .cargo/audit.toml ignoring exactly that advisory with a comment noting it should be removed once syntect drops bincode. Note: cargo-audit reads its config from .cargo/audit.toml, not a repo-root audit.toml, hence the location. Verified locally with cargo-audit 0.22.2: cargo audit --deny warnings fails without the config and passes with it.

5. Language aliases (low). Added an alias map consulted before find_syntax_by_token: sh/shell/zsh→bash, yml→yaml, csharp/cs→C#, c++→cpp, jsonc→json. The bundled syntax set contains no TypeScript grammar, so ts/tsx/typescript map to JavaScript instead (per-alias resolution is asserted by a test; unknown languages still fall back to plain).

6. Documented at the add_generic_attributes(["class", "id"]) site that the global class allowance is intentional and that highlight safety comes from post-sanitize splicing, not the allowlist.

Also gated the Duration/Instant imports in tests/cli.rs behind cfg(unix) — they are only used by unix-gated helpers, so cargo clippy --all-targets -- -D warnings previously failed on Windows hosts.

Quality gates: cargo fmt, cargo clippy --all-targets -- -D warnings, and cargo test (76 tests) all green locally; cargo audit --deny warnings passes with the exemption. PR left in draft.

claude-code-fable-5 on behalf of maphew

@maphew
maphew changed the base branch from feature/mdo-4pm-1-markdown-upgrade to main August 23, 2026 18:11
maphew added 7 commits August 23, 2026 11:11
Preserve conservative front-matter stripping and no-JS alert palettes while integrating the stacked base branch.

Agent-Signature: codex-gpt-5.6-sol-high on behalf of matt wilkie
Resolve the changelog release-boundary conflict by keeping the new alert and front-matter entries under Unreleased while preserving the 0.6.1 notes from main.

Agent-Signature: codex-gpt-5.6-sol-medium on behalf of maphew
Integrate the current #37 head, including its merge from main, so this branch is compatible with the retargeted stack.

Agent-Signature: codex-gpt-5.6-sol-high on behalf of matt wilkie
Splice generated highlighting into raw renderer output before sanitization so normalized placeholder spoofs cannot redirect post-sanitize markup. Add no-JS light and OS-dark palettes with explicit manual theme overrides.

Agent-Signature: codex-gpt-5.6-sol-high on behalf of matt wilkie
Resolve the squash-merge overlap by retaining the already-integrated front-matter and alert fixes alongside the syntax-highlighting implementation.

Agent-Signature: codex-gpt-5.6-sol-medium on behalf of maphew
Integrate the merged orb-sizing PR before publishing the syntax-highlighting draft.

Agent-Signature: codex-gpt-5.6-sol-medium on behalf of maphew
Do not claim a Beads tracking issue exists before the maintenance-risk exemption has been accepted and tracked.

Agent-Signature: codex-gpt-5.6-sol-medium on behalf of maphew
@maphew

maphew commented Aug 23, 2026

Copy link
Copy Markdown
Owner Author

PR sweep update: pushed repaired, current-main candidate through ab1b5c1 and kept the PR in draft.

Addressed code blockers:

  1. Integrated Add GFM alerts and front-matter support #37 and current main, preserving conservative front-matter stripping, visible mid-document --- content, and no-JavaScript alert palettes.
  2. Moved generated highlight splicing before the single ammonia sanitizer pass. This closes the combined entity-normalized spoof plus destroyed-real-placeholder case because no attacker-positioned string insertion occurs after sanitization. Added the combined regression and retained bounded collision/plain-fence termination behavior.
  3. Added no-JavaScript syntax-color defaults: light base palette, OS-dark media palette, then higher-specificity manual theme overrides. Added selector-scoping and palette tests.
  4. Removed the unverified claim that a Beads follow-up already exists.

Independent review found no remaining functional or security blocker. Local validation passed: formatting, clippy with warnings denied, 90 Rust tests, cargo audit --deny warnings, git diff --check, plus the newly merged orb-skill tests and setup shell syntax.

Human decision required before ready/merge: accept or reject the narrow RUSTSEC-2025-0141 audit exemption for bincode 1.x. RustSec classifies it as unmaintained rather than a known vulnerability; it is transitive through syntect's trusted bundled dumps, no feature-flag remedy exists, and syntect's maintained replacement remains unreleased. If accepted, create/link durable follow-up work to remove the exemption when syntect drops bincode. If not accepted, this PR should wait for upstream or use an explicitly approved maintained fork.

codex-gpt-5.6-sol-medium on behalf of maphew

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants