feat(search): client side fts

[larbish]

Summary

Move search to client-side full-text search: a web worker owns a browser comark-content instance (sqlite-wasm FTS5) hydrated from per-commit snapshot artifacts, replacing the search-sections endpoint that shipped a flat section list on every page hydration.

Built on comarkdown/comark-content#109 (persisted snapshot & manifest artifacts): the webhook parses once per content push and update the cached artifactfs so consumers reuse the cached version.

How it works

Client.

useSearch pins hydration to the production commit via the new /api/content/head endpoint, then a worker fetches /api/content/blob/<sha>/manifest.json and /snapshot/content.json (SHA-pinned, immutable, isr: true cached at the CDN forever) and builds the FTS index through the plugin's cache-first path.

Queries run in the worker: BM25 ranking, heading-weighted scores, snippets.

Server.

The webhook's cache warm becomes warmSnapshot(): one full parse per push, which persists the snapshot artifact into the per-SHA Runtime Cache namespace. The first request on a new SHA is a single cache read, then ISR pins the URL; old SHA URLs simply become unreachable, so search needs no purges. Head-of-branch /blob/<sha> requests reuse the shared prod instance instead of minting a duplicate preview instance. The search-sections plugin, endpoint, route rules, and per-page payload are deleted.

Debug mode

On by default in dev; in production via ?debug=search. Both threads log onto one page-relative timeline — fixed-width lines showing stage durations, decoded/wire sizes, and result counts:

[search]    +412ms  manifest              14.2ms     1.3 KB  1.0 KB on the wire
[search]   +1284ms  index:built          118.4ms             net of the snapshot fetch above
[search]   +1290ms  index:rows                                87 results · sections in the FTS table

Next steps to discuss

• Extract the search stack into a plugin — useful for nuxt.com (migration on comark-content in progress).

• Version-aware search — the hydration URLs are already per-SHA (/api/content/blob/<sha>/…), so searching any historical version is just pointing the worker at that commit's artifacts, which the /blob/<sha> routes already serve and self-heal on first request. Would pair with the existing version-history UI; main open question is worker lifecycle when switching versions (one instance per SHA vs. reset + rehydrate).

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
comark-docs-layer	Ready	Preview	Sep 4, 2026 5:14pm UTC
comark-docs-playground	Ready	Preview	Sep 4, 2026 5:14pm UTC

[pkg-pr-new]

npm i https://pkg.pr.new/comarkdown/comark-docs@22

commit: 1b49f44

[coldtea-pr-lens]

◈ PR Lens

🟢 +7 new · 🟠 ~10 changed · 🔴 -0 removed · 3 flows · 36 files · commit 1b49f44

Architecture

17 components touched across 6 lanes.

Open full size

Inside the changed components — 3 views

Component view — Client-Side Search Engine

Browser-side full-text search running sqlite-wasm and FTS5 inside a Web Worker, hydrated from immutable CDN-cached snapshot artifacts.

Component view — Build-Time Snapshot & Cold-Start Seeding

Nuxt build module that verifies commit tree integrity, generates pre-parsed content snapshots, and stores them in Nitro server assets for instant cold starts.

Component view — Revalidation & Invalidation Pipeline

Webhook handler that diffs incoming commit changes against live manifests, warms new commit search artifacts, and purges affected ISR edge cache routes.

Data flow

Client search hydration and query · Build-time content snapshot and cold-start hydration · Push webhook revalidation and cache purging

Open full size

The other flows — 2 sequences

Build-time content snapshot and cold-start hydration

Push webhook revalidation and cache purging

Drill down

Client ＆ Browser — 8 components

🟡 CHANGED App Shell ＆ Layouts

Root application layout and shell, mounting the search palette and handling page navigation.

• app/app.vue
• app/components/layouts/LayoutsDocs.vue
• app/components/docs/DocsAsideMobileBar.vue
• app/components/layouts/LayoutsPage.vue
• app/app.config.ts

🟢 NEW Client Search Engine

Client-side full-text search engine (sqlite-wasm FTS5) running in a Web Worker and hydrated from snapshot artifacts.

• app/workers/search.ts:1-118
• app/workers/internal/search-logger.ts:1-69
• app/composables/useSearch.ts:1-61
• app/components/AppSearch.vue:1-67

🟢 NEW AppSearch Palette

Command palette modal rendering navigation sections and live search results from the worker.

• app/components/AppSearch.vue:1-67

🟢 NEW useSearch

Composable managing search state, fetching commit pin SHA from /api/content/head, and warming the worker.

• app/composables/useSearch.ts:1-61

🟢 NEW Search Web Worker

Web Worker managing the browser-standalone comark-content instance, artifact fetching, and SQLite FTS queries.

• app/workers/search.ts:1-118
• app/workers/internal/search-logger.ts:1-69

🟢 NEW SQLite WASM FTS5 Database

In-memory SQLite WASM database indexing document text for client-side full-text searches.

🟡 CHANGED useDocsContent

Client content composable providing reactive navigation and document fetching without server search plugins.

• app/composables/useDocsContent.ts
• app/composables/useNavigation.ts
• app/utils/navigation.ts

🟡 CHANGED Docs ＆ Landing Pages

Documentation, landing, error, and dynamic content page components.

• app/pages/[...slug].vue
• app/pages/index.vue
• app/pages/logos.vue
• app/error.vue
• app/components/landing/LandingHeroDemo.vue

Edge ＆ Routing — 2 components

🟡 CHANGED Edge ISR Cache

Vercel Edge ISR cache rules, caching immutable commit snapshot artifacts and invalidating page routes.

• modules/config/index.ts:159-178
• modules/config/utils.ts:1-66

🟡 CHANGED Markdown Content Negotiator

Rewrites inbound agent and raw requests to direct markdown endpoints.

• modules/markdown-rewrite/index.ts
• modules/markdown-rewrite/utils.ts

Server Routes ＆ APIs — 3 components

🟡 CHANGED Content API

Unified content endpoints dispatching document queries, tree navigation, commit pins, and snapshot artifacts.

• server/api/content/[...path].get.ts
• server/api/content/blob/[sha]/[...path].get.ts:20-35
• server/api/content/head.get.ts:1-10

🟡 CHANGED Revalidate Webhook

Webhook endpoint processing GitHub push events, computing content diffs, warming artifacts, and purging ISR.

• server/api/revalidate.post.ts:1-255
• server/utils/webhook.ts:1-117
• server/utils/timing.ts:1-29

🟡 CHANGED Raw Markdown Route

Serves unrendered raw markdown documents for AI agents and CLI consumers.

• server/routes/raw/[...slug].md.get.ts
• server/utils/markdown.ts

Content Engine — 2 components

🟡 CHANGED comark-content Engine

Instantiates content parsing runtime using withSnapshot to hydrate pre-parsed build snapshots on cold starts without GitHub roundtrips.

• server/utils/content.ts:1-135
• utils/content.ts:1-60

🟢 NEW Build-Time Snapshot Generator

Nuxt build module that verifies commit tree parity, generates parsed content snapshot seeds at build time, and embeds them into Nitro server assets.

• modules/snapshot/index.ts:1-73
• modules/snapshot/utils.ts:1-79

Cache ＆ Datastores — 2 components

🟢 NEW Build Content Seed Asset

Nitro server asset bundle (assets:comark-content) holding pre-rendered build snapshot and manifest files for instant cold-start hydration.

🟡 CHANGED Ref Pointer Cache

Unstorage cache mapping branch names and ref queries to resolved commit SHAs.

• server/utils/github.ts:79-110
• utils/github.ts:1-36
• utils/git.ts:1-104

View

• Architecture lens
• Data flow lens
• Expand every detail
• Show unchanged neighbours

Tip

Run PR Lens on your own machine: npx skills add coldteadotai/pr-lens installs the agent skill. Then tell your coding agent: "Diagram the change you just made with PR Lens and attach it to the pull request."

🪧 More tips

• Draw a diff before it is even a pull request: npx @coldtea/pr-lens-cli analyze --base origin/main reads the diff with your own model key, and npx @coldtea/pr-lens-cli render .pr-lens/graph.json draws the same lenses on your machine.
• The boxes under View are live. Tick Architecture lens or Data flow lens to choose which diagrams appear, or Expand every detail to open every drill-down at once. The comment redraws in place a few seconds later.
• Show unchanged neighbours lists the components this change did not touch alongside the ones it did, so the drill-down shows what the changed code sits next to.
• GitHub will not let you zoom an image in a comment. The link under each diagram opens it full size on a page of its own, where you can.
• The CLI's render picks up .github/pr-lens.yml automatically and applies your corrections (renames, exclusions, lane pins) at draw time.
• Would you rather run it from CI on a key of your own? Add .github/workflows/pr-lens.yml with coldteadotai/pr-lens/packages/action@v0 and a model key in your repository secrets, say GEMINI_API_KEY. The Action asks Gemini by default, or OpenAI and any endpoint speaking /chat/completions through its provider input.
• PR Lens is free for open source. A star on the repository is what keeps it going.
• Push a new commit and the whole comment re-renders for the new head. An older run never overwrites a newer one, so a slow render cannot put a stale diagram back.
• The diagrams follow your GitHub theme, so dark mode gets the dark render and light mode the light one, and the moving dots show this pull request's data in motion.

---

◈ Rendered by PR Lens · crafted with ❤️ by the Coldtea team · Come say hi on Discord