Demo stability (lockup + leak) and continuous Pages deployment (#20)

.github/workflows/pages.yml

@@ -0,0 +1,103 @@
+# Continuous deployment of the demo site to GitHub Pages.
+#
+# The site was previously BUILT ON A LAPTOP and committed: every rebuild
+# added an ~11 MB engine composite to git history, and "keep Pages
+# updated" was a manual step that could silently drift from the source.
+# Here the artifact is built from the source tree on every push instead,
+# so the deployed demo is always the committed spike.
+#
+# Build runs on PRs too (a broken demo build should fail the PR, not the
+# deploy); only main deploys.
+#
+# CUTOVER, in this order — the committed docs/spike-demo/ is still what
+# serves today, and is deliberately NOT deleted by the change that
+# introduced this workflow:
+#   1. land this workflow and confirm a green `build` job on main;
+#   2. switch Pages source from "branch: main /docs" to "GitHub Actions"
+#      (repo settings, or `gh api -X POST .../pages -f build_type=workflow`)
+#      and confirm the `deploy` job publishes the same site;
+#   3. THEN delete docs/spike-demo/ from git and gitignore it — the
+#      artifacts stop accruing in history only once nothing serves them.
+# Doing (3) first would take the demo offline for the length of the gap.
+
+name: Pages
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  # One deploy at a time; a superseded run is cancelled rather than
+  # racing another deployment of the same site.
+  group: pages-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    # The engine composite pulls keyhive + subduction + automerge from
+    # source; a cold cache is a long build.
+    timeout-minutes: 60
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v5
+
+      - name: Install Deno (the deltic host; stock, no flags)
+        uses: denoland/setup-deno@v2
+        with:
+          deno-version: "2.9.5"
+
+      - name: Set up siblings and tools
+        # Same script developers run: sibling ports pinned by commit, the
+        # toolchain from rust-toolchain.toml, wasm-tools/wac/just pinned.
+        run: ./scripts/setup.sh
+
+      - name: Cache cargo
+        uses: Swatinem/rust-cache@v2
+        with:
+          workspaces: |
+            spikes/tasks-engine
+            spikes/demo/guest-app
+            spikes/demo/guest-panel-s3
+            spikes/demo/guest-panel-dropbox
+
+      - name: Build the demo site
+        working-directory: spikes/demo
+        env:
+          IROH_CHECKOUT: ../../../polymorph-iroh
+        run: just pages
+
+      - name: Assemble the site
+        # docs/ is the site root: the index and the todomvc spike are
+        # committed builds; spike-demo is the one this workflow rebuilds.
+        run: |
+          test -f docs/spike-demo/engine.component.wasm
+          test -f docs/spike-demo/panel-dropbox.component.wasm
+          test -f docs/index.html
+          du -sh docs
+
+      - name: Upload Pages artifact
+        uses: actions/upload-pages-artifact@v4
+        with:
+          path: docs
+
+  deploy:
+    # Only the default branch deploys; PR builds stop at the artifact.
+    if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
+    needs: build
+    runs-on: ubuntu-latest
+    permissions:
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

docs/spike-demo/index.html

@@ -101,6 +101,11 @@ <h2>Alice — tablet <small>no connections, your bucket only</small></h2>
       <div id="tablet-app"></div>
     </div>
   </div>
-  <script type="module" src="./demo.js"></script>
+  <!-- Build stamp: the copy step rewrites 1a94ffd-1787054037 (see the justfile).
+       Unversioned assets meant returning visitors ran a cached bundle
+       against fresh artifacts — the miniature of NOTES §Release
+       integrity's rule: a tiny mutable root, everything else versioned. -->
+  <meta name="pm-build" content="1a94ffd-1787054037">
+  <script type="module" src="./demo.js?v=1a94ffd-1787054037"></script>
 </body>
 </html>

scripts/setup.sh

@@ -0,0 +1,84 @@
+#!/usr/bin/env bash
+# Sibling checkouts and pinned tools for building the demo site — the
+# single source of truth shared by local developers and CI (the same
+# shape as polymorph-iroh's scripts/setup.sh).
+#
+# The demo's deno.json resolves the deltic ports through SIBLING paths
+# (../../../polymorph-*), so the checkouts must sit next to this repo.
+# Idempotent: existing checkouts are fetched and pinned, never clobbered.
+#
+# Environment:
+#   SIBLINGS_DIR         where sibling repos live (default: the parent dir)
+#   WASM_TOOLS_VERSION   wasm-tools version (default below)
+#   WAC_VERSION          wac-cli version (default below)
+#   JUST_VERSION         just version (default below)
+#   SKIP_TOOLS=1         skip tool installation (they are already present)
+set -euo pipefail
+cd "$(dirname "$0")/.."
+REPO_ROOT="$(pwd)"
+
+SIBLINGS_DIR="${SIBLINGS_DIR:-$(cd .. && pwd)}"
+WASM_TOOLS_VERSION="${WASM_TOOLS_VERSION:-1.247.0}"
+WAC_VERSION="${WAC_VERSION:-0.10.1}"
+JUST_VERSION="${JUST_VERSION:-1.54.0}"
+
+# Pinned to the revisions the demo was last verified against. Bumping one
+# is a deliberate act: the deltic ports carry embedder conventions that
+# have already broken this demo once (see spikes/demo/README.md).
+IROH_REPO=https://github.com/polymorph-components/polymorph-iroh.git
+IROH_PIN=1808cccc437fd2eafe66003e3c0b00518fb94f78
+WEBCRYPTO_REPO=https://github.com/polymorph-components/polymorph-webcrypto.git
+WEBCRYPTO_PIN=b13d25230d34bbb65ba657be906fd59151a201f7
+WEBRTC_REPO=https://github.com/polymorph-components/polymorph-webrtc-datachannels.git
+WEBRTC_PIN=8a8347766df9035747fb87f85f13eee16c14c1f4
+
+log() { printf '\n==> %s\n' "$1"; }
+
+pin_repo() { # url pin dir
+    local url="$1" pin="$2" dir="$3"
+    if [ ! -d "$dir/.git" ]; then
+        log "Cloning $(basename "$dir")"
+        git clone --filter=blob:none "$url" "$dir"
+    fi
+    if ! git -C "$dir" cat-file -e "$pin^{commit}" 2>/dev/null; then
+        git -C "$dir" fetch --filter=blob:none origin
+    fi
+    log "Pinning $(basename "$dir") at ${pin:0:12}"
+    git -C "$dir" checkout --quiet --detach "$pin"
+}
+
+mkdir -p "$SIBLINGS_DIR"
+pin_repo "$IROH_REPO" "$IROH_PIN" "$SIBLINGS_DIR/polymorph-iroh"
+pin_repo "$WEBCRYPTO_REPO" "$WEBCRYPTO_PIN" "$SIBLINGS_DIR/polymorph-webcrypto"
+pin_repo "$WEBRTC_REPO" "$WEBRTC_PIN" "$SIBLINGS_DIR/polymorph-webrtc-datachannels"
+
+if [ "${SKIP_TOOLS:-0}" != "1" ]; then
+    log "Installing pinned Rust toolchain (rust-toolchain.toml) and wasm targets"
+    (cd "$REPO_ROOT/spikes/tasks-engine" && (rustup show active-toolchain >/dev/null 2>&1 || rustup toolchain install))
+    # The engine + fetcher are wasip2 (pinned by rust-toolchain.toml); the
+    # app and panel guests are plain wasm32-unknown-unknown and carry no
+    # toolchain file, so that target is added explicitly.
+    rustup target add wasm32-unknown-unknown
+
+    for tool in "wasm-tools@$WASM_TOOLS_VERSION" "wac-cli@$WAC_VERSION" "just@$JUST_VERSION"; do
+        name="${tool%@*}"
+        if command -v "${name/wac-cli/wac}" >/dev/null 2>&1; then
+            log "${name} already present"
+        else
+            log "Installing ${tool}"
+            cargo install --locked "${name}" --version "${tool#*@}"
+        fi
+    done
+fi
+
+# polymorph-iroh vendors its own dependencies (a TLS profile crate among
+# them) through its setup script, so a fresh clone cannot build the
+# endpoint until that has run. Defer to its contract rather than
+# reimplementing it here; it is idempotent.
+log "Running polymorph-iroh's own setup (its vendored deps)"
+(cd "$SIBLINGS_DIR/polymorph-iroh" && ./scripts/setup.sh)
+
+log "Building the iroh endpoint component (the demo composite plugs it)"
+(cd "$SIBLINGS_DIR/polymorph-iroh" && cargo build -p iroh-endpoint --target wasm32-wasip2 --release)
+
+log "Setup complete. Siblings in $SIBLINGS_DIR"

spikes/demo/README.md

@@ -92,6 +92,24 @@ per pane (×3, one browser page):
   `kh-knows-agent(doc)` → subscriptions → bucket grant/flush → tablet
   cold boot → apps mounted.
 
+## Deployment
+
+The hosted build is **continuously deployed**: `.github/workflows/pages.yml`
+runs `scripts/setup.sh` (sibling ports pinned by commit, toolchain from
+`rust-toolchain.toml`, `wasm-tools`/`wac`/`just` pinned), builds the site
+from source on every push, and deploys `docs/` to Pages from `main`. PRs
+build the site too but do not deploy — a broken demo fails the PR
+instead of the site.
+
+`docs/spike-demo/` is **still committed for now** — the cutover order is
+in the workflow header: prove the Actions build, switch the Pages source
+to GitHub Actions, and only then stop committing the artifacts (four
+rebuilds of an ~11 MB engine composite are already in history). Deleting
+them first would take the live demo offline for the length of the gap.
+`just pages` writes the same tree locally for preview. Bumping a sibling
+pin in `scripts/setup.sh` is deliberate: those ports carry embedder
+conventions that have broken this demo before.
+
 ## Run it
 
 ```
@@ -119,6 +137,19 @@ Headless bring-up phases (`just bringup solo|wire|bucket`) retire the
 platform layers one at a time under Deno; `wire soak` runs a 30 s
 post-revocation stress loop.
 
+Memory/backpressure probes (added while chasing a reported lockup):
+
+```sh
+deno run -A host/leak-probe.ts 90 pulls   # engines + live subscriptions, RSS
+deno run -A host/table-probe.ts           # 400 pulls, guest table sizes + RSS
+deno run -A host/cdp-heap.ts <url> 300    # real headless Chromium heap via CDP
+```
+
+`cdp-heap.ts` needs a Chromium binary (`CHROME=…`, or the Playwright
+cache default) and forces a GC at the end — the only reading that
+separates retention from uncollected garbage. In-page, `__demo.health()`
+reports background queue depth and per-timer skip counts.
+
 ## Findings
 
 - **deltic 0.1.0 renamed the embedder conventions** (`WitError` →
@@ -169,6 +200,76 @@ post-revocation stress loop.
   (stats stand down). Worth carrying into the framework's chrome: a
   status surface that mixes ambient telemetry with consequential
   one-shot messages needs priority, not last-writer-wins.
+- **A bare transport error is undiagnosable, and one of them killed the
+  whole setup.** A live run failed with
+  `fetch: send: ErrorCode::InternalError(Some("NetworkError…"))` — no
+  method, no host, no operation — after ~20 s of a single
+  "configuring storage…" line. Three fixes, all in this commit: every
+  provider request **names itself** in transport errors
+  (`PUT host/path: transport failed after 3 attempts: …`); transport
+  failures (never statuses — 429/5xx go to the caller untouched) **retry
+  up to 3×**, which is safe because every provider call here is
+  idempotent by construction; and setup **announces each step**
+  (`configuring storage: grant: bob (pickup link)…`), so a failure says
+  *which* of the ~20 sequential calls died and the remaining message is
+  actionable advice rather than "check endpoint + CORS".
+- **A duplicate "Save & connect" re-ran the entire setup**, re-minting
+  container links and republishing pickups under the first run. The
+  guard's placement is the subtle part: the background chain serializes
+  work, so a flag checked *inside* the job always finds the previous run
+  finished — it has to be claimed **synchronously at call time**.
+  (Verified by driving two calls in one tick; the second is refused.)
+- **Unversioned assets served returning visitors a stale bundle.** The
+  page loaded `demo.js` by bare name, so a rebuilt demo kept running the
+  cached script against fresh components — it cost an hour of chasing a
+  fix that was already deployed. The build now stamps a mutable root
+  (`<meta name="pm-build">` + `demo.js?v=…`) and artifacts inherit the
+  stamp: NOTES §Release integrity's bootloader shape in miniature, and
+  the thing that makes a Pages republish actually take effect.
+- **Console-generated Dropbox tokens expire in ~4 h**, and the failure
+  is now legible (`create_folder_v2 …: 401 expired_access_token`). The
+  OAuth path is the real fix: PKCE with `token_access_type=offline`
+  returns a refresh token, and the engine refreshes on 401 and retries
+  once. Paste-a-token remains the dev fallback with a stated cliff.
+- **Fixed-rate timers with no in-flight guard were the lockup.** Every
+  periodic driver — app `poll` (400 ms x 3 panes), reconciliation pulls
+  (2.5 s), auto bucket-sync (4 s), stats (4 s) — appended to an
+  unbounded promise chain unconditionally, while the work behind them
+  routinely outlives the period (consumer-API storage runs 1-3 s/op).
+  Fixed-rate scheduling + slower-than-period work diverges: the queue
+  *is* the leak, and user input ends up behind hundreds of pending jobs
+  (sluggish, then wedged, then dead). All periodic work now **skips a
+  tick whose predecessor is still running** — correct semantics anyway:
+  a reconciliation pull is a refresh, not a transaction. Measured with a
+  1.5 s/op delay proxy in front of MinIO: **180 ticks skipped in 3
+  minutes** (jobs the old code would have queued), background depth
+  bounded at 3-4, and a UI-path task add still completing in **3 ms**
+  while storage churns. `__demo.health()` exposes depth + per-timer skip
+  counts.
+- **The "leak" was the queue, plus a measurement artifact — chased to
+  ground.** After the backpressure fix, growth persisted in the paseo
+  webview (~1 MB/s, monotonic over 5 minutes), so it was bisected:
+  500 driver/tasks calls leak nothing; app polls at 400 ms x 3 for 75 s
+  are flat; **reconciliation pulls leak** (35 MB / 75 s). Two independent
+  checks then cleared the engine: `host/table-probe.ts` runs 400 pulls
+  headless and shows every guest table flat with **RSS plateauing at
+  ~300 MB**, and a real headless **Chromium via CDP** (`cdp-heap.ts`)
+  runs the identical page for 150 s — heap sawtooths normally and
+  **returns to 7.5 MB after a forced GC, net -1.5 MB**. So there is no
+  leak in the engine, in subduction, or in the deltic browser ports; the
+  unbounded growth was (a) the queue divergence above, which retains one
+  closure per queued job, and (b) the paseo webview's own instrumentation
+  retaining objects (and/or never idling long enough to GC). **Measure
+  memory in a real browser, not in the automation webview** — the
+  earlier version of this section blamed the port layer on the strength
+  of webview numbers, and was wrong.
+- **One real leak was found and fixed on the way**: the engine's `syncs`
+  table inserted a result per sync and never removed it, while
+  `sync-status` only read it — unbounded by construction at ~48 syncs a
+  minute. Statuses are one-shot by contract, so the entry is now removed
+  as it is read, and `stats()` publishes the guest's table sizes
+  (`tables syncs=… conns=… parts=…`) precisely because a growth bug in
+  them is invisible from outside the component.
 - **Panel teardown is a deltic open question** (same one #22 lists for
   app kill): switching provider tabs clears the region and drops the
   references, but there is no explicit instance-terminate API — the