`. Prod (`--env prod`) serves both from ONE
+ port: `--frontend-port --backend-port
` with `REFLEX_API_URL=http://localhost:
`. The
+ first run does a bun install (1–2 min); poll the frontend URL until it returns 200 for up to ~6
+ minutes before concluding failure. A dev server's first page render can lag the 200: wait for
+ text the page must show before reading it.
+- Redis: start your own on the last backend port of YOUR range, which no app server then uses
+ (`redis-server --port --save '' --appendonly no &`), and point the app at it with
+ `REFLEX_REDIS_URL`. Prod with Redis forks several granian workers (capped at 8 by default;
+ `GRANIAN_WORKERS` overrides).
+- Chromium does not apply Playwright `extra_http_headers` to the websocket upgrade: inject headers
+ (`X-Forwarded-For`, cookies) through a small forwarding proxy, or a raw websocket client.
- `--loglevel debug` gives verbose server logs; redirect to a file and actually read it.
### What "testing" means here
@@ -91,6 +124,11 @@ and hunt for anomalies. You REPORT issues; you never fix framework code.
falling back (this environment blocks that mirror). Do report it if the fallback itself fails.
- Transient bun "incorrect peer dependency" warnings during a one-time upgrade migration, provided
the final lockfile is consistent.
+- A `/favicon.ico` 404 on apps without a favicon; AG Grid / AG Charts unlicensed-trial banners; the
+ uv `UV_NATIVE_TLS` deprecation warning; the "Compiling 100% N+1/N" counter overshoot; granian's
+ "more workers than CPU cores" warning.
+- Everything already filed or open in `prerelease-testing/REGISTRY.md` on `testing/prerelease` —
+ report those only if they CHANGED.
If you see something surprising that is not on this list, investigate it — several real findings
have surfaced first as an unexplained warning.
@@ -99,15 +137,36 @@ have surfaced first as an unexplained warning.
1. Copy reusable artifacts into the repo (plain `cp`/`rsync`, no git):
`DEST=//`
- - the app source dirs, EXCLUDING `.web/`, `node_modules/`, `.states/`, `assets/external/`,
- `*.db`, venvs
+ - the app source dirs, EXCLUDING `.web/`, `node_modules/`, `.states/`, `reflex.lock/`,
+ `assets/external/`, `*.db`, venvs, `__pycache__`
- your Playwright/repro scripts
- - `NOTES.md`: what you tested, how to rerun it (exact commands), what you observed, including
- benign quirks
-2. Return structured findings: every discrete check as a test entry (pass/fail/anomaly/skipped)
- with enough repro detail that another agent can reproduce a failure from `NOTES.md` and your
- scripts alone, without your conversation. An "anomaly" is anything surprising (console error,
- warning, traceback, visual glitch, perf cliff) even when functionality works.
+ - `NOTES.md`, written incrementally (the orchestrator commits DEST periodically, so a session
+ that dies mid-cluster still leaves a record): what you tested, how to rerun it (exact
+ commands), what you observed per check, including benign quirks and baseline results
+ - only the trimmed logs and screenshots that are evidence for a finding; keep DEST under ~2 MB.
+ Full logs stay under `$SB/apps//`.
+2. Your FINAL MESSAGE is this structured report (the orchestrator parses it). When a Workflow
+ gives you a result schema, return the schema's fields instead; this text form is for plain
+ agent calls:
+ ```
+ CLUSTER:
+ SUMMARY: <2-4 sentences>
+ ARTIFACTS:
+ TESTS:
+ - [pass|fail|anomaly|skipped] :
+ REVERIFIED:
+ - : fixed|still-broken|changed —
+ ISSUES:
+ - TITLE:
+ SEVERITY: critical|high|medium|low
+ REGRESSION: yes|no|unknown (yes = the previous release behaves correctly)
+ REPRO:
+ EVIDENCE:
+ NOT_COVERED:
+ ```
+ Every ISSUE must be reproducible by a stranger from `NOTES.md` and your scripts alone, without
+ your conversation. An "anomaly" is anything surprising (console error, warning, traceback,
+ visual glitch, perf cliff) even when functionality works.
### Timeboxing
diff --git a/.claude/skills/prerelease-test/references/clusters.md b/.claude/skills/prerelease-test/references/clusters.md
index 5c5d86aba5f..5739349a500 100644
--- a/.claude/skills/prerelease-test/references/clusters.md
+++ b/.claude/skills/prerelease-test/references/clusters.md
@@ -38,6 +38,10 @@ unassigned, and as a source of interaction ideas.
| Registration / multi-app | Two apps in one process; `AppHarness` twice | Sequential and simultaneous; check for registration leakage between them |
| Optional dependencies | Bare install, each extra, and the upgrade path | Import the optional package's feature without the extra and judge the error message quality |
| DevTools / perf | App with nested substates, custom components, client-only wrappers | Walk React fibers for `displayName`; record navigation main-thread cost; verify perf claims rather than trusting them |
+| Frontend state store | Every mutation kind (append/sort/reverse/delete key/type change/nested model) with the DOM compared to the backend state after each step | A third-party component that mutates the objects it is given (AG Grid cell edits) next to a *memoized* consumer of the same var (`rx.memo`, a chart, a second grid); a handler that stores the edited value as is vs one that normalizes it; recharts on the same data (it deep-freezes what it receives) |
+| Process model | Prod with Redis and several workers, dev with hot reload | Import-time side effects under a forking supervisor (threads, logging `QueueListener`, schedulers, DB engines, module-level random values); every start method; SIGTERM/SIGINT/kill -9 of one worker; a broken-then-fixed module in dev |
+| Forms | The same legacy forms (dict handlers, id-only, `id == name`, checkbox / switch / groups, sliders, selects) run on old and new | Diff exactly what the handler receives between versions — every changed key or value needs a changelog line; empty selections in every handler kind (dict, TypedDict, form-data object) |
+| Network / security headers | A probe app recording `router.session` and headers, plus a forwarding proxy and a raw websocket client | Spoofed / repeated / non-UTF-8 headers; trusted-proxy depth 0, 1, 2; every surface that reads the value — websocket, uploads, API routes, enterprise REST / MCP — must agree |
## Downstream and packaging clusters
@@ -49,4 +53,9 @@ Two clusters that are not feature-shaped but catch the highest-severity problems
names. A bare `AttributeError`/`ImportError` with no pointer to the replacement is a finding even
when the removal itself was intended.
- **Packaging.** `.claude/skills/prerelease-test/scripts/audit_pyi.py`, plus a check that each package's declared dependency pins
- match what the changelog says and that installing from the sdist works.
+ match what the changelog says and that installing from the sdist works — and that a wheel *built*
+ from each changed sdist keeps every stub (SKILL.md, Phase 5). Read the `reflex` wheel's sibling
+ floors: a stock `pip install -U reflex` that keeps old siblings does not deliver their fixes.
+- **Hosting CLI.** Never against the real Reflex Cloud: point the CLI at a local fake control
+ plane (derive the payload shapes from the CLI source and the PRs' tests) and record the requests
+ it receives. Compare deploy archives (file list, exclusions, compression) between versions.
diff --git a/.claude/skills/prerelease-test/references/orchestration.md b/.claude/skills/prerelease-test/references/orchestration.md
index da311d6812b..baa43e81416 100644
--- a/.claude/skills/prerelease-test/references/orchestration.md
+++ b/.claude/skills/prerelease-test/references/orchestration.md
@@ -128,9 +128,9 @@ produce confusing, unreproducible results.
Every cluster prompt should open with the same preamble, then the cluster-specific brief:
```
-FIRST read /AGENT_BRIEF.md and follow every rule in it (isolated PyPI-only installs — never
-install from the checkout; end-to-end browser testing; artifact + NOTES.md deliverables; kill your
-processes).
+FIRST read /CAMPAIGN_STATE.md, then /AGENT_BRIEF.md, and follow every rule in
+it (isolated PyPI-only installs — never install from the checkout; end-to-end browser testing;
+artifact + NOTES.md deliverables; kill your processes).
Your cluster key: "". Working dir: /apps//. Artifact destination: //.
Your RESERVED ports: frontend -, backend - — always pass them
explicitly, and never bind outside them.
@@ -146,9 +146,32 @@ verifier's range, which is the collision the map exists to prevent.
## Running it
- Two agents at a time is right for a 4-CPU box; the workflow's own concurrency cap handles the
- rest of the queue.
+ rest of the queue. A third agent fits when one of them runs no app server (a CLI or packaging
+ cluster, a verifier working through Python-level probes).
- Between phases, read the results before launching the next one.
- Schedule a check-in (`send_later`, ~45 min) during long runs so a wedged agent or a leaked dev
server gets noticed. Check `ps` for stray `reflex run` processes and commit finished clusters.
- When the workflow completes, its full structured result is in the task output file; parse it for
the report rather than re-reading transcripts.
+- Record each agent's report as it arrives (a short `reports/.md` in the run directory)
+ and commit it. `FINDINGS.md` is then assembled from files rather than from conversation context,
+ which may have been compacted by the time the last agent reports.
+
+## Surviving interruptions
+
+A full campaign runs for hours, and usage limits, spend limits and container restarts all happen
+mid-run. Plan for them:
+
+- **Commit continuously.** Agents cannot run git, so run a background loop that commits and pushes
+ the run directory every ~10 minutes, skipping any file over 2 MB (`git reset -q -- `).
+ Start it with `setsid nohup &` so it outlives the tool call, and stop it with a sentinel
+ file at the end. A restarted container loses the loop: start it again.
+- **Resume, don't respawn.** An agent stopped by a limit keeps its transcript. When the limit
+ resets, message it (`SendMessage` to its id) to continue, and tell it that every server, Redis,
+ proxy and browser it had running is gone after a restart. A fresh agent would redo the work
+ without its notes.
+- **Scratch survives, processes do not.** `$SB` (venvs, apps, logs) normally survives a worker
+ restart; check `ps` and the ports before resuming anything.
+- **When the configured model is unavailable** (a spend limit with no reset in sight), finish the
+ remaining verifications yourself rather than shipping unverified findings. Say so in
+ `FINDINGS.md` and to the user, since it departs from the independence the method relies on.
diff --git a/.claude/skills/prerelease-test/references/reporting.md b/.claude/skills/prerelease-test/references/reporting.md
index 242eae436f7..2c15894ba87 100644
--- a/.claude/skills/prerelease-test/references/reporting.md
+++ b/.claude/skills/prerelease-test/references/reporting.md
@@ -25,9 +25,16 @@ Index:
- FINDING-001: (SEVERITY)
- ...
+## Re-verification table
+| registry id | what | positive control (version, result) | new result | cluster |
+|---|---|---|---|---|
+<Every REGISTRY.md item re-run this pass: fixed / still broken / changed, with the control that
+proves the check can fail.>
+
## FINDING-00N: <title> (SEVERITY)
-- Cluster: `<cluster>` | Regression vs <prev stable>: yes/no | Verifier: CONFIRMED
+- Cluster: `<cluster>` | Regression vs <prev stable>: yes/no | Verifier: CONFIRMED / NARROWED /
+ RECLASSIFIED (say what changed: severity, scope, regression status)
- Repro: <exact commands/steps, self-contained>
- Evidence: <log excerpt, console error, screenshot path>
- Root cause (verifier analysis): <file:line and mechanism, when known>
diff --git a/.claude/skills/prerelease-test/scripts/bootstrap_envs.sh b/.claude/skills/prerelease-test/scripts/bootstrap_envs.sh
new file mode 100755
index 00000000000..97ba0d6ff81
--- /dev/null
+++ b/.claude/skills/prerelease-test/scripts/bootstrap_envs.sh
@@ -0,0 +1,97 @@
+#!/usr/bin/env bash
+# Build the isolated, PyPI-only virtualenvs a pre-release pass uses, as $SB/envs/<name>.
+#
+# usage: NEW_VERSION=0.10.1a1 PREV_VERSION=0.10.0 bootstrap_envs.sh <scratch-dir> [--enterprise]
+# then: export SB=<scratch-dir>
+#
+# Venvs (names the agent brief refers to):
+# new reflex[db]==$NEW_VERSION the version under test
+# prev reflex[db]==$PREV_VERSION the previous release (before/after baseline)
+# ctrl-X reflex[db]==X for each X in $CTRL_VERSIONS positive controls (versions a finding reproduced on)
+# driver playwright, httpx, websockets, pip browser drivers and HTTP probes
+# new-ent / prev-ent (with --enterprise) + reflex-enterprise and oidc-provider-mock
+#
+# Enterprise: set ENT_WHEEL to a user-supplied offline wheel (bypasses the login gate; never commit it), or
+# ENT_SPEC to a PyPI requirement such as 'reflex-enterprise[mcp]==0.9.8' (run enterprise apps with CI=true).
+#
+# Resolution: a version under test that is a pre-release resolves with --prerelease=allow. A final one (prev, a
+# final control) resolves without it, and with every reflex package it requires held to that reflex release's
+# PyPI upload time (--exclude-newer-package; reflex uploads last in its train). Otherwise the previous release's
+# `>=` sibling floors pick up the new train's alphas or finals, and the baseline silently runs code under test.
+# Everything else resolves as a user's install would today. PYDANTIC_PIN (default
+# 'pydantic<2.15') keeps the transitive graph user-like. Packages install from wheels only (ONLY_BINARY, default
+# ':all:'): uv reuses a cached sdist build for later installs of the same version, so one --no-binary experiment
+# would otherwise leak into every venv built after it. Do sdist experiments with their own UV_CACHE_DIR.
+#
+# Never run uv with a reflex checkout as the working directory: its pyproject's exclude-newer window hides fresh
+# packages, and the checkout would shadow the installed package. A venv whose install completed is reused; delete
+# it to rebuild. Index variables are cleared, so everything resolves from PyPI.
+set -euo pipefail
+SB="${1:?usage: NEW_VERSION=... PREV_VERSION=... bootstrap_envs.sh <scratch-dir> [--enterprise]}"; ENT="${2:-}"
+unset UV_INDEX UV_INDEX_URL UV_DEFAULT_INDEX UV_EXTRA_INDEX_URL UV_FIND_LINKS
+[ -n "${ENT_WHEEL:-}" ] && ENT_WHEEL="$(realpath -m "$ENT_WHEEL")"
+: "${NEW_VERSION:?set NEW_VERSION, e.g. 0.10.1a1}"
+: "${PREV_VERSION:?set PREV_VERSION, e.g. 0.10.0}"
+PY="${PYTHON_VERSION:-3.12}"
+PYDANTIC_PIN="${PYDANTIC_PIN:-pydantic<2.15}"
+ONLY_BINARY="${ONLY_BINARY-:all:}"
+mkdir -p "$SB/envs" "$SB/apps" "$SB/logs" "$SB/downloads"; SB="$(cd "$SB" && pwd -P)"; cd "$SB"
+
+is_pre() { [[ "$1" =~ (a|b|rc|\.dev)[0-9]+ ]]; }
+
+held_siblings() { # reflex version, python: an --exclude-newer-package flag per reflex package it requires
+ curl -fsS "https://pypi.org/pypi/reflex/$1/json" | "$2" -I -c 'import json, re, sys
+d = json.load(sys.stdin)
+cut = max(u["upload_time_iso_8601"] for u in d["urls"])
+reqs = [re.match(r"[A-Za-z0-9._-]+", r).group().lower() for r in d["info"]["requires_dist"] or []]
+print(" ".join(f"--exclude-newer-package={n}={cut}" for n in sorted({"reflex", *reqs}) if n.startswith("reflex")))'
+}
+
+mk() { # name, version-that-decides-resolution, pip args...
+ local name="$1" ver="$2" dir="$SB/envs/$1" held; shift 2
+ [ -f "$dir/.bootstrap-ok" ] && return 0
+ local opts=()
+ [ -n "$ONLY_BINARY" ] && opts+=(--only-binary "$ONLY_BINARY")
+ uv --no-config venv -q --clear --python "$PY" "$dir"
+ if is_pre "$ver"; then
+ opts+=(--prerelease=allow)
+ elif [ "$ver" != 0 ]; then
+ held="$(held_siblings "$ver" "$dir/bin/python")"
+ read -ra held_opts <<< "$held"
+ opts+=("${held_opts[@]}")
+ fi
+ uv --no-config pip install -q --python "$dir/bin/python" "${opts[@]}" "$@"
+ touch "$dir/.bootstrap-ok"
+}
+# greenlet must arrive through the db extra: never add it by hand to new/prev.
+# The published reflex wheel pins reflex-base to its own version, so reflex[db]==X brings the matching reflex-base.
+mk new "$NEW_VERSION" "reflex[db]==$NEW_VERSION" "$PYDANTIC_PIN"
+# PREV_EXTRAS adds packages to prev only (e.g. greenlet for a 0.9.x prev, whose db extra predates it).
+# shellcheck disable=SC2086
+mk prev "$PREV_VERSION" "reflex[db]==$PREV_VERSION" "$PYDANTIC_PIN" ${PREV_EXTRAS:-}
+for v in ${CTRL_VERSIONS:-}; do
+ # 0.9.x controls need greenlet by hand (SQLAlchemy 2.1 dropped it from its default install).
+ mk "ctrl-$v" "$v" "reflex[db]==$v" "$PYDANTIC_PIN" greenlet
+done
+mk driver 0 playwright httpx websockets pip
+[ -x /opt/pw-browsers/chromium ] || "$SB/envs/driver/bin/python" -m playwright install chromium >/dev/null 2>&1 \
+ || echo "playwright chromium install failed; set the Chromium path manually"
+
+if [ "$ENT" = "--enterprise" ]; then
+ ENT_REQ="${ENT_WHEEL:-${ENT_SPEC:-}}"
+ [ -n "$ENT_REQ" ] || { echo "--enterprise needs ENT_WHEEL=<offline wheel path> or ENT_SPEC=<requirement>"; exit 3; }
+ case "$ENT_REQ" in *.whl) [ -f "$ENT_REQ" ] || { echo "enterprise wheel missing: $ENT_REQ"; exit 3; }; ENT_REQ="$ENT_REQ[mcp]";; esac
+ mk new-ent "$NEW_VERSION" "reflex[db]==$NEW_VERSION" "$PYDANTIC_PIN" "$ENT_REQ" oidc-provider-mock
+ # shellcheck disable=SC2086
+ mk prev-ent "$PREV_VERSION" "reflex[db]==$PREV_VERSION" "$PYDANTIC_PIN" ${PREV_EXTRAS:-} "$ENT_REQ" oidc-provider-mock
+fi
+
+# Print every reflex package per venv: a baseline that resolved a sibling to the version under test shows here.
+for e in new prev $(for v in ${CTRL_VERSIONS:-}; do echo "ctrl-$v"; done) new-ent prev-ent; do
+ [ -x "$SB/envs/$e/bin/python" ] || continue
+ echo "== $e: $(cd "$SB" && "$SB/envs/$e/bin/python" -I -c 'import importlib.metadata as m
+names = sorted({d.metadata["Name"].lower() for d in m.distributions()})
+keep = [n for n in names if n.startswith("reflex")] + ["sqlalchemy", "greenlet", "pydantic"]
+print(" ".join(f"{n}={m.version(n)}" for n in keep if n in names))')"
+done
+printf 'export SB=%q\n' "$SB"
diff --git a/.claude/skills/prerelease-test/scripts/check_wheel_sources.py b/.claude/skills/prerelease-test/scripts/check_wheel_sources.py
new file mode 100644
index 00000000000..be20a5c82f6
--- /dev/null
+++ b/.claude/skills/prerelease-test/scripts/check_wheel_sources.py
@@ -0,0 +1,193 @@
+"""Check that the installed release packages carry exactly the source a release tag contains.
+
+Lists every Python package source file that changed between two git refs (``reflex/`` and
+``packages/<pkg>/src/<module>/``), maps each to its location in a venv's site-packages,
+and compares the bytes. A file that differs, or one deleted at the tag that is still
+installed, means PyPI serves something other than the tag: a mis-built or mis-tagged
+release. A changed file of a package that is not installed in the venv is reported
+separately — expected for unpublished workspace packages and for ones outside the
+install graph, a finding when the package should be there.
+
+Usage (PEP 723 script, no dependencies; run it from any directory):
+ uv run --script check_wheel_sources.py --repo <checkout> --base v0.10.0 --ref v0.10.1a1 \
+ --site-packages $SB/envs/new/lib/python3.12/site-packages
+"""
+
+# /// script
+# requires-python = ">=3.11"
+# ///
+
+from __future__ import annotations
+
+import argparse
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+PROJECT_NAME = re.compile(r"^name\s*=\s*[\"']([^\"']+)[\"']", re.MULTILINE)
+PACKAGE_SOURCE = re.compile(r"packages/([^/]+)/src/(.+)")
+
+
+def git(repo: str, *args: str) -> subprocess.CompletedProcess[bytes]:
+ """Run a git command in the checkout.
+
+ Args:
+ repo: Path to the reflex checkout.
+ *args: The git arguments.
+
+ Returns:
+ The completed process, with stdout and stderr as bytes.
+ """
+ return subprocess.run(["git", "-C", repo, *args], capture_output=True, check=False)
+
+
+def installed_dists(site: Path) -> set[str]:
+ """Collect the normalized distribution names installed in a site-packages directory.
+
+ Args:
+ site: The site-packages directory.
+
+ Returns:
+ Normalized names (lowercase, ``-``/``_``/``.`` folded to ``-``).
+ """
+ return {
+ re.sub(
+ r"[-_.]+", "-", d.name.removesuffix(".dist-info").rsplit("-", 1)[0]
+ ).lower()
+ for d in site.glob("*.dist-info")
+ }
+
+
+def dist_name(repo: str, ref: str, package_dir: str) -> str:
+ """Read the distribution name a workspace package declares.
+
+ Args:
+ repo: Path to the reflex checkout.
+ ref: The git ref to read ``pyproject.toml`` at.
+ package_dir: The directory name under ``packages/``.
+
+ Returns:
+ The normalized distribution name, falling back to the directory name.
+ """
+ shown = git(repo, "show", f"{ref}:packages/{package_dir}/pyproject.toml")
+ match = (
+ PROJECT_NAME.search(shown.stdout.decode()) if shown.returncode == 0 else None
+ )
+ return re.sub(r"[-_.]+", "-", match.group(1) if match else package_dir).lower()
+
+
+def classify(
+ repo: str, ref: str, path: str, site: Path, names: dict[str, str]
+) -> tuple[str, str] | None:
+ """Compare one changed file of the checkout with its installed copy.
+
+ Args:
+ repo: Path to the reflex checkout.
+ ref: The tag under test.
+ path: The changed file, relative to the checkout.
+ site: The site-packages directory.
+ names: Cache of distribution names by ``packages/`` directory, filled as needed.
+
+ Returns:
+ The distribution and one of ``same``, ``differs``, ``stale`` (deleted at the tag,
+ still installed) or ``deleted``; ``None`` for a file no distribution ships.
+ """
+ if not path.endswith((".py", ".pyi", ".js", ".jsx", ".json", ".css")):
+ return None
+ if path.startswith("reflex/"):
+ target, dist = site / path, "reflex"
+ elif match := PACKAGE_SOURCE.fullmatch(path):
+ package_dir, rel = match.groups()
+ if package_dir not in names:
+ names[package_dir] = dist_name(repo, ref, package_dir)
+ target, dist = site / rel, names[package_dir]
+ else:
+ return None
+ at_ref = git(repo, "show", f"{ref}:{path}")
+ if at_ref.returncode != 0:
+ return dist, "stale" if target.exists() else "deleted"
+ if not target.exists() or target.read_bytes() != at_ref.stdout:
+ return dist, "differs"
+ return dist, "same"
+
+
+def report(results: dict[str, list[str]], uninstalled: dict[str, list[str]]) -> int:
+ """Print the files that do not match the tag and the overall verdict.
+
+ Args:
+ results: Changed files of installed distributions, by status.
+ uninstalled: Changed files of distributions missing from the venv, by name.
+
+ Returns:
+ ``1`` when a file differs or a deleted file is still installed, otherwise ``0``.
+ """
+ for path in results["differs"]:
+ print(f" DIFFERS OR MISSING {path}")
+ for path in results["stale"]:
+ print(f" DELETED AT TAG, STILL INSTALLED {path}")
+ for dist, paths in sorted(uninstalled.items()):
+ print(f" not installed: {dist} ({len(paths)} changed files)")
+ if results["differs"] or results["stale"]:
+ print("FAIL: the installed packages do not match the tag")
+ return 1
+ print("PASS")
+ return 0
+
+
+def main() -> int:
+ """Compare the changed source files and print a summary.
+
+ Returns:
+ ``1`` when an installed file differs from the tag or a deleted file is still
+ installed, ``2`` when the refs or the site-packages directory cannot be read or
+ ``reflex`` is not installed there, otherwise ``0``.
+ """
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument("--repo", default=".", help="path to the reflex checkout")
+ parser.add_argument("--base", required=True, help="previous release tag")
+ parser.add_argument("--ref", required=True, help="tag under test")
+ parser.add_argument(
+ "--site-packages", required=True, type=Path, help="site-packages of the venv"
+ )
+ args = parser.parse_args()
+ site: Path = args.site_packages
+ dists = installed_dists(site) if site.is_dir() else set()
+ if "reflex" not in dists:
+ print(f"reflex is not installed in {site}", file=sys.stderr)
+ return 2
+ # Without renames, a moved file is a deletion plus an addition, so a stale copy at
+ # the old path is still caught.
+ diff = git(
+ args.repo,
+ "diff",
+ "--no-renames",
+ "--name-only",
+ f"{args.base}..{args.ref}",
+ "--",
+ )
+ if diff.returncode != 0:
+ print(diff.stderr.decode().strip(), file=sys.stderr)
+ return 2
+
+ names: dict[str, str] = {}
+ results: dict[str, list[str]] = {"same": [], "differs": [], "stale": []}
+ uninstalled: dict[str, list[str]] = {}
+ for path in diff.stdout.decode().splitlines():
+ if (found := classify(args.repo, args.ref, path, site, names)) is None:
+ continue
+ dist, status = found
+ if dist not in dists:
+ uninstalled.setdefault(dist, []).append(path)
+ elif status in results:
+ results[status].append(path)
+
+ print(
+ f"{args.base}..{args.ref}: {len(results['same'])} changed source files "
+ f"identical in {site}"
+ )
+ return report(results, uninstalled)
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/.claude/skills/prerelease-test/scripts/drive_app.py b/.claude/skills/prerelease-test/scripts/drive_app.py
index ce4a56963b3..5c0869c0620 100644
--- a/.claude/skills/prerelease-test/scripts/drive_app.py
+++ b/.claude/skills/prerelease-test/scripts/drive_app.py
@@ -208,7 +208,10 @@ def main() -> int:
read_error: str | None = None
with sync_playwright() as p:
- browser = p.chromium.launch(executable_path=CHROMIUM, headless=not args.headed)
+ browser = p.chromium.launch(
+ executable_path=CHROMIUM if Path(CHROMIUM).exists() else None,
+ headless=not args.headed,
+ )
page = browser.new_context().new_page()
page.on("console", lambda m: console.append({"type": m.type, "text": m.text}))
page.on("pageerror", lambda e: page_errors.append(str(e)))
@@ -278,8 +281,12 @@ def main() -> int:
else [m for m in console if not is_benign(m["text"])]
)
problems = [m for m in shown if m["type"] in ("error", "warning")]
+ # A page that rendered no text inspected nothing: a dev server's first load often
+ # outlasts --settle, and a crashed render leaves the body empty too. Neither is clean.
+ empty_page = not read_error and not load_error and not body.strip()
clean = not (
problems
+ or empty_page
or page_errors
or failed
or bad_status
@@ -298,6 +305,7 @@ def main() -> int:
"action_error": action_error,
"screenshot_error": screenshot_error,
"read_error": read_error,
+ "empty_page": empty_page,
"console": shown,
"page_errors": page_errors,
"failed_requests": failed,
@@ -325,6 +333,10 @@ def main() -> int:
print(f" ACTION FAILED: {action_error}")
if read_error:
print(f" PAGE UNREADABLE: {read_error}")
+ if empty_page:
+ print(
+ " EMPTY PAGE: no text rendered (slow first dev load? add an expect_text action)"
+ )
if screenshot_error:
print(f" SCREENSHOT FAILED: {screenshot_error}")
for m in shown: