Repository navigation
Conversation
Point the skill at the long-lived testing/prerelease branch (registry, fixtures, runs/, close-out) and keep run bulk out of git. New preflight helpers: - scripts/bootstrap_envs.sh builds the PyPI-only venvs. A final baseline now resolves without pre-releases, so prev no longer picks up the alphas under test. Packages install from wheels only, so a cached sdist build cannot taint later venvs. - scripts/check_wheel_sources.py compares every changed source file between two tags with the installed copy. - drive_app.py reports a page that rendered no text as an anomaly instead of clean. Document the checks that found real issues this pass: - sibling floors of the reflex wheel versus a stock pip install; - wheels built from sdists losing stubs (with a throwaway uv cache); - parity of security changes across enterprise surfaces (REST vs websocket); - structural sharing with components that mutate state objects; - import-time side effects under forked workers; - legacy form payload diffs; - a fake control plane for the hosting CLI. The agent brief gains: - process-group server handling (never pkill -f) and a port-free check; - --no-config / --only-binary installs and prod single-port and Redis setup; - more known-benign noise; - the structured final report. orchestration.md gains a section on surviving usage limits and restarts: periodic commits, resuming agents, and verifying when the model is unavailable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SjCrbdguMkfe4bhbaREiwo
Contributor
 [Low impact] The PR changes the workflow used by operators and agents running prerelease campaigns. Fix the bootstrap example before merging; the documented setup currently stops before creating environments. Findings
SummaryUpdates the prerelease-testing skill with lessons from the 0.10.1 campaign.
The setup example needs fixing before use. Retry handling, Redis port selection, and renamed-file checks also need attention. Reviews (1) · Last reviewed commit: "Fold 0.10.1 pre-release campaign lessons..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+97
to
+99
| NEW_VERSION=<new> PREV_VERSION=<prev> CTRL_VERSIONS="<positive-control versions>" \ | ||
| ENT_SPEC='reflex-enterprise[mcp]==<ent>' # or ENT_WHEEL=<user-supplied offline wheel> | ||
| .claude/skills/prerelease-test/scripts/bootstrap_envs.sh "$SB" --enterprise |
Contributor
There was a problem hiding this comment.
The comment after ENT_SPEC ends the assignment command. The next line runs bootstrap_envs.sh without those variables exported. In a fresh shell, the script exits at its required-variable checks before creating any environments. Move the comment above the command and continue the assignments through the script invocation.
|
|
||
| mk() { # name, version-that-decides-resolution, pip args... | ||
| local name="$1" ver="$2"; shift 2 | ||
| [ -x "$SB/envs/$name/bin/python" ] && return 0 |
Contributor
There was a problem hiding this comment.
mk treats an executable Python as proof that an environment is ready. If uv pip install fails after uv venv succeeds, the next run skips that environment and leaves its packages missing. Record success only after installation finishes, or retry the install before reusing the environment.
Comment on lines
+96
to
+97
| - Redis: start your own on a port in YOUR range (`redis-server --port <BP+9> --save '' | ||
| --appendonly no &`) and point the app at it with `REFLEX_REDIS_URL`. Prod with Redis forks |
Contributor
There was a problem hiding this comment.
The suggested <BP+9> Redis port falls outside a verifier's assigned range. orchestration.md gives each verifier four backend ports. A verifier starting at 8600 would start Redis on 8609, inside another verifier's range. This can cause collisions and misleading test results. Choose an unused port within the assigned range, or reserve a separate Redis range.
| if not site.is_dir(): | ||
| print(f"not a directory: {site}", file=sys.stderr) | ||
| return 2 | ||
| diff = git(args.repo, "diff", "--name-only", f"{args.base}..{args.ref}", "--") |
Contributor
There was a problem hiding this comment.
With git diff --name-only, a detected rename lists only the new path. The old path never reaches the deleted-file check, so a wheel containing both the correct new file and a stale old file can receive PASS. Disable rename detection with --no-renames, or read both paths from rename records.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
19 issues found across 8 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".claude/skills/prerelease-test/SKILL.md">
<violation number="1" location=".claude/skills/prerelease-test/SKILL.md:98">
P1: This bootstrap example cannot run as written: the inline comment ends the environment-assignment continuation, and the relative script path is unavailable from the required neutral directory. Keep the assignment continued and invoke the script through `<checkout>`.</violation>
<violation number="2" location=".claude/skills/prerelease-test/SKILL.md:107">
P2: The smoke test does not select the venv it just built, so `reflex` may be unavailable or resolve to the wrong installation. Invoke the new venv explicitly and run the driver with the dedicated driver interpreter.</violation>
</file>
<file name=".claude/skills/prerelease-test/scripts/check_wheel_sources.py">
<violation number="1" location=".claude/skills/prerelease-test/scripts/check_wheel_sources.py:100">
P2: Git rename detection makes this diff list only the destination path, so a stale installed file at the old path is never checked. Disable rename detection so renames are checked as a deletion and an addition.</violation>
<violation number="2" location=".claude/skills/prerelease-test/scripts/check_wheel_sources.py:105">
P2: A site-packages without `reflex` still exits 0 and prints `PASS`, because every `reflex/` change is only reported as `not installed`. The package under test should be required, so fail when `reflex` is missing from `dists`.</violation>
</file>
<file name=".claude/skills/prerelease-test/references/orchestration.md">
<violation number="1" location=".claude/skills/prerelease-test/references/orchestration.md:149">
P3: This adds a third-agent exception that contradicts the SKILL.md concurrency guidance ('more just makes everything slow and flaky') without evidence. Either cite a measurement or keep the two-agent limit and drop the exception.</violation>
<violation number="2" location=".claude/skills/prerelease-test/references/orchestration.md:165">
P2: The commit loop's per-file 2 MB skip does not enforce the documented budget of about 2 MB per cluster and 10 MB per pass. Logs and screenshots under 2 MB will still be committed on every pass, which is the bloat SKILL.md warns about. Have the loop commit only the trimmed evidence files, or check the total size per cluster and per pass before committing.</violation>
<violation number="3" location=".claude/skills/prerelease-test/references/orchestration.md:166">
P2: `git reset` without a path unstages the entire index, not just the oversized file, so one large artifact can prevent other reports from being committed. Reset only the oversized path.</violation>
</file>
<file name=".claude/skills/prerelease-test/scripts/bootstrap_envs.sh">
<violation number="1" location=".claude/skills/prerelease-test/scripts/bootstrap_envs.sh:33">
P2: Relative scratch paths are resolved twice after `cd "$SB"`, so `bootstrap_envs.sh scratch` builds under `scratch/scratch` and prints an unusable `SB`. Canonicalize `SB` before changing directories.</violation>
<violation number="2" location=".claude/skills/prerelease-test/scripts/bootstrap_envs.sh:39">
P2: A venv whose `uv pip install` failed still has `bin/python`, so the next run reuses it without reinstalling. Mark completion only after the install succeeds, for example with a marker file, or build into a temp directory and rename it.</violation>
<violation number="3" location=".claude/skills/prerelease-test/scripts/bootstrap_envs.sh:44">
P2: `--no-config` does not guarantee PyPI-only resolution, because uv index environment variables still override or add indexes. Pin the install to PyPI and clear additional index settings before building the campaign environments.</violation>
<violation number="4" location=".claude/skills/prerelease-test/scripts/bootstrap_envs.sh:57">
P2: The Playwright fallback installs Chromium somewhere the driver never uses; when `/opt/pw-browsers/chromium` is absent, bootstrap can pass and every browser run still fails. Use the installed path or fail when the fixed executable is missing.</violation>
<violation number="5" location=".claude/skills/prerelease-test/scripts/bootstrap_envs.sh:63">
P2: A relative `ENT_WHEEL` is checked after the script changes into the scratch directory, so a valid wheel path from the caller's directory is rejected. Resolve the wheel path before `cd "$SB"`.</violation>
<violation number="6" location=".claude/skills/prerelease-test/scripts/bootstrap_envs.sh:65">
P2: `prev-ent` ignores `PREV_EXTRAS`, unlike `prev`. For a 0.9.x previous release the enterprise baseline will lack greenlet. Pass `${PREV_EXTRAS:-}` here too.</violation>
<violation number="7" location=".claude/skills/prerelease-test/scripts/bootstrap_envs.sh:76">
P3: The printed `export SB` command is not shell-escaped, so scratch paths containing spaces or shell metacharacters cannot be reused. Print it with `printf '%q'`.</violation>
</file>
<file name=".claude/skills/prerelease-test/references/reporting.md">
<violation number="1" location=".claude/skills/prerelease-test/references/reporting.md:29">
P3: The re-verification table is missing its Markdown separator row, so generated `FINDINGS.md` reports will not render this required data as a table. Add the separator immediately after the header.</violation>
</file>
<file name=".claude/skills/prerelease-test/references/agent-brief.md">
<violation number="1" location=".claude/skills/prerelease-test/references/agent-brief.md:3">
P2: The brief’s new run-directory path disagrees with the orchestration prompt, which still tells agents to read `<SB>/AGENT_BRIEF.md`. Agents can therefore start without the isolation rules; update the prompt to pass the run-directory brief path, or create the documented scratchpad copy.</violation>
<violation number="2" location=".claude/skills/prerelease-test/references/agent-brief.md:57">
P2: The documented bootstrap script path does not exist from the campaign’s documented working locations. Point agents to `.claude/skills/prerelease-test/scripts/bootstrap_envs.sh` so the required environment setup is runnable.</violation>
<violation number="3" location=".claude/skills/prerelease-test/references/agent-brief.md:96">
P2: The Redis example violates the reserved-port rule for verifier agents: their range is only four ports wide, so `BP+9` is outside it. Reserve and pass an explicit Redis port in each cluster’s port map instead of deriving it this way.</violation>
<violation number="4" location=".claude/skills/prerelease-test/references/agent-brief.md:147">
P2: The new final-message template does not match the Workflow schema that consumes agent results. It uses `ARTIFACTS` and string regression values, while the schema requires `artifacts_dir` and a boolean `regression`; align the template and parser before running the campaign.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
Comment on lines
+98
to
+99
| ENT_SPEC='reflex-enterprise[mcp]==<ent>' # or ENT_WHEEL=<user-supplied offline wheel> | ||
| .claude/skills/prerelease-test/scripts/bootstrap_envs.sh "$SB" --enterprise |
Contributor
There was a problem hiding this comment.
P1: This bootstrap example cannot run as written: the inline comment ends the environment-assignment continuation, and the relative script path is unavailable from the required neutral directory. Keep the assignment continued and invoke the script through <checkout>.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/SKILL.md, line 98:
<comment>This bootstrap example cannot run as written: the inline comment ends the environment-assignment continuation, and the relative script path is unavailable from the required neutral directory. Keep the assignment continued and invoke the script through `<checkout>`.</comment>
<file context>
@@ -63,17 +92,39 @@ is what lets you test it as a user rather than as a checklist.
+Build the campaign's venvs first, from a neutral directory (never from the checkout):
+
+ NEW_VERSION=<new> PREV_VERSION=<prev> CTRL_VERSIONS="<positive-control versions>" \
+ ENT_SPEC='reflex-enterprise[mcp]==<ent>' # or ENT_WHEEL=<user-supplied offline wheel>
+ .claude/skills/prerelease-test/scripts/bootstrap_envs.sh "$SB" --enterprise
+
</file context>
Suggested change
| ENT_SPEC='reflex-enterprise[mcp]==<ent>' # or ENT_WHEEL=<user-supplied offline wheel> | |
| .claude/skills/prerelease-test/scripts/bootstrap_envs.sh "$SB" --enterprise | |
| ENT_SPEC='reflex-enterprise[mcp]==<ent>' \ | |
| <checkout>/.claude/skills/prerelease-test/scripts/bootstrap_envs.sh "$SB" --enterprise |
Comment on lines
+107
to
+108
| Before any fan-out: `reflex init --template blank`, `reflex run`, then drive it in Chromium with | ||
| `.claude/skills/prerelease-test/scripts/drive_app.py`, in dev and in prod. This shakes out |
Contributor
There was a problem hiding this comment.
P2: The smoke test does not select the venv it just built, so reflex may be unavailable or resolve to the wrong installation. Invoke the new venv explicitly and run the driver with the dedicated driver interpreter.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/SKILL.md, line 107:
<comment>The smoke test does not select the venv it just built, so `reflex` may be unavailable or resolve to the wrong installation. Invoke the new venv explicitly and run the driver with the dedicated driver interpreter.</comment>
<file context>
@@ -63,17 +92,39 @@ is what lets you test it as a user rather than as a checklist.
+new alphas, and the "baseline" quietly runs half the code under test. Check the printed versions
+before trusting a baseline.
+
+Before any fan-out: `reflex init --template blank`, `reflex run`, then drive it in Chromium with
+`.claude/skills/prerelease-test/scripts/drive_app.py`, in dev and in prod. This shakes out
+environment problems (proxy, bun installs, browser path) once, in a context where you can debug
</file context>
Suggested change
| Before any fan-out: `reflex init --template blank`, `reflex run`, then drive it in Chromium with | |
| `.claude/skills/prerelease-test/scripts/drive_app.py`, in dev and in prod. This shakes out | |
| Before any fan-out: `$SB/envs/new/bin/reflex init --template blank`, `$SB/envs/new/bin/reflex run`, then | |
| drive it in Chromium with `$SB/envs/driver/bin/python <checkout>/.claude/skills/prerelease-test/scripts/drive_app.py`, in dev and in prod. This shakes out |
| if not site.is_dir(): | ||
| print(f"not a directory: {site}", file=sys.stderr) | ||
| return 2 | ||
| diff = git(args.repo, "diff", "--name-only", f"{args.base}..{args.ref}", "--") |
Contributor
There was a problem hiding this comment.
P2: Git rename detection makes this diff list only the destination path, so a stale installed file at the old path is never checked. Disable rename detection so renames are checked as a deletion and an addition.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/scripts/check_wheel_sources.py, line 100:
<comment>Git rename detection makes this diff list only the destination path, so a stale installed file at the old path is never checked. Disable rename detection so renames are checked as a deletion and an addition.</comment>
<file context>
@@ -0,0 +1,146 @@
+ if not site.is_dir():
+ print(f"not a directory: {site}", file=sys.stderr)
+ return 2
+ diff = git(args.repo, "diff", "--name-only", f"{args.base}..{args.ref}", "--")
+ if diff.returncode != 0:
+ print(diff.stderr.decode().strip(), file=sys.stderr)
</file context>
Suggested change
| diff = git(args.repo, "diff", "--name-only", f"{args.base}..{args.ref}", "--") | |
| diff = git(args.repo, "diff", "--no-renames", "--name-only", f"{args.base}..{args.ref}", "--") |
| A full campaign runs for hours, and usage limits, spend limits and container restarts all happen | ||
| mid-run. Plan for them: | ||
|
|
||
| - **Commit continuously.** Agents cannot run git, so run a background loop that commits and pushes |
Contributor
There was a problem hiding this comment.
P2: The commit loop's per-file 2 MB skip does not enforce the documented budget of about 2 MB per cluster and 10 MB per pass. Logs and screenshots under 2 MB will still be committed on every pass, which is the bloat SKILL.md warns about. Have the loop commit only the trimmed evidence files, or check the total size per cluster and per pass before committing.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/references/orchestration.md, line 165:
<comment>The commit loop's per-file 2 MB skip does not enforce the documented budget of about 2 MB per cluster and 10 MB per pass. Logs and screenshots under 2 MB will still be committed on every pass, which is the bloat SKILL.md warns about. Have the loop commit only the trimmed evidence files, or check the total size per cluster and per pass before committing.</comment>
<file context>
@@ -146,9 +146,32 @@ verifier's range, which is the collision the map exists to prevent.
+A full campaign runs for hours, and usage limits, spend limits and container restarts all happen
+mid-run. Plan for them:
+
+- **Commit continuously.** Agents cannot run git, so run a background loop that commits and pushes
+ the run directory every ~10 minutes, skipping any file over 2 MB (`git reset` it from the index).
+ Start it with `setsid nohup <loop> &` so it outlives the tool call, and stop it with a sentinel
</file context>
|
|
||
| mk() { # name, version-that-decides-resolution, pip args... | ||
| local name="$1" ver="$2"; shift 2 | ||
| [ -x "$SB/envs/$name/bin/python" ] && return 0 |
Contributor
There was a problem hiding this comment.
P2: A venv whose uv pip install failed still has bin/python, so the next run reuses it without reinstalling. Mark completion only after the install succeeds, for example with a marker file, or build into a temp directory and rename it.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/scripts/bootstrap_envs.sh, line 39:
<comment>A venv whose `uv pip install` failed still has `bin/python`, so the next run reuses it without reinstalling. Mark completion only after the install succeeds, for example with a marker file, or build into a temp directory and rename it.</comment>
<file context>
@@ -0,0 +1,76 @@
+
+mk() { # name, version-that-decides-resolution, pip args...
+ local name="$1" ver="$2"; shift 2
+ [ -x "$SB/envs/$name/bin/python" ] && return 0
+ local opts=()
+ is_pre "$ver" && opts+=(--prerelease=allow)
</file context>
Comment on lines
+96
to
+97
| - Redis: start your own on a port in YOUR range (`redis-server --port <BP+9> --save '' | ||
| --appendonly no &`) and point the app at it with `REFLEX_REDIS_URL`. Prod with Redis forks |
Contributor
There was a problem hiding this comment.
P2: The Redis example violates the reserved-port rule for verifier agents: their range is only four ports wide, so BP+9 is outside it. Reserve and pass an explicit Redis port in each cluster’s port map instead of deriving it this way.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/references/agent-brief.md, line 96:
<comment>The Redis example violates the reserved-port rule for verifier agents: their range is only four ports wide, so `BP+9` is outside it. Reserve and pass an explicit Redis port in each cluster’s port map instead of deriving it this way.</comment>
<file context>
@@ -62,10 +84,20 @@ and hunt for anomalies. You REPORT issues; you never fix framework code.
+ first run does a bun install (1–2 min); poll the frontend URL until it returns 200 for up to ~6
+ minutes before concluding failure. A dev server's first page render can lag the 200: wait for
+ text the page must show before reading it.
+- Redis: start your own on a port in YOUR range (`redis-server --port <BP+9> --save ''
+ --appendonly no &`) and point the app at it with `REFLEX_REDIS_URL`. Prod with Redis forks
+ several granian workers (capped at 8 by default; `GRANIAN_WORKERS` overrides).
</file context>
Suggested change
| - Redis: start your own on a port in YOUR range (`redis-server --port <BP+9> --save '' | |
| --appendonly no &`) and point the app at it with `REFLEX_REDIS_URL`. Prod with Redis forks | |
| - Redis: start your own on an explicitly assigned port in YOUR range (`<REDIS_PORT>`): | |
| `redis-server --port <REDIS_PORT> --save '' --appendonly no &` and point the app at it with `REFLEX_REDIS_URL`. Prod with Redis forks |
|
|
||
| Copy this into the campaign scratchpad (e.g. `$SB/AGENT_BRIEF.md`), fill the `<...>` placeholders, | ||
| and tell every test agent to read it first. It carries the rules that keep results trustworthy and | ||
| Copy this into the run directory (`prerelease-testing/runs/<run>/AGENT_BRIEF.md` on |
Contributor
There was a problem hiding this comment.
P2: The brief’s new run-directory path disagrees with the orchestration prompt, which still tells agents to read <SB>/AGENT_BRIEF.md. Agents can therefore start without the isolation rules; update the prompt to pass the run-directory brief path, or create the documented scratchpad copy.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/references/agent-brief.md, line 3:
<comment>The brief’s new run-directory path disagrees with the orchestration prompt, which still tells agents to read `<SB>/AGENT_BRIEF.md`. Agents can therefore start without the isolation rules; update the prompt to pass the run-directory brief path, or create the documented scratchpad copy.</comment>
<file context>
@@ -1,7 +1,8 @@
-Copy this into the campaign scratchpad (e.g. `$SB/AGENT_BRIEF.md`), fill the `<...>` placeholders,
-and tell every test agent to read it first. It carries the rules that keep results trustworthy and
+Copy this into the run directory (`prerelease-testing/runs/<run>/AGENT_BRIEF.md` on
+`testing/prerelease`), fill the `<...>` placeholders, and tell every test agent to read it first,
+after `CAMPAIGN_STATE.md`. It carries the rules that keep results trustworthy and
</file context>
|
|
||
| - Two agents at a time is right for a 4-CPU box; the workflow's own concurrency cap handles the | ||
| rest of the queue. | ||
| rest of the queue. A third agent fits when one of them runs no app server (a CLI or packaging |
Contributor
There was a problem hiding this comment.
P3: This adds a third-agent exception that contradicts the SKILL.md concurrency guidance ('more just makes everything slow and flaky') without evidence. Either cite a measurement or keep the two-agent limit and drop the exception.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/references/orchestration.md, line 149:
<comment>This adds a third-agent exception that contradicts the SKILL.md concurrency guidance ('more just makes everything slow and flaky') without evidence. Either cite a measurement or keep the two-agent limit and drop the exception.</comment>
<file context>
@@ -146,9 +146,32 @@ verifier's range, which is the collision the map exists to prevent.
- Two agents at a time is right for a 4-CPU box; the workflow's own concurrency cap handles the
- rest of the queue.
+ rest of the queue. A third agent fits when one of them runs no app server (a CLI or packaging
+ cluster, a verifier working through Python-level probes).
- Between phases, read the results before launching the next one.
</file context>
| - ... | ||
|
|
||
| ## Re-verification table | ||
| | registry id | what | positive control (version, result) | new result | cluster | |
Contributor
There was a problem hiding this comment.
P3: The re-verification table is missing its Markdown separator row, so generated FINDINGS.md reports will not render this required data as a table. Add the separator immediately after the header.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/references/reporting.md, line 29:
<comment>The re-verification table is missing its Markdown separator row, so generated `FINDINGS.md` reports will not render this required data as a table. Add the separator immediately after the header.</comment>
<file context>
@@ -25,9 +25,15 @@ Index:
- ...
+## Re-verification table
+| registry id | what | positive control (version, result) | new result | cluster |
+<Every REGISTRY.md item re-run this pass: fixed / still broken / changed, with the control that
+proves the check can fail.>
</file context>
Suggested change
| | registry id | what | positive control (version, result) | new result | cluster | | |
| | registry id | what | positive control (version, result) | new result | cluster | | |
| |---|---|---|---|---| |
| keep = [n for n in names if n.startswith("reflex")] + ["sqlalchemy", "greenlet", "pydantic"] | ||
| print(" ".join(f"{n}={m.version(n)}" for n in keep if n in names))')" | ||
| done | ||
| echo "export SB=$SB" |
Contributor
There was a problem hiding this comment.
P3: The printed export SB command is not shell-escaped, so scratch paths containing spaces or shell metacharacters cannot be reused. Print it with printf '%q'.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/prerelease-test/scripts/bootstrap_envs.sh, line 76:
<comment>The printed `export SB` command is not shell-escaped, so scratch paths containing spaces or shell metacharacters cannot be reused. Print it with `printf '%q'`.</comment>
<file context>
@@ -0,0 +1,76 @@
+keep = [n for n in names if n.startswith("reflex")] + ["sqlalchemy", "greenlet", "pydantic"]
+print(" ".join(f"{n}={m.version(n)}" for n in keep if n in names))')"
+done
+echo "export SB=$SB"
</file context>
Suggested change
| echo "export SB=$SB" | |
| printf 'export SB=%q\n' "$SB" |
Merging this PR will not alter performance1 warning
Comparing 
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
All Submissions:
Type of change
.claude/skills/prerelease-test/, no framework or package source.Changes To Core Features:
Description
Updates the
prerelease-testskill with what the 0.10.1a1 pre-release pass taught, so the next pass starts from a working setup instead of rediscovering it. Process only: no campaign results, findings or artifacts are included (those live on thetesting/prereleasebranch).Workflow
testing/prereleasebranch:REGISTRY.md, reusablefixtures/, aruns/<date>-v<version>/directory per pass, and a close-out step intohistory/.CAMPAIGN_STATE.mdand a filledAGENT_BRIEF.md, plus size limits for run bulk (about 2 MB per cluster, 10 MB per pass). The previous archive grew to 320 MB.r/pre-*branches share a date, test the one whose head carries the release tag.New and changed scripts
scripts/bootstrap_envs.sh(new) builds the PyPI-only venvs (new,prev,ctrl-X,driver,new-ent,prev-ent) and prints every reflex package each one resolved. It fixes a baseline problem seen this pass:--prerelease=allow, the previous release's>=sibling floors resolved to the new alphas, so the "0.10.0 baseline" ran 0.10.1a1 code, moment and lucide among them. A finalprevnow resolves without pre-releases.scripts/check_wheel_sources.py(new) compares every source file changed between two tags with the installed copy. It replaces an ad-hoc shell loop.scripts/drive_app.pynow reports a page that rendered no text as an anomaly. A slow first dev load used to come back as "clean" with an empty body.Checks added, each one surfaced a real finding this pass
reflexwheel versus a stockpip install -U reflex..pyistubs (isolated build env plus unpinned hook dependencies), with a throwaway uv cache.X-Forwarded-Forover the REST API versus the websocket.Agent brief
pkill -fkills its own shell.--no-configand--only-binaryinstalls.Orchestration: a new "Surviving interruptions" section, written after this pass hit two usage limits and a spend limit. It covers periodic commits of the run directory, resuming agents with
SendMessageinstead of respawning them, and what to do when the configured model is unavailable.Test plan
check_wheel_sources.pyagainst the published 0.10.1a1 install: 77 identical, and only the 3 unpublished workspace packages reported as not installed. Against a tampered copy ofreflex/app.py:DIFFERS, exit 1.bootstrap_envs.sh NEW=0.10.1a1 PREV=0.10.0in a fresh scratch dir:prevresolved every sibling to its final (core/code/moment 0.10.0, lucide 1.1.0, hosting-cli 0.2.0, build-sdk 0.1.0). The old script gave code/moment/lucide/hosting-cli alphas.drive_app.pyagainst a static page with an empty#root:EMPTY PAGEreported. A page with text: no empty-page report.ruff checkandruff formatare clean on the changed scripts;bash -npasses on the bootstrap script.Generated by Claude Code