Skip to content

test(e2e): add the afterModel continuation fixture - #1617

Merged
tombeckenham merged 1 commit into
mainfrom
fix/e2e-after-model-continuation
Oct 4, 2026
Merged

tombeckenham merged 1 commit into
mainfrom
fix/e2e-after-model-continuation

Conversation

@tombeckenham

@tombeckenham tombeckenham commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

The test resolves a typed generic interrupt at afterModel stays open after the test resolves the review. The resume starts a second model call. The aimock fixture matches only sequenceIndex 0. The second call returns 404 No fixture matched. This PR adds a sequenceIndex 1 fixture with the same content, AFTER_MODEL_CONTENT.

🎯 Changes

The generic afterModel interrupt resume calls the model a second time. testing/e2e/fixtures/middleware-test/generic-after-model.json had no fixture for that call. The new entry uses the same user message and the same response text. No published package changes, so there is no changeset. The change is not user-facing, so docs/ stays as it is.

✅ Checklist

  • I have followed the steps in the Contributing guide.
  • I have tested code changes locally with pnpm run test:pr, or these tests do not apply to this pull request.
  • I fully understand the code in this pull request, including any code generated with AI assistance.
  • Docs: I updated docs/ for this change, or this change is not user-facing.
  • Changeset: I added a changeset (pnpm changeset), or this PR does not change a published package.

🚀 Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

Root cause

Issue. The e2e test resolves a typed generic interrupt at afterModel fails. The interrupt count stays at 1 after resolve. The input is the generic afterModel scenario with a resolved review.

Cause. onInterruptResolution returns continue for this scenario. The chat loop then calls the model again. generic-after-model.json matches only sequenceIndex 0. aimock returns 404 for the next call.

Fix. This PR adds a sequenceIndex 1 fixture with the same text. The second call matches. The test can finish.

Possible alternatives

  • Skip the second model call. The chat loop calls the model again when toolResume is continue. The before-tools fixture already has that second call. A skip in product code changes the agent loop. This PR does not do that.
  • Use turnIndex. turnIndex counts assistant messages. It does not use the server sequence counter. X-Test-Id already gives this test its own sequence. The two calls are sequence 0 and sequence 1. This PR keeps sequenceIndex.

Testing

Commands run. I ran one agent-written aimock command on clean main 7216c7b58 and on this branch b8502b5f2. Main exited 1. This branch exited 0. I did not run pnpm test:pr. I did not run Playwright. Vite in the earlier worktree failed to resolve @tanstack/ai, so the browser suite did not start.

The command loads generic-after-model.json into aimock and sends two POST /v1/chat/completions requests. Both requests use the user message [generic-after-model] run test. Both responses must be status 200 and contain AFTER_MODEL_CONTENT. Then the command exits 0.

Clean main:

first 200
{"id":"chatcmpl-ueMlmHR_t6Rld19H","object":"chat.completion","created":1791113662,"model":"gpt-4o","choices":[{"index":0,"message":{"role":"assistant","content":"AFTER_MODEL_CONTENT","refusal":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":8,"completion_tokens":5,"total_tokens":13}}
second 404
{"error":{"message":"No fixture matched","type":"invalid_request_error","param":null,"code":"no_fixture_match"}}
MAIN_EXIT=1

This branch:

first 200
{"id":"chatcmpl-8GWfeTQgnBPyFxhK","object":"chat.completion","created":1791113662,"model":"gpt-4o","choices":[{"index":0,"message":{"role":"assistant","content":"AFTER_MODEL_CONTENT","refusal":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":8,"completion_tokens":5,"total_tokens":13}}
second 200
{"id":"chatcmpl-QRMn44RywTe-7UYz","object":"chat.completion","created":1791113662,"model":"gpt-4o","choices":[{"index":0,"message":{"role":"assistant","content":"AFTER_MODEL_CONTENT","refusal":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":8,"completion_tokens":5,"total_tokens":13}}
PR_EXIT=0

A Playwright trace for this test showed the same 404 on the resume call.

Manual test.

  1. On main, start aimock with testing/e2e/fixtures/middleware-test/generic-after-model.json.
  2. Send two POST /v1/chat/completions requests with the user message [generic-after-model] run test.
  3. Make sure that the second response status is 404 and the code is no_fixture_match.
  4. On this branch, send the same two requests to aimock.
  5. Make sure that both responses are status 200 and the content is AFTER_MODEL_CONTENT.

How this PR makes testing easy. The new fixture entry is on the branch. CI runs tests/generic-middleware-interrupts.spec.ts.

Risk / rollback

Risk is low. The new entry matches only the second call for this user message. Revert this PR to remove it.

Summary by CodeRabbit

  • Tests
    • Added a test fixture for the response returned after the model-generated content.

Resolving that interrupt starts another model turn. The aimock fixture only matched the first turn, so the continuation returned 404 and the interrupt stayed open.
@changeset-bot

changeset-bot Bot commented Oct 4, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: b8502b5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: TanStack/ai/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 270ac8d0-fb45-486b-a14a-327e8cd4358f
📥 Commits

Reviewing files that changed from the base of the PR and between 7216c7b and b8502b5.

📒 Files selected for processing (1)
  • testing/e2e/fixtures/middleware-test/generic-after-model.json

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

The middleware test fixture now includes a second entry for the same user message. The entry matches sequenceIndex 1 and returns AFTER_MODEL_CONTENT.

Changes

Middleware test fixture

Layer / File(s) Summary
Add sequence index 1 response
testing/e2e/fixtures/middleware-test/generic-after-model.json
The fixture adds an entry for the same user message at sequenceIndex 1, with response content AFTER_MODEL_CONTENT.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~3 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to b8502

This adds the expected second response for the generic afterModel scenario without matching the other parameterized scenarios. No concrete merge-blocking risk remains.

Architecture Summary

Architecture risk: 🔵 Low · up to b8502

The change affects 1 system.

Changed systems: testing

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — testing (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in testing/e2e/fixtures/middleware-test/generic-after-model.json: Added a fixture entry for the same user message at sequenceIndex 1, with response content AFTER_MODEL_CONTENT.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely identifies the main change: adding an afterModel continuation fixture.
Description check ✅ Passed The description explains the cause, fix, release impact, and testing performed. It also states that the prescribed test suite and Playwright were not run.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@nx-cloud

nx-cloud Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

View your CI Pipeline Execution ↗ for commit b8502b5

Command Status Duration Result
nx affected --targets=test:sherif,test:knip,tes... ✅ Succeeded 58s View ↗
nx run-many --targets=build --exclude=examples/... ✅ Succeeded 3s View ↗

☁️ Nx Cloud last updated this comment at 2026-10-04 11:39:03 UTC

@pkg-pr-new

pkg-pr-new Bot commented Oct 4, 2026

Copy link
Copy Markdown

Open in StackBlitz

@tanstack/ai

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai@1617

@tanstack/ai-acp

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-acp@1617

@tanstack/ai-angular

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-angular@1617

@tanstack/ai-anthropic

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-anthropic@1617

@tanstack/ai-bedrock

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-bedrock@1617

@tanstack/ai-byteplus

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-byteplus@1617

@tanstack/ai-claude-code

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-claude-code@1617

@tanstack/ai-client

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-client@1617

@tanstack/ai-cloudflare

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-cloudflare@1617

@tanstack/ai-code-mode

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-code-mode@1617

@tanstack/ai-code-mode-snippets

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-code-mode-snippets@1617

@tanstack/ai-codex

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-codex@1617

@tanstack/ai-cohere

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-cohere@1617

@tanstack/ai-compaction

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-compaction@1617

@tanstack/ai-devtools-core

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-devtools-core@1617

@tanstack/ai-durable-stream

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-durable-stream@1617

@tanstack/ai-elevenlabs

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-elevenlabs@1617

@tanstack/ai-event-client

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-event-client@1617

@tanstack/ai-fal

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-fal@1617

@tanstack/ai-gemini

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-gemini@1617

@tanstack/ai-grok

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-grok@1617

@tanstack/ai-grok-build

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-grok-build@1617

@tanstack/ai-groq

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-groq@1617

@tanstack/ai-isolate-cloudflare

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-cloudflare@1617

@tanstack/ai-isolate-daytona

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-daytona@1617

@tanstack/ai-isolate-node

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-node@1617

@tanstack/ai-isolate-quickjs

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-quickjs@1617

@tanstack/ai-isolate-quickjs-bun

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-quickjs-bun@1617

@tanstack/ai-llmgateway

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-llmgateway@1617

@tanstack/ai-lovable

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-lovable@1617

@tanstack/ai-mcp

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-mcp@1617

@tanstack/ai-memory

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-memory@1617

@tanstack/ai-mistral

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-mistral@1617

@tanstack/ai-octane

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-octane@1617

@tanstack/ai-ollama

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-ollama@1617

@tanstack/ai-ollaya

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-ollaya@1617

@tanstack/ai-openai

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-openai@1617

@tanstack/ai-opencode

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-opencode@1617

@tanstack/ai-openrouter

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-openrouter@1617

@tanstack/ai-perplexity

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-perplexity@1617

@tanstack/ai-persistence

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-persistence@1617

@tanstack/ai-preact

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-preact@1617

@tanstack/ai-react

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-react@1617

@tanstack/ai-react-ui

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-react-ui@1617

@tanstack/ai-reactor

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-reactor@1617

@tanstack/ai-remix

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-remix@1617

@tanstack/ai-sandbox

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox@1617

@tanstack/ai-sandbox-blaxel

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-blaxel@1617

@tanstack/ai-sandbox-boxd

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-boxd@1617

@tanstack/ai-sandbox-cloudflare

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-cloudflare@1617

@tanstack/ai-sandbox-daytona

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-daytona@1617

@tanstack/ai-sandbox-docker

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-docker@1617

@tanstack/ai-sandbox-e2b

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-e2b@1617

@tanstack/ai-sandbox-local-process

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-local-process@1617

@tanstack/ai-sandbox-sprites

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-sprites@1617

@tanstack/ai-sandbox-upstash-box

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-upstash-box@1617

@tanstack/ai-sandbox-vercel

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-vercel@1617

@tanstack/ai-skills

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-skills@1617

@tanstack/ai-solid

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-solid@1617

@tanstack/ai-solid-ui

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-solid-ui@1617

@tanstack/ai-svelte

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-svelte@1617

@tanstack/ai-typesafe

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-typesafe@1617

@tanstack/ai-utils

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-utils@1617

@tanstack/ai-vercel-gateway

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vercel-gateway@1617

@tanstack/ai-vertex

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vertex@1617

@tanstack/ai-vue

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vue@1617

@tanstack/ai-vue-ui

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vue-ui@1617

@tanstack/ai-worldlabs

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-worldlabs@1617

@tanstack/openai-base

npm i https://pkg.pr.new/TanStack/ai/@tanstack/openai-base@1617

@tanstack/preact-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/preact-ai-devtools@1617

@tanstack/react-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/react-ai-devtools@1617

@tanstack/solid-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/solid-ai-devtools@1617

@tanstack/svelte-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/svelte-ai-devtools@1617

commit: b8502b5

@tombeckenham
tombeckenham enabled auto-merge (squash) October 4, 2026 11:43
@tombeckenham
tombeckenham merged commit f35fec4 into main Oct 4, 2026
11 checks passed
@tombeckenham
tombeckenham deleted the fix/e2e-after-model-continuation branch October 4, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant