Skip to content

Skip Grok in incompatible agent e2e tests #378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
andy-xu-db merged 1 commit into from
Aug 25, 2026
Merged

Skip Grok in incompatible agent e2e tests #378

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions tests/test_e2e.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,8 @@ class TestCodexLaunch:
# ("invalid response from an upstream server"), so the launch fails after codex-cli
# exhausts its five reconnects. Nothing ucode writes can fix it.
"gpt-5-3-codex",
# Bedrock Grok rejects the Responses tool schema Codex sends (missing nested `function`).
"grok",
)

def _codex_models(self, e2e_state: dict) -> list[str]:
Expand Down Expand Up @@ -914,6 +916,8 @@ class TestCopilotLaunch:
"gpt-5-5",
# gpt-5.6 models similarly reject /chat/completions with 404.
"gpt-5-6",
# Bedrock Grok rejects the gateway's llm/v1/chat task type.
"grok",
)

def _all_models(self, e2e_state: dict) -> list[tuple[str, str]]:
Expand Down Expand Up @@ -977,9 +981,12 @@ class TestPiLaunch:
test exercises each one end-to-end through the validation path.
"""

# The CI project's upstream OpenAI account cannot serve this snapshot in
# its geography. Codex's e2e excludes the same otherwise-discoverable model.
INCOMPATIBLE_MODEL_FRAGMENTS = ("gpt-5-3-codex",)
INCOMPATIBLE_MODEL_FRAGMENTS = (
# The CI project's upstream OpenAI account cannot serve this snapshot in its geography.
"gpt-5-3-codex",
# Bedrock Grok currently rejects Pi's OpenAI request with HTTP 400.
"grok",
)

def _all_models(self, e2e_state: dict) -> list[tuple[str, str]]:
out: list[tuple[str, str]] = []
Expand Down
Loading