Skip to content

Fix false approvals when adversarial review turns fail#465

Open
CapCap wants to merge 1 commit into
openai:mainfrom
CapCap:fix/adversarial-review-fail-closed
Open

Fix false approvals when adversarial review turns fail#465
CapCap wants to merge 1 commit into
openai:mainfrom
CapCap:fix/adversarial-review-fail-closed

Conversation

@CapCap

@CapCap CapCap commented Jul 10, 2026

Copy link
Copy Markdown

Summary

  • fail closed when an adversarial review turn exits nonzero, even if an interim assistant message is valid structured review JSON
  • render the runtime failure and discard interim review text from stdout
  • persist the runtime error in the tracked review payload
  • add an integration regression for an interim approve followed by a failed turn

Reproduction

A real adversarial review emitted schema-valid progress messages with verdict: approve, then failed with No tool output found for custom tool call .... The companion marked the job failed internally but parsed and rendered the last progress message as an approval.

Verification

  • npm run build
  • npm run check-version
  • claude plugin validate plugins/codex
  • node --test --test-reporter=dot tests/*.test.mjs
  • reran the original review input: collaboration tracks ran and the completed turn returned needs-attention with six findings

@CapCap CapCap requested a review from a team July 10, 2026 05:26

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ab9a971c30

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

parseError:
fallback.failureMessage ??
`Codex run failed with status ${fallback.status} before returning a final structured message.`,
rawOutput: rawOutput ?? "",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear discarded review text from JSON payloads

When a failed adversarial turn first emits a schema-valid progress message and the caller uses adversarial-review --json or reads the stored JSON result, this new failure path still exposes that discarded message as rawOutput (and executeReviewRun also serializes the same finalMessage as codex.stdout). In that context stdout for the failed command can still contain "verdict":"approve", so machine consumers that read the raw final-output fields can continue to record a false approval even though the rendered output hides it. Clear these raw-output fields on nonzero statuses or move the interim text to a diagnostic-only field.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant