Skip to content

fix(review-pr): fail closed on incomplete sub-agent output - #82

Open
Sayt-0 wants to merge 1 commit into
mainfrom
fix/review-pr-fail-closed-subagent-output
Open

fix(review-pr): fail closed on incomplete sub-agent output#82
Sayt-0 wants to merge 1 commit into
mainfrom
fix/review-pr-fail-closed-subagent-output

Conversation

@Sayt-0

@Sayt-0 Sayt-0 commented Aug 14, 2026

Copy link
Copy Markdown
Member

Summary

  • harden drafter and verifier schemas against empty values, placeholders, and invalid line numbers
  • treat failed, refused, malformed, or partial sub-agent responses as incomplete reviews
  • aggregate multi-chunk drafter results fail-closed at every finding count
  • pair verifier verdicts one-to-one using deterministic finding IDs
  • prevent incomplete reviews from carrying the completed-run assessment marker

Context

A structured-output child response terminated with stop_reason: refusal was returned by transfer_task as successful partial JSON.

This change provides action-side containment. It does not change Docker Agent runtime semantics: refusal, structured-output length termination, and native schema validation still require a separate upstream fix.

Observed run: https://github.com/docker/sandboxes/actions/runs/31797481426

Expectation mapping

Expectation Status Handling
Refused structured-output child makes transfer_task fail upstream required tool errors and malformed partial responses now produce an incomplete review
Partial child transcript remains available upstream required no runtime transcript behavior changed
Tool error exposes child and finish reason upstream required action fallback reports the affected chunk as incomplete
Completed native output is schema-validated upstream required action schemas are hardened against invalid semantic values
Structured max_tokens termination fails equivalently upstream required malformed or partial action responses fail closed
line >= 1 and meaningful strings implemented both drafter and verifier schemas enforce the constraints
Verdicts correspond one-to-one with findings implemented deterministic finding_id pairing plus exact file and line cross-check
One failed chunk prevents an approving review implemented incomplete state overrides all confidence-based assessment labels

Validation

Check Result
Unit tests pass, 857 tests
Typecheck pass
Biome pass
Build pass
Actionlint pass
YAML parsing pass
Docker Agent config dry-run pass
Integration assertions 15/15 pass; local Vitest teardown timeout also reproduces on origin/main
Live model evaluation skipped, requires provider credentials

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant