TEST: Preserve Crescendo success for leaky refusals

[Roman Lutz (romanlutz)]

Description

A target response can include refusal language while still revealing enough harmful information to achieve the attack objective. Refusal detection must therefore not override the configured objective scorer's verdict.

This adds a regression contract for Crescendo's no-backtrack path: when refusal detection is true and the objective score is true, the attack returns AttackOutcome.SUCCESS with the objective-achieved reason. The test also preserves the refusal and backtracking state expectations. There is no production-code change.

Tests and Documentation

• uv run pytest tests\unit\executor\attack\multi_turn\test_crescendo.py -q (84 passed)
• uv run ruff check pyrit\executor\attack\multi_turn\crescendo.py tests\unit\executor\attack\multi_turn\test_crescendo.py
• uv run ruff format --check pyrit\executor\attack\multi_turn\crescendo.py tests\unit\executor\attack\multi_turn\test_crescendo.py
• uv run ty check pyrit\executor\attack\multi_turn\crescendo.py tests\unit\executor\attack\multi_turn\test_crescendo.py
• uv run python -m compileall -q pyrit\executor\attack\multi_turn\crescendo.py tests\unit\executor\attack\multi_turn\test_crescendo.py
• git diff --check

Documentation not updated because this PR codifies the existing documented scorer-driven behavior. JupyText was not run because no documentation or notebook files changed.

[Justin Song (jsong468)]

A bit confused on this behavior and whether this ever does/should happen. If a refusal is detected and backtracking is available, then the last turn will get erased and we backtrack, which in my head means the attack was not successful. But, if no more backtracking is available, then the attack could be marked successful, meaning the outcome could be variable depending on an axis that theoretically shouldn't have impact on ultimate outcome.

Basically, shouldn't "A target response can include refusal language while still revealing enough harmful information" be graded as True for objective achieved no matter if backtracking is allowed or not (since it's still harmful and I, a red teamer, would want to know about that)? If this is truly something that does happen often in practice, I imagine it might require a structural change (objective scorer kicks in no matter what).

[Roman Lutz (romanlutz)]

A bit confused on this behavior and whether this ever does/should happen. If a refusal is detected and backtracking is available, then the last turn will get erased and we backtrack, which in my head means the attack was not successful. But, if no more backtracking is available, then the attack could be marked successful, meaning the outcome could be variable depending on an axis that theoretically shouldn't have impact on ultimate outcome.

Basically, shouldn't "A target response can include refusal language while still revealing enough harmful information" be graded as True for objective achieved no matter if backtracking is allowed or not (since it's still harmful and I, a red teamer, would want to know about that)? If this is truly something that does happen often in practice, I imagine it might require a structural change (objective scorer kicks in no matter what).

It's an interesting problem. It used to be pretty easy to detect refusal and it was actually always (99.9% of the time) a refusal. Refusals come in different shapes now. Sometimes it's an outright refusal, sometimes there's a long explanation of why the model refuses just to then pivot and help out anyway 🤷🏻 Crescendo has not changed in the meantime, though. Maybe we should just run refusal and other scorers in parallel and only backtrack if "refusal AND not success" while continuing if "not refusal AND not success" and in the success case we end execution? That would change the logic somewhat, though.

This case actually came up because GHCP told me the existing behavior (in the last iteration if refusal AND success --> success) is a bug and wanted to fix it. I decided it's not a bug (which is not 100% clear either) and wanted to put in a test case to assert this behavior. But we can bring this up in standup.

[Justin Song (jsong468)]

A bit confused on this behavior and whether this ever does/should happen. If a refusal is detected and backtracking is available, then the last turn will get erased and we backtrack, which in my head means the attack was not successful. But, if no more backtracking is available, then the attack could be marked successful, meaning the outcome could be variable depending on an axis that theoretically shouldn't have impact on ultimate outcome.
Basically, shouldn't "A target response can include refusal language while still revealing enough harmful information" be graded as True for objective achieved no matter if backtracking is allowed or not (since it's still harmful and I, a red teamer, would want to know about that)? If this is truly something that does happen often in practice, I imagine it might require a structural change (objective scorer kicks in no matter what).

It's an interesting problem. It used to be pretty easy to detect refusal and it was actually always (99.9% of the time) a refusal. Refusals come in different shapes now. Sometimes it's an outright refusal, sometimes there's a long explanation of why the model refuses just to then pivot and help out anyway 🤷🏻 Crescendo has not changed in the meantime, though. Maybe we should just run refusal and other scorers in parallel and only backtrack if "refusal AND not success" while continuing if "not refusal AND not success" and in the success case we end execution? That would change the logic somewhat, though.

This case actually came up because GHCP told me the existing behavior (in the last iteration if refusal AND success --> success) is a bug and wanted to fix it. I decided it's not a bug (which is not 100% clear either) and wanted to put in a test case to assert this behavior. But we can bring this up in standup.

Yea..makes sense! I would agree with having a "if not refusal AND not success" check if refusals and failure have become less synonymous! For now, test as is, is okay and approving!